How to identify Pandas' backend for Parquet

Cedric H. picture Cedric H. · Jun 8, 2018 · Viewed 17.4k times · Source

I understand that Pandas can read and write to and from Parquet files using different backends: pyarrow and fastparquet.

I have a Conda distribution with the Intel distribution and "it works": I can use pandas.DataFrame.to_parquet. However I do not have pyarrow installed so I guess that fastparquet is used (which I cannot find either).

Is there a way to identify which backend is used?

Answer

ANKIT CHOPADE picture ANKIT CHOPADE · Mar 25, 2019

Just execute these 2 commands in linux shell/bash

pip install pyarrow

pip install fastparquet