I understand that Pandas can read and write to and from Parquet files using different backends: pyarrow
and fastparquet
.
I have a Conda distribution with the Intel distribution and "it works": I can use pandas.DataFrame.to_parquet
. However I do not have pyarrow
installed so I guess that fastparquet
is used (which I cannot find either).
Is there a way to identify which backend is used?
Just execute these 2 commands in linux shell/bash
pip install pyarrow
pip install fastparquet