I installed the 64-bit windows version of python 3.7 by installing anaconda 5.3.0. Then I tried installing pyarrow ("conda install pyarrow").
Anaconda ended up DOWNGRADING python to 3.6. And things did not work so well thereafter (weird error messages). I ended up having to uninstall Anaconda and re-installing it (had to uninstall since it does not do repair or re-install if the program folder is not empty).
That leaves me python 3.7 but no pyarrow. Is there another package that will give me parquet support with python and pandas? Or is there a way to get pyarrow to work with python 3.7?
UPDATE: As per Omri374's suggestion, I tried
pip install pyarrow
That doesn't solve my separate anaconda rollback to python 3.6 problem (i.e., when doing "conda install pyarrow"), but it does install pyarrow. Note that it gives the following output though--trying to update pip produced a rollback to python 3.6, so I don't recommend it:
twisted 18.7.0 requires PyHamcrest>=1.9.0, which is not installed.
Installing collected packages: pyarrow
Successfully installed pyarrow-0.11.0
You are using pip version 10.0.1, however version 18.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
This is the pip version that comes with 64-bit win anaconda 5.3.0 (python 3.7):
>conda list pip
#Name Version Build Channel
pip 10.0.1 py37_0
p.s. Since pyarrow seems to need the conda-forge channel, this is my channel list ("conda config --show channels"): channels:
- conda-forge
- anaconda-fusion
- defaults
The latest pyarrow package version solves the problem. I can now install pyarrow using anaconda under python 3.7:
(base) C:\>conda install pyarrow
Solving environment: done
## Package Plan ##
environment location: C:\Users\xxx
added / updated specs:
- pyarrow
The following packages will be downloaded:
package | build
---------------------------|-----------------
pyarrow-0.11.0 | py37h33f27b4_0 1.5 MB
libboost-1.67.0 | hd9e427e_4 31.8 MB
conda-4.5.11 | py37_1000 654 KB conda-forge
gflags-2.2.1 | vc14_0 283 KB conda-forge
lz4-c-1.8.3 | he025d50_1000 254 KB conda-forge
xz-5.2.4 | h2fa13f4_1001 805 KB conda-forge
thrift-cpp-0.11.0 | vc14_2 2.2 MB conda-forge
arrow-cpp-0.11.0 | py37h8e05e8c_0 7.5 MB
glog-0.3.5 | h6538335_1 130 KB
zstd-1.3.3 | vc14_1 1.2 MB conda-forge
------------------------------------------------------------
Total: 46.2 MB
The following NEW packages will be INSTALLED:
arrow-cpp: 0.11.0-py37h8e05e8c_0
gflags: 2.2.1-vc14_0 conda-forge [vc14]
glog: 0.3.5-h6538335_1
libboost: 1.67.0-hd9e427e_4
lz4-c: 1.8.3-he025d50_1000 conda-forge
pyarrow: 0.11.0-py37h33f27b4_0
thrift-cpp: 0.11.0-vc14_2 conda-forge [vc14]
xz: 5.2.4-h2fa13f4_1001 conda-forge
zstd: 1.3.3-vc14_1 conda-forge [vc14]
The following packages will be UPDATED:
conda: 4.5.11-py37_0 --> 4.5.11-py37_1000 conda-forge
FURTHER UPDATE: I discovered that there is a problem affecting the conda-forge channel and the new version of python/anaconda. (Or there may be two problems.)
What seems to solve the problem for now is to have only "defaults" in the channels list, not conda-forge. ("conda config --show channels") (But that may be only because the "defaults" channel does not have some problem update package.)
The symptoms are these:
The following packages will be DOWNGRADED: ipykernel: 4.10.0-py37_0 --> 4.9.0-py36_1 pyarrow: 0.11.1-py37h33f27b4_0 --> 0.8.0-py36_vc14_0 conda-forge [vc14] python: 3.7.0-hea74fb7_0 --> 3.6.6-hea74fb7_0
ERROR conda.core.link:_execute(502): An error occurred while installing package 'conda-forge::automat-0.7.0-py_1'. CondaError: Cannot link a source that does not exist. C:\Users\xxx\anaconda3\Scripts\conda.exe Running conda clean --packages may resolve your problem. Attempting to roll back. Rolling back transaction: Rolling back transaction: done CondaError: Cannot link a source that does not exist. C:\Users\xxx\anaconda3\Scripts\conda.exe Running conda clean --packages may resolve your problem.