Issues Installing Python 3.x via Pyenv

Question

I have just gotten a new Macbook Air with M1 chip and I am trying to install Python 3.8.3 (or any 3.x version) via pyenv. I was able to install pyenv via Homebrew, but when I try to install a new python version I get an error like the below. I believe it is something to do with the new chip and/or the Big Sur OS. I have tried the instructions at these links and the error message is the same:

https://github.com/pyenv/pyenv/issues/1643#issuecomment-655710632

https://dev.to/kojikanao/install-python-3-8-0-via-pyenv-on-bigsur-4oee

Here's what I type into the terminal and what I get in return:

$ CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" pyenv install --patch 3.8.3 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)
python-build: use [email protected] from homebrew
python-build: use readline from homebrew
Downloading Python-3.8.3.tar.xz...
-> https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tar.xz
Installing Python-3.8.3...
patching file Misc/NEWS.d/next/macOS/2020-06-24-13-51-57.bpo-41100.mcHdc5.rst
patching file configure
Hunk #1 succeeded at 3398 (offset -28 lines).
patching file configure.ac
Hunk #1 succeeded at 498 (offset -12 lines).
python-build: use readline from homebrew
python-build: use zlib from xcode sdk
BUILD FAILED (OS X 11.1 using python-build 20180424)
Inspect or clean up the working tree at /var/folders/w3/wh28vlqs2txcyl67cjcd689h0000gn/T/python-build.20201217143449.26458
Results logged to /var/folders/w3/wh28vlqs2txcyl67cjcd689h0000gn/T/python-build.20201217143449.26458.log
Last 10 log lines:
checking size of _Bool... 1
checking size of off_t... 8
checking whether to enable large file support... no
checking size of time_t... 8
checking for pthread_t... yes
checking size of pthread_t... 8
checking size of pthread_key_t... 8
checking whether pthread_key_t is compatible with int... no
configure: error: Unexpected output of 'arch' on OSX
make: *** No targets specified and no makefile found. Stop.

Answer 1

I hit the same issue. As mentioned in my initial comment, a bug report and patch already exist for this in the Python codebase - though the fix is yet to be released.

In the meantime, the solution I went for is to run everything (including Homebrew) in "x86_64 mode" under Rosetta 2. To do this for any given command, prepend the command with arch -x86_64. Or see this for how to set up Terminal to run all commands under Rosetta.

Specific steps:

1. Install the x86_64 version of Homebrew, per https://stackoverflow.com/a/64883440/8285141:

arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"

Once installed, ensure that you are running the x86_64 version of brew from /usr/local/bin/brew. I.e. you should get the following:

$ which brew
/usr/local/bin/brew

If you're running the arm64 version from /opt/homebrew/bin/brew you need to modify your system path so that /usr/local/bin takes precedence. FWIW, the arm64 and x86_64 Homebrew installs can coexist peacefully.

2. Install pyenv and some Python build dependencies

arch -x86_64 brew install pyenv bzip2 zlib

3. Install your desired version of Python through pyenv (per this comment that OP referenced)

CFLAGS="-I$(brew --prefix openssl)/include -I$(brew --prefix bzip2)/include -I$(brew --prefix readline)/include -I$(xcrun --show-sdk-path)/usr/include" LDFLAGS="-L$(brew --prefix openssl)/lib -L$(brew --prefix readline)/lib -L$(brew --prefix zlib)/lib -L$(brew --prefix bzip2)/lib" \
pyenv install --patch 3.8.3 < <(curl -sSL https://github.com/python/cpython/commit/8ea6353.patch\?full_index\=1)

---

Epilogue:

Although I did not follow the exact process above, I believe it reflects what's required. I skipped steps like installing the XCode Command Line Tools, which are well-documented elsewhere.

If you get errors like

zipimport.ZipImportError: can't decompress data; zlib not available

or warnings like

WARNING: The Python bz2 extension was not compiled. Missing the bzip2 lib?
WARNING: The Python readline extension was not compiled. Missing the GNU readline lib?

it probably means those libraries aren't actually installed properly by Homebrew. Doing e.g. brew uninstall zlib && brew install zlib took care of those for me.