3

I need to Install Python3.7 to support Spark 2 on an M1 Mac.

I can get Python3.7 installed using pyenv, but when I try to install any of the data science libs, like numpy, I get failure: ModuleNotFoundError: No module named '_ctypes'

This seems to be related to libffi. Looking through the build logs for pyenv, I'm seeing this error:

.../Python-3.7.10/Modules/_ctypes/libffi_osx/include/fficonfig.h:51:2: error: "Unknown CPU type"
#error "Unknown CPU type"
 ^
1 error generated.

Python build finished successfully!
Failed to build these modules:
_ctypes               _decimal

Has anyone been able to successfully install 3.7 with numpy natively? I'm not looking for answers that involve Rosetta.

kellanburket
  • 11,202
  • 3
  • 41
  • 68
  • 2
    Make sure the pyenv build does in fact include ctypes. I'd imagine it's autodetecting things wrong... – AKX Nov 19 '21 at 16:43
  • Looks like it's failing to build ctypes. will post logs – kellanburket Nov 19 '21 at 16:47
  • 1
    Yeah, based on that error I think Python 3.7 just doesn't support the M1 without some manual patching to make it understand the cpu it's working with... – AKX Nov 19 '21 at 17:01

1 Answers1

1

Use miniconda to install things: see this post.

Miniconda is an open source version of the conda package/environment manager, as an alternative to pyenv + brew.

It will install ARM native versions of python by default, as well as native versions of modules. This definitely include ARM native numpy.

I've quite sucessfully installed native python 3.7-3.10 and native numpy without any problems at all. (In fact ARM native versions of almost every package I use).

With conda you use conda to install packages, not pip.

So to install numpy you do conda install numpy in the environment you're in.

Richard
  • 2,506
  • 1
  • 12
  • 35
  • Not super famiiar with conda, but was able to find an installer through Miniforge. However, it doesn't seem like python3.7 is available: "PackagesNotFoundError: The following packages are not available from current channels: - python=3.7" – kellanburket Nov 19 '21 at 20:07
  • did you try this command? `conda create -n test_env python=3.7`. That should create a new environment (called 'test_env') with python 3.7 as the python version it uses. This works for me, and I checked the **conda-forge** repository (where miniforge conda will go to get packages) has python 3.5 and later available. – Richard Nov 19 '21 at 20:21
  • 'channels' are conda-speak for repositories. by default miniforge conda should look in the conda-forge repository. At a prompt, type 'conda search python`. That should show you it's using the 'conda-forge' channel, and it will then list all the available versions of python you can install – Richard Nov 19 '21 at 20:23
  • 4
    very weird. I'm on the conda-forge channel, but not seeing anything before 3.8 when I search. same issue with the `conda create` command. It can't find 3.7 – kellanburket Nov 19 '21 at 21:14
  • 2
    I've put a significant amount of time into trying to get 3.7 installed and I'm pretty sure it's not possible. – James Parker Mar 03 '22 at 19:25
  • This is strange!! When I search conda-forge I can find 3.7 (actually all versions back to 1.0.1!). – Richard Mar 04 '22 at 19:27