Some Python module is crashing on arm device while getting installed

[guyluz11]

When I tested the code on a python package named aioesphomeapi I am getting a big error.

I am running the code on a Linux arm device, on my main computer Linux x64 the same code runs fine.

import "package:python_shell/python_shell.dart";

void main() async {
  var shell = PythonShell(PythonShellConfig());
  await shell.initialize();

  var instance = ShellManager.getInstance("default");
  instance.installRequires(["aioesphomeapi"]);
  await instance.runString(
    """
print('before')
import os
print('after os')

import aioesphomeapi
print ('after aioesphomeapi')

""",
    echo: true,
  );

  print("finished");
}

Some packages do not support arm (I think), in this case it does.
Installing it outside of the virtual env worked for me

$ pip install aioesphomeapi
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: aioesphomeapi in /home/jinnime/.local/lib/python3.10/site-packages (13.0.1)
Requirement already satisfied: noiseprotocol<1.0,>=0.3.1 in /home/jinnime/.local/lib/python3.10/site-packages (from aioesphomeapi) (0.3.1)
Requirement already satisfied: async-timeout>=4.0 in /home/jinnime/.local/lib/python3.10/site-packages (from aioesphomeapi) (4.0.2)
Requirement already satisfied: protobuf>=3.19.0 in /home/jinnime/.local/lib/python3.10/site-packages (from aioesphomeapi) (4.21.10)
Requirement already satisfied: zeroconf<1.0,>=0.36.0 in /home/jinnime/.local/lib/python3.10/site-packages (from aioesphomeapi) (0.39.4)
Requirement already satisfied: cryptography>=2.8 in /usr/lib/python3/dist-packages (from noiseprotocol<1.0,>=0.3.1->aioesphomeapi) (3.4.8)
Requirement already satisfied: ifaddr>=0.1.7 in /home/jinnime/.local/lib/python3.10/site-packages (from zeroconf<1.0,>=0.36.0->aioesphomeapi) (0.2.0)

I have added some prints of Stdout and Stderr to the process, this is why the log is so big

Log:

$ ~/dart-sdk/bin/dart example/python_shell_example.dart 
Initializing shell...
Default settings for virtualenv...
result Requirements installed successfully.
Virtualenv settings finished.
virtualenvSettings Requirements installed successfully.
Shell initialized.
Installing requirements...
Error while installing requirements, exist code 1
Stdout: Collecting aioesphomeapi
  Using cached aioesphomeapi-13.0.1-py2.py3-none-any.whl (51 kB)
Collecting zeroconf<1.0,>=0.36.0
  Using cached zeroconf-0.39.4-py3-none-any.whl (106 kB)
Collecting async-timeout>=4.0
  Using cached async_timeout-4.0.2-py3-none-any.whl (5.8 kB)
Collecting noiseprotocol<1.0,>=0.3.1
  Using cached noiseprotocol-0.3.1-py3-none-any.whl (20 kB)
Collecting protobuf>=3.19.0
  Using cached protobuf-4.21.10-py3-none-any.whl (291 kB)
Collecting cryptography>=2.8
  Downloading cryptography-38.0.4.tar.gz (599 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 599.8/599.8 kB 1.5 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'error'

Stderr:   error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [126 lines of output]
      Collecting setuptools!=60.9.0,>=40.6.0
        Using cached setuptools-65.6.3-py3-none-any.whl (1.2 MB)
      Collecting wheel
        Using cached wheel-0.38.4-py3-none-any.whl (36 kB)
      Collecting cffi>=1.12
        Using cached cffi-1.15.1.tar.gz (508 kB)
        Preparing metadata (setup.py): started
        Preparing metadata (setup.py): finished with status 'done'
      Collecting setuptools-rust>=0.11.4
        Using cached setuptools_rust-1.5.2-py3-none-any.whl (23 kB)
      Collecting pycparser
        Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
      Collecting typing-extensions>=3.7.4.3
        Using cached typing_extensions-4.4.0-py3-none-any.whl (26 kB)
      Collecting semantic-version<3,>=2.8.2
        Using cached semantic_version-2.10.0-py2.py3-none-any.whl (15 kB)
      Building wheels for collected packages: cffi
        Building wheel for cffi (setup.py): started
        Building wheel for cffi (setup.py): finished with status 'error'
        error: subprocess-exited-with-error
      
        × python setup.py bdist_wheel did not run successfully.
        │ exit code: 1
        ╰─> [38 lines of output]
            /home/jinnime/.python_shell.dart/instances/default/env/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
              warnings.warn(msg, warning_class)
            running bdist_wheel
            running build
            running build_py
            creating build
            creating build/lib.linux-armv7l-cpython-310
            creating build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/verifier.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/vengine_gen.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/recompiler.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/pkgconfig.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/model.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/lock.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/ffiplatform.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/error.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/cparser.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/commontypes.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/api.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/__init__.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/_cffi_include.h -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/parse_c_type.h -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/_embedding.h -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-cpython-310/cffi
            running build_ext
            building '_cffi_backend' extension
            creating build/temp.linux-armv7l-cpython-310
            creating build/temp.linux-armv7l-cpython-310/c
            arm-linux-gnueabihf-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/home/jinnime/.python_shell.dart/instances/default/env/include -I/usr/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-armv7l-cpython-310/c/_cffi_backend.o
            c/_cffi_backend.c:2:10: fatal error: Python.h: No such file or directory
                2 | #include <Python.h>
                  |          ^~~~~~~~~~
            compilation terminated.
            error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for cffi
        Running setup.py clean for cffi
      Failed to build cffi
      Installing collected packages: wheel, typing-extensions, setuptools, semantic-version, pycparser, setuptools-rust, cffi
        Running setup.py install for cffi: started
        Running setup.py install for cffi: finished with status 'error'
        error: subprocess-exited-with-error
      
        × Running setup.py install for cffi did not run successfully.
        │ exit code: 1
        ╰─> [40 lines of output]
            /home/jinnime/.python_shell.dart/instances/default/env/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
              warnings.warn(msg, warning_class)
            running install
            /home/jinnime/.python_shell.dart/instances/default/env/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
              warnings.warn(
            running build
            running build_py
            creating build
            creating build/lib.linux-armv7l-cpython-310
            creating build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/verifier.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/vengine_gen.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/vengine_cpy.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/setuptools_ext.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/recompiler.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/pkgconfig.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/model.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/lock.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/ffiplatform.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/error.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/cparser.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/commontypes.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/cffi_opcode.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/backend_ctypes.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/api.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/__init__.py -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/_cffi_include.h -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/parse_c_type.h -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/_embedding.h -> build/lib.linux-armv7l-cpython-310/cffi
            copying cffi/_cffi_errors.h -> build/lib.linux-armv7l-cpython-310/cffi
            running build_ext
            building '_cffi_backend' extension
            creating build/temp.linux-armv7l-cpython-310
            creating build/temp.linux-armv7l-cpython-310/c
            arm-linux-gnueabihf-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/home/jinnime/.python_shell.dart/instances/default/env/include -I/usr/include/python3.10 -c c/_cffi_backend.c -o build/temp.linux-armv7l-cpython-310/c/_cffi_backend.o
            c/_cffi_backend.c:2:10: fatal error: Python.h: No such file or directory
                2 | #include <Python.h>
                  |          ^~~~~~~~~~
            compilation terminated.
            error: command '/usr/bin/arm-linux-gnueabihf-gcc' failed with exit code 1
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
      error: legacy-install-failure
      
      × Encountered error while trying to install package.
      ╰─> cffi
      
      note: This is an issue with the package mentioned above, not pip.
      hint: See above for output from the failure.
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.

before
after os
finished

[guyluz11]

As written here
https://stackoverflow.com/questions/21530577/fatal-error-python-h-no-such-file-or-directory
I was missing python3-dev on my computer

If someone gets a simular error execute sudo apt-get install python3-dev in the command line.

After that I got another error about ffi, and found that I need to install another program on my computer sudo apt install libffi-dev

And after that I got another error about rust compiler missing so I had to install another program with sudo apt install rustc

And after that I was missing cargo so sudo apt install cargo

It looks to me that this packet needs to show more information about errors in the console as I did.

[guyluz11]

Added error logs to pr #4