pip install fails on Cygwin

[allenatdecisiv]

I was updating my AWSCLI v2 from 2.1.29 to 2.2.3 and I got this AWSCRT build/install error:

{...}
Building wheels for collected packages: awscrt, cryptography
    ERROR: Command errored out with exit status 1:
     command: /cygdrive/c/Users/allen/.virtualenvs/awscliv2/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py'"'"'; __file__='"'"'/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-z_23s1ai/install-record.txt --single-version-externally-managed --compile --install-headers /cygdrive/c/Users/allen/.virtualenvs/awscliv2/include/site/python3.8/awscrt
         cwd: /tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/
    Complete output (56 lines):
    running install
    running build
    running build_py
    creating build/lib.cygwin-3.2.0-x86_64-3.8
    creating build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/auth.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/common.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/crypto.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/exceptions.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/http.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/io.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/mqtt.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/s3.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/_test.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    copying awscrt/__init__.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt
    creating build/lib.cygwin-3.2.0-x86_64-3.8/awscrt/eventstream
    copying awscrt/eventstream/rpc.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt/eventstream
    copying awscrt/eventstream/__init__.py -> build/lib.cygwin-3.2.0-x86_64-3.8/awscrt/eventstream
    running build_ext
    --- Building dependency: aws-lc (RelWithDebInfo) ---
    /cygdrive/c/ProgramData/scoop/shims/cmake -DCMAKE_PREFIX_PATH=/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/build/deps/install -DCMAKE_INSTALL_PREFIX=/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/build/deps/install -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTING=OFF -DCMAKE_POSITION_INDEPENDENT_CODE=ON -DBUILD_LIBSSL=OFF /tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/crt/aws-lc
    CMake Error: The source directory "/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/crt/aws-lc" does not exist.
    Specify --help for usage, or press the help button on the CMake GUI.
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py", line 263, in <module>
        setuptools.setup(
      File "/cygdrive/c/Users/allen/.virtualenvs/awscliv2/lib/python3.8/site-packages/setuptools/__init__.py", line 153, in setup
        return distutils.core.setup(**attrs)
      File "/usr/lib/python3.8/distutils/core.py", line 148, in setup
        dist.run_commands()
      File "/usr/lib/python3.8/distutils/dist.py", line 966, in run_commands
        self.run_command(cmd)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/cygdrive/c/Users/allen/.virtualenvs/awscliv2/lib/python3.8/site-packages/setuptools/command/install.py", line 61, in run
        return orig.install.run(self)
      File "/usr/lib/python3.8/distutils/command/install.py", line 556, in run
        self.run_command('build')
      File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/usr/lib/python3.8/distutils/command/build.py", line 135, in run
        self.run_command(cmd_name)
      File "/usr/lib/python3.8/distutils/cmd.py", line 313, in run_command
        self.distribution.run_command(command)
      File "/usr/lib/python3.8/distutils/dist.py", line 985, in run_command
        cmd_obj.run()
      File "/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py", line 197, in run
        self._build_dependency(lib)
      File "/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py", line 175, in _build_dependency
        subprocess.check_call(cmake_args)
      File "/usr/lib/python3.8/subprocess.py", line 364, in check_call
        raise CalledProcessError(retcode, cmd)
    subprocess.CalledProcessError: Command '['/cygdrive/c/ProgramData/scoop/shims/cmake', '-DCMAKE_PREFIX_PATH=/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/build/deps/install', '-DCMAKE_INSTALL_PREFIX=/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/build/deps/install', '-DBUILD_SHARED_LIBS=OFF', '-DCMAKE_BUILD_TYPE=RelWithDebInfo', '-DBUILD_TESTING=OFF', '-DCMAKE_POSITION_INDEPENDENT_CODE=ON', '-DBUILD_LIBSSL=OFF', '/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/crt/aws-lc']' returned non-zero exit status 1.
    ----------------------------------------
ERROR: Command errored out with exit status 1: /cygdrive/c/Users/allen/.virtualenvs/awscliv2/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py'"'"'; __file__='"'"'/tmp/pip-install-logv0wqy/awscrt_3b94cb75e77f40b88a9c8fd739e1e147/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-z_23s1ai/install-record.txt --single-version-externally-managed --compile --install-headers /cygdrive/c/Users/allen/.virtualenvs/awscliv2/include/site/python3.8/awscrt Check the logs for full command output.

Basic Python information:

$ python -V
Python 3.8.10
$ pip -V
pip 21.0.1 from /cygdrive/c/Users/allen/.virtualenvs/awscliv2/lib/python3.8/site-packages/pip (python 3.8)
$ virtualenv --version
virtualenv 20.4.2 from /cygdrive/c/Users/allen/.virtualenvs/awscliv2/lib/python3.8/site-packages/virtualenv/__init__.py
$ aws --version
aws-cli/2.1.29 Python/3.8.10 CYGWIN_NT-10.0-19042/3.2.0-340.x86_64 source/x86_64 prompt/off
$ uname -a
CYGWIN_NT-10.0 FOUREYES 3.2.0(0.340/5/3) 2021-03-29 08:42 x86_64 Cygwin

[jmklix]

Are you still unable to install the cli v2 on Cygwin?

I would recommend trying this suggestion from stackoverflow:

The solution I opted for in the end was to install the MSI installer for Windows. Although this results in the binary being installed to C:\Program Files rather than directly to Cygwin's /bin directory like I would have liked, the installer automatically adds the binary to the PATH so it can be immediately run from within a Cygwin shell (you may need to reboot to get Windows to recognise the changed PATH).

Please let us know if that doesn't work for you

[allenatdecisiv]

Hi @jmklix ,

I've left Windows for Fedora and don't have this issue any longer. I create wheels/install from source now.

But this issue is from AWSCRT io using edge triggered epoll on Linux, which isn't portable/available on Cygwin, vs using something that would be cross platform portable like libevent, asio, llibuv, libxev, etc. Cygwin is very Linux/UNIX like but it doesn't support epoll currently.

Essentially, AWSCRT doesn't support Cygwin in any meaningful way. It's major brand Linux, macOS and Windows and that's it.

Installing AWSCLI in the way you describe, installs AWSCLI into Windows and not Cygwin (per se). I understand I might be splitting hairs, but Windows isn't Cygwin and Cygwin isn't Windows. Generally speaking, to be happy using Cygwin, end-users would want programs installed in the Cygwin environment and not in Windows (to be used by pathing in Cygwin).

Then there's line endings, quoting issues, piping issues when using AWSCLI for Windows from Cygwin. It's just a pain to deal with this way.

Thanks,