requests==2.32.0 breakage: Not supported URL scheme http+docker

[ssbarnea]

SUMMARY

Apparently requests==2.32.0 which was released few hours ago is not compatible with docker.

https://github.com/psf/requests/releases/tag/v2.32.0

TASK [Create a container] ******************************************************
failed: [localhost] (item={'image': 'ubuntu:18.04', 'name': 'molecule-ubuntu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"image": "ubuntu:18.04", "name": "molecule-ubuntu"}, "msg": "Error connecting: Error while fetching server API version: Not supported URL scheme http+docker"}

ISSUE TYPE

• Bug Report

COMPONENT NAME

ANSIBLE VERSION

ansible [core 2.16.6]
  config file = None
  configured module search path = ['/Users/ssbarnea/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /Users/ssbarnea/.asdf/installs/python/3.12.3/lib/python3.12/site-packages/ansible
  ansible collection location = /Users/ssbarnea/.ansible/collections:/usr/share/ansible/collections
  executable location = /Users/ssbarnea/.asdf/installs/python/3.12.3/bin/ansible
  python version = 3.12.3 (main, May 10 2024, 13:04:47) [Clang 15.0.0 (clang-1500.3.9.4)] (/Users/ssbarnea/.asdf/installs/python/3.12.3/bin/python3.12)
  jinja version = 3.1.4
  libyaml = True

COLLECTION VERSION

$ ansible-galaxy collection list community.docker

# /Users/ssbarnea/.ansible/collections/ansible_collections
Collection       Version
---------------- -------
community.docker 3.10.0 

CONFIGURATION

CONFIG_FILE() = None
DEFAULT_DEBUG(env: ANSIBLE_DEBUG) = False
DEVEL_WARNING(env: ANSIBLE_DEVEL_WARNING) = False
EDITOR(env: EDITOR) = code -r -a
PAGER(env: PAGER) = less

OS / ENVIRONMENT

macos aarch64

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

[felixfontein]

There is no commit that could change that. I'm pretty sure this is not related to this collection, but to some third-party code.

Also without any information on which task is actually failing, this is impossible to debug or even further investigate.

[ssbarnea]

I agree with you. Is likely coming from another location, but it does reproduce on CI: https://github.com/ansible/molecule/actions/runs/9162868658/job/25190776037?pr=4183#step:10:95

I am still working on finding the source. I will close it for now as it is clearly not caused by 3.10, tried 3.9 and 3.8.1 and got the same error.

Still, is must be something that happened in the last 5 days, as I do have a success job that is this old.

[felixfontein]

If I understand it correctly, the first CI failure showed up roughly one hour ago in ansible/molecule?

There's a new release of requests (2.32.0) that's two hours old. Could be related to that.

[felixfontein]

CI for this collection was still running through this morning (11 hours ago), but is no longer.

My guess is that it's requests; also that error message fits pretty well to this hypothesis since the code registers a handler for http+docker with requests.

[felixfontein]

According to git bisect this is caused by psf/requests@c0813a2.

[felixfontein]

Ok, this isn't very surprising, since that commit makes send() call _get_connection() instead of get_connection(). Now the Docker SDK for Python code (both the vendored one and the one in Docker SDK for Python) overwrites get_connection(), but of course doesn't magically overwrite _get_connection() as well...

[felixfontein]

docker/docker-py#3256 is the corresponding issue in Docker SDK for Python.

[felixfontein]

#861 has a hotfix for this. I'll try to get it out today so it will get included in tomorrow's Ansible releases.

[felixfontein]

community.docker 3.10.1 is out with that fix. Depending on which modules/plugins you use it should be fixed. If you're using Docker Swarm stuff, or the deprecated docker_compose v1 module, you will need to pin requests for now.

[vanga]

I got hit by this and has been a nasty thing that broke some critical things :(
Whichever is the component responsible, always a good idea to pin the versions

[vanga]

This is breaking again for me even with the 3.10.1, it worked yesterday, strange that it is running into same issue now. I can confirm from the collections list command that I have 3.10.1.

I ended up changing my tasks to install requests modules first, by pinning its version and then installing docker module.

[felixfontein]

Did you try community.docker 3.10.2, which also works with requests 2.32.2+? requests 2.32.2 changed the way the part Docker SDK for Python has to patch again, this time making the necessary interface public and documented. community.docker 3.10.2 works with both requests 2.32.0/1 and 2.32.2+, assuming you're not using Docker Swarm modules or docker_compose v1.

(requests 2.32.0 and 2.32.1 have been yanked from PyPI BTW.)

[vanga]

I just tried with 3.10.2, it works.
But, I will probably stick to pinning requests module, that sounds like a safer, predictable setup in general.
Though, it is possible that pinning can also break if some other dependency needs latest version of requests, but then I would imagine that I would need to update the community.docker plugin that enforces the latest version of requests module.

[felixfontein]

But, I will probably stick to pinning requests module, that sounds like a safer, predictable setup in general.

It is not. requests 2.32.0 fixes a security problem.

community.docker 3.10.2 uses a workaround supported by a new public API of requests (introduced in its 2.32.2 release), so that should keep working for some time (at least it shouldn't break because of this problem).

[helloanoop]

I was facing the same issue.
Just did ansible-galaxy collection install community.docker --force to get the latest version on my local machine and it worked.