Skip to content

Conversation

@saurabh-hirani
Copy link

@saurabh-hirani saurabh-hirani commented Apr 14, 2019

As python3 uses recv_into and it is not patched by current gevent_openssl, a grequests call with pyopenssl installed falls back to https://github.com/urllib3/urllib3/blob/1e9ab5aee042ff0158d0f443bc600ef3a2e7bf9a/src/urllib3/contrib/pyopenssl.py#L302 making the code slow.

This issue is explained in depth at:

However, to keep the focus on the current patch, this issue can be replicated by following these steps:

  1. Start an https server which has a delayed response functionality by following this gist.
  2. Install gevent_openssl - pip3 install gevent-openssl
  3. Run test_grequests.py which calls https://localhost:8082/delay/1 to introduce a 1 second delay in response.
  4. We will see a 10 second delay for fetching 10 urls as shown in this gist.
  5. Uninstall gevent_openssl - pip3 uninstall gevent-openssl
  6. Install this patch and rerun test_grequests.py
  7. As python3 will make a recv_into call supported by this patch, the code will run in around 1 second for fetching 10 urls as shown in this gist.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant