From 8ce1c14ef6eafbd1b0afb59ad0f47cbc01e5194c Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Thu, 5 Dec 2024 18:08:26 -0600 Subject: [PATCH 01/23] Increment version to 3.11.11.dev0 (#10130) --- aiohttp/__init__.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/aiohttp/__init__.py b/aiohttp/__init__.py index 8c80ff3ab7d..f4d732b8674 100644 --- a/aiohttp/__init__.py +++ b/aiohttp/__init__.py @@ -1,4 +1,4 @@ -__version__ = "3.11.10" +__version__ = "3.11.11.dev0" from typing import TYPE_CHECKING, Tuple From 7b868436beb3aa1aaf1b6d90da3fe830aa97f632 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Fri, 6 Dec 2024 00:56:24 +0000 Subject: [PATCH 02/23] [PR #10131/7f92bebb backport][3.11] Bump Python version for benchmarks to 3.13 (#10132) Co-authored-by: J. Nick Koston --- .github/workflows/ci-cd.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 765047b933f..95238b93687 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -250,11 +250,11 @@ jobs: uses: actions/checkout@v4 with: submodules: true - - name: Setup Python 3.12 + - name: Setup Python 3.13 id: python-install uses: actions/setup-python@v5 with: - python-version: 3.12 + python-version: 3.13 cache: pip cache-dependency-path: requirements/*.txt - name: Update pip, wheel, setuptools, build, twine From 0e4a0e4829894e8aabe175120ea74311a25a1a8e Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Fri, 6 Dec 2024 04:07:44 +0000 Subject: [PATCH 03/23] [PR #10131/7f92bebb backport][3.12] Bump Python version for benchmarks to 3.13 (#10133) Co-authored-by: J. Nick Koston --- .github/workflows/ci-cd.yml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 765047b933f..95238b93687 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -250,11 +250,11 @@ jobs: uses: actions/checkout@v4 with: submodules: true - - name: Setup Python 3.12 + - name: Setup Python 3.13 id: python-install uses: actions/setup-python@v5 with: - python-version: 3.12 + python-version: 3.13 cache: pip cache-dependency-path: requirements/*.txt - name: Update pip, wheel, setuptools, build, twine From 87f0f4866c9e7710d0d66367a94425e0e7e3d642 Mon Sep 17 00:00:00 2001 From: "dependabot[bot]" <49699333+dependabot[bot]@users.noreply.github.com> Date: Fri, 6 Dec 2024 10:43:25 +0000 Subject: [PATCH 04/23] Bump actions/cache from 4.1.2 to 4.2.0 (#10136) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bumps [actions/cache](https://github.com/actions/cache) from 4.1.2 to 4.2.0.
Release notes

Sourced from actions/cache's releases.

v4.2.0

⚠️ Important Changes

The cache backend service has been rewritten from the ground up for improved performance and reliability. actions/cache now integrates with the new cache service (v2) APIs.

The new service will gradually roll out as of February 1st, 2025. The legacy service will also be sunset on the same date. Changes in these release are fully backward compatible.

We are deprecating some versions of this action. We recommend upgrading to version v4 or v3 as soon as possible before February 1st, 2025. (Upgrade instructions below).

If you are using pinned SHAs, please use the SHAs of versions v4.2.0 or v3.4.0

If you do not upgrade, all workflow runs using any of the deprecated actions/cache will fail.

Upgrading to the recommended versions will not break your workflows.

Read more about the change & access the migration guide: reference to the announcement.

Minor changes

Minor and patch version updates for these dependencies:

  • @​actions/core: 1.11.1
  • @​actions/io: 1.1.3
  • @​vercel/ncc: 0.38.3

Full Changelog: https://github.com/actions/cache/compare/v4...v4.2.0

Changelog

Sourced from actions/cache's changelog.

4.2.0

TLDR; The cache backend service has been rewritten from the ground up for improved performance and reliability. actions/cache now integrates with the new cache service (v2) APIs.

The new service will gradually roll out as of February 1st, 2025. The legacy service will also be sunset on the same date. Changes in these release are fully backward compatible.

We are deprecating some versions of this action. We recommend upgrading to version v4 or v3 as soon as possible before February 1st, 2025. (Upgrade instructions below).

If you are using pinned SHAs, please use the SHAs of versions v4.2.0 or v3.4.0

If you do not upgrade, all workflow runs using any of the deprecated actions/cache will fail.

Upgrading to the recommended versions will not break your workflows.

Commits

[![Dependabot compatibility score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=actions/cache&package-manager=github_actions&previous-version=4.1.2&new-version=4.2.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) ---
Dependabot commands and options
You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot merge` will merge this PR after your CI passes on it - `@dependabot squash and merge` will squash and merge this PR after your CI passes on it - `@dependabot cancel merge` will cancel a previously requested merge and block automerging - `@dependabot reopen` will reopen this PR if it is closed - `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually - `@dependabot show ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- .github/workflows/ci-cd.yml | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/.github/workflows/ci-cd.yml b/.github/workflows/ci-cd.yml index 95238b93687..d5e119b779d 100644 --- a/.github/workflows/ci-cd.yml +++ b/.github/workflows/ci-cd.yml @@ -47,7 +47,7 @@ jobs: with: python-version: 3.11 - name: Cache PyPI - uses: actions/cache@v4.1.2 + uses: actions/cache@v4.2.0 with: key: pip-lint-${{ hashFiles('requirements/*.txt') }} path: ~/.cache/pip @@ -99,7 +99,7 @@ jobs: with: submodules: true - name: Cache llhttp generated files - uses: actions/cache@v4.1.2 + uses: actions/cache@v4.2.0 id: cache with: key: llhttp-${{ hashFiles('vendor/llhttp/package*.json', 'vendor/llhttp/src/**/*') }} @@ -163,7 +163,7 @@ jobs: echo "dir=$(pip cache dir)" >> "${GITHUB_OUTPUT}" shell: bash - name: Cache PyPI - uses: actions/cache@v4.1.2 + uses: actions/cache@v4.2.0 with: key: pip-ci-${{ runner.os }}-${{ matrix.pyver }}-${{ matrix.no-extensions }}-${{ hashFiles('requirements/*.txt') }} path: ${{ steps.pip-cache.outputs.dir }} From 489b6649a37b1850ef7c3dc042252cfb90f7daf5 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Fri, 6 Dec 2024 17:34:12 +0000 Subject: [PATCH 05/23] [PR #10138/dbd77ad6 backport][3.11] Bump sphinx to 8.1.3 along with required dependencies (#10139) Co-authored-by: J. Nick Koston --- requirements/constraints.txt | 14 +++++++------- requirements/dev.txt | 14 +++++++------- requirements/doc-spelling.txt | 14 +++++++------- requirements/doc.txt | 14 +++++++------- 4 files changed, 28 insertions(+), 28 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index d32acc7b773..740e3e2d559 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -14,7 +14,7 @@ aiohttp-theme==0.1.7 # via -r requirements/doc.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx annotated-types==0.7.0 # via pydantic @@ -236,22 +236,22 @@ slotscheck==0.19.1 # via -r requirements/lint.in snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-spelling # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" # via -r requirements/doc-spelling.in diff --git a/requirements/dev.txt b/requirements/dev.txt index 168ce639d19..72e49ed9edf 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -14,7 +14,7 @@ aiohttp-theme==0.1.7 # via -r requirements/doc.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx annotated-types==0.7.0 # via pydantic @@ -210,21 +210,21 @@ slotscheck==0.19.1 # via -r requirements/lint.in snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-towncrier==0.4.0a0 # via -r requirements/doc.in diff --git a/requirements/doc-spelling.txt b/requirements/doc-spelling.txt index df393012548..892ae6b164c 100644 --- a/requirements/doc-spelling.txt +++ b/requirements/doc-spelling.txt @@ -6,7 +6,7 @@ # aiohttp-theme==0.1.7 # via -r requirements/doc.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx babel==2.16.0 # via sphinx @@ -46,22 +46,22 @@ requests==2.32.3 # via sphinx snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-spelling # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" # via -r requirements/doc-spelling.in diff --git a/requirements/doc.txt b/requirements/doc.txt index 43b7c6b7e8b..f7f98330e1f 100644 --- a/requirements/doc.txt +++ b/requirements/doc.txt @@ -6,7 +6,7 @@ # aiohttp-theme==0.1.7 # via -r requirements/doc.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx babel==2.16.0 # via sphinx @@ -44,21 +44,21 @@ requests==2.32.3 # via sphinx snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-towncrier==0.4.0a0 # via -r requirements/doc.in From f52f60a4ba8f023d08c4b319a0d8c0e7d290d4cf Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Fri, 6 Dec 2024 17:39:22 +0000 Subject: [PATCH 06/23] [PR #10138/dbd77ad6 backport][3.12] Bump sphinx to 8.1.3 along with required dependencies (#10140) Co-authored-by: J. Nick Koston --- requirements/constraints.txt | 14 +++++++------- requirements/dev.txt | 14 +++++++------- requirements/doc-spelling.txt | 14 +++++++------- requirements/doc.txt | 14 +++++++------- 4 files changed, 28 insertions(+), 28 deletions(-) diff --git a/requirements/constraints.txt b/requirements/constraints.txt index d32acc7b773..740e3e2d559 100644 --- a/requirements/constraints.txt +++ b/requirements/constraints.txt @@ -14,7 +14,7 @@ aiohttp-theme==0.1.7 # via -r requirements/doc.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx annotated-types==0.7.0 # via pydantic @@ -236,22 +236,22 @@ slotscheck==0.19.1 # via -r requirements/lint.in snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-spelling # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" # via -r requirements/doc-spelling.in diff --git a/requirements/dev.txt b/requirements/dev.txt index 168ce639d19..72e49ed9edf 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -14,7 +14,7 @@ aiohttp-theme==0.1.7 # via -r requirements/doc.in aiosignal==1.3.1 # via -r requirements/runtime-deps.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx annotated-types==0.7.0 # via pydantic @@ -210,21 +210,21 @@ slotscheck==0.19.1 # via -r requirements/lint.in snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-towncrier==0.4.0a0 # via -r requirements/doc.in diff --git a/requirements/doc-spelling.txt b/requirements/doc-spelling.txt index df393012548..892ae6b164c 100644 --- a/requirements/doc-spelling.txt +++ b/requirements/doc-spelling.txt @@ -6,7 +6,7 @@ # aiohttp-theme==0.1.7 # via -r requirements/doc.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx babel==2.16.0 # via sphinx @@ -46,22 +46,22 @@ requests==2.32.3 # via sphinx snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-spelling # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-spelling==8.0.0 ; platform_system != "Windows" # via -r requirements/doc-spelling.in diff --git a/requirements/doc.txt b/requirements/doc.txt index 43b7c6b7e8b..f7f98330e1f 100644 --- a/requirements/doc.txt +++ b/requirements/doc.txt @@ -6,7 +6,7 @@ # aiohttp-theme==0.1.7 # via -r requirements/doc.in -alabaster==0.7.13 +alabaster==1.0.0 # via sphinx babel==2.16.0 # via sphinx @@ -44,21 +44,21 @@ requests==2.32.3 # via sphinx snowballstemmer==2.2.0 # via sphinx -sphinx==7.1.2 +sphinx==8.1.3 # via # -r requirements/doc.in # sphinxcontrib-towncrier -sphinxcontrib-applehelp==1.0.4 +sphinxcontrib-applehelp==2.0.0 # via sphinx -sphinxcontrib-devhelp==1.0.2 +sphinxcontrib-devhelp==2.0.0 # via sphinx -sphinxcontrib-htmlhelp==2.0.1 +sphinxcontrib-htmlhelp==2.1.0 # via sphinx sphinxcontrib-jsmath==1.0.1 # via sphinx -sphinxcontrib-qthelp==1.0.3 +sphinxcontrib-qthelp==2.0.0 # via sphinx -sphinxcontrib-serializinghtml==1.1.5 +sphinxcontrib-serializinghtml==2.0.0 # via sphinx sphinxcontrib-towncrier==0.4.0a0 # via -r requirements/doc.in From eb42db85a461f3cc883c3e41a43bb02c9b87e308 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Sun, 8 Dec 2024 15:26:09 +0000 Subject: [PATCH 07/23] Fix type of SSLContext for some static type checkers (#10099) (#10144) (cherry picked from commit 6200513d8fd34a820a4d10d238ca92d9f73ce7ee) Co-authored-by: AlanBogarin --- CHANGES/10099.bugfix.rst | 1 + CONTRIBUTORS.txt | 1 + aiohttp/client_exceptions.py | 10 +++++++--- aiohttp/client_reqrep.py | 12 ++++++++---- aiohttp/connector.py | 14 +++++++++----- aiohttp/web.py | 10 +++++++--- aiohttp/web_runner.py | 12 +++++++----- aiohttp/worker.py | 15 ++++++++++----- docs/spelling_wordlist.txt | 1 + 9 files changed, 51 insertions(+), 25 deletions(-) create mode 100644 CHANGES/10099.bugfix.rst diff --git a/CHANGES/10099.bugfix.rst b/CHANGES/10099.bugfix.rst new file mode 100644 index 00000000000..718420a6ad5 --- /dev/null +++ b/CHANGES/10099.bugfix.rst @@ -0,0 +1 @@ +Fixed type of ``SSLContext`` for some static type checkers (e.g. pyright). diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index c3abc66bebf..94d003a1719 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -9,6 +9,7 @@ Adam Mills Adrian Krupa Adrián Chaves Ahmed Tahri +Alan Bogarin Alan Tse Alec Hanefeld Alejandro Gómez diff --git a/aiohttp/client_exceptions.py b/aiohttp/client_exceptions.py index 667da8d5084..1d298e9a8cf 100644 --- a/aiohttp/client_exceptions.py +++ b/aiohttp/client_exceptions.py @@ -8,13 +8,17 @@ from .typedefs import StrOrURL -try: +if TYPE_CHECKING: import ssl SSLContext = ssl.SSLContext -except ImportError: # pragma: no cover - ssl = SSLContext = None # type: ignore[assignment] +else: + try: + import ssl + SSLContext = ssl.SSLContext + except ImportError: # pragma: no cover + ssl = SSLContext = None # type: ignore[assignment] if TYPE_CHECKING: from .client_reqrep import ClientResponse, ConnectionKey, Fingerprint, RequestInfo diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py index e97c40ce0e5..43b48063c6e 100644 --- a/aiohttp/client_reqrep.py +++ b/aiohttp/client_reqrep.py @@ -72,12 +72,16 @@ RawHeaders, ) -try: +if TYPE_CHECKING: import ssl from ssl import SSLContext -except ImportError: # pragma: no cover - ssl = None # type: ignore[assignment] - SSLContext = object # type: ignore[misc,assignment] +else: + try: + import ssl + from ssl import SSLContext + except ImportError: # pragma: no cover + ssl = None # type: ignore[assignment] + SSLContext = object # type: ignore[misc,assignment] __all__ = ("ClientRequest", "ClientResponse", "RequestInfo", "Fingerprint") diff --git a/aiohttp/connector.py b/aiohttp/connector.py index 93bc2513b20..a9123f82bc0 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -60,14 +60,18 @@ ) from .resolver import DefaultResolver -try: +if TYPE_CHECKING: import ssl SSLContext = ssl.SSLContext -except ImportError: # pragma: no cover - ssl = None # type: ignore[assignment] - SSLContext = object # type: ignore[misc,assignment] - +else: + try: + import ssl + + SSLContext = ssl.SSLContext + except ImportError: # pragma: no cover + ssl = None # type: ignore[assignment] + SSLContext = object # type: ignore[misc,assignment] EMPTY_SCHEMA_SET = frozenset({""}) HTTP_SCHEMA_SET = frozenset({"http", "https"}) diff --git a/aiohttp/web.py b/aiohttp/web.py index f975b665331..d6ab6f6fad4 100644 --- a/aiohttp/web.py +++ b/aiohttp/web.py @@ -9,6 +9,7 @@ from contextlib import suppress from importlib import import_module from typing import ( + TYPE_CHECKING, Any, Awaitable, Callable, @@ -287,10 +288,13 @@ ) -try: +if TYPE_CHECKING: from ssl import SSLContext -except ImportError: # pragma: no cover - SSLContext = Any # type: ignore[misc,assignment] +else: + try: + from ssl import SSLContext + except ImportError: # pragma: no cover + SSLContext = object # type: ignore[misc,assignment] # Only display warning when using -Wdefault, -We, -X dev or similar. warnings.filterwarnings("ignore", category=NotAppKeyWarning, append=True) diff --git a/aiohttp/web_runner.py b/aiohttp/web_runner.py index f8933383435..bcfec727c84 100644 --- a/aiohttp/web_runner.py +++ b/aiohttp/web_runner.py @@ -3,7 +3,7 @@ import socket import warnings from abc import ABC, abstractmethod -from typing import Any, List, Optional, Set +from typing import TYPE_CHECKING, Any, List, Optional, Set from yarl import URL @@ -11,11 +11,13 @@ from .web_app import Application from .web_server import Server -try: +if TYPE_CHECKING: from ssl import SSLContext -except ImportError: - SSLContext = object # type: ignore[misc,assignment] - +else: + try: + from ssl import SSLContext + except ImportError: # pragma: no cover + SSLContext = object # type: ignore[misc,assignment] __all__ = ( "BaseSite", diff --git a/aiohttp/worker.py b/aiohttp/worker.py index 9b307697336..8ed121ac955 100644 --- a/aiohttp/worker.py +++ b/aiohttp/worker.py @@ -6,7 +6,7 @@ import signal import sys from types import FrameType -from typing import Any, Awaitable, Callable, Optional, Union # noqa +from typing import TYPE_CHECKING, Any, Optional from gunicorn.config import AccessLogFormat as GunicornAccessLogFormat from gunicorn.workers import base @@ -17,13 +17,18 @@ from .web_app import Application from .web_log import AccessLogger -try: +if TYPE_CHECKING: import ssl SSLContext = ssl.SSLContext -except ImportError: # pragma: no cover - ssl = None # type: ignore[assignment] - SSLContext = object # type: ignore[misc,assignment] +else: + try: + import ssl + + SSLContext = ssl.SSLContext + except ImportError: # pragma: no cover + ssl = None # type: ignore[assignment] + SSLContext = object # type: ignore[misc,assignment] __all__ = ("GunicornWebWorker", "GunicornUVLoopWebWorker") diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index a1f3d944584..c4e10b44987 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -245,6 +245,7 @@ py pydantic pyenv pyflakes +pyright pytest Pytest Quickstart From b770b1ac2cde2ec77d234a04d771df4f7c573626 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Sun, 8 Dec 2024 15:30:15 +0000 Subject: [PATCH 08/23] Fix type of SSLContext for some static type checkers (#10099) (#10145) (cherry picked from commit 6200513d8fd34a820a4d10d238ca92d9f73ce7ee) Co-authored-by: AlanBogarin --- CHANGES/10099.bugfix.rst | 1 + CONTRIBUTORS.txt | 1 + aiohttp/client_exceptions.py | 10 +++++++--- aiohttp/client_reqrep.py | 12 ++++++++---- aiohttp/connector.py | 14 +++++++++----- aiohttp/web.py | 10 +++++++--- aiohttp/web_runner.py | 12 +++++++----- aiohttp/worker.py | 15 ++++++++++----- docs/spelling_wordlist.txt | 1 + 9 files changed, 51 insertions(+), 25 deletions(-) create mode 100644 CHANGES/10099.bugfix.rst diff --git a/CHANGES/10099.bugfix.rst b/CHANGES/10099.bugfix.rst new file mode 100644 index 00000000000..718420a6ad5 --- /dev/null +++ b/CHANGES/10099.bugfix.rst @@ -0,0 +1 @@ +Fixed type of ``SSLContext`` for some static type checkers (e.g. pyright). diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 6adb3b97fb1..ded6c463e40 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -9,6 +9,7 @@ Adam Mills Adrian Krupa Adrián Chaves Ahmed Tahri +Alan Bogarin Alan Tse Alec Hanefeld Alejandro Gómez diff --git a/aiohttp/client_exceptions.py b/aiohttp/client_exceptions.py index 667da8d5084..1d298e9a8cf 100644 --- a/aiohttp/client_exceptions.py +++ b/aiohttp/client_exceptions.py @@ -8,13 +8,17 @@ from .typedefs import StrOrURL -try: +if TYPE_CHECKING: import ssl SSLContext = ssl.SSLContext -except ImportError: # pragma: no cover - ssl = SSLContext = None # type: ignore[assignment] +else: + try: + import ssl + SSLContext = ssl.SSLContext + except ImportError: # pragma: no cover + ssl = SSLContext = None # type: ignore[assignment] if TYPE_CHECKING: from .client_reqrep import ClientResponse, ConnectionKey, Fingerprint, RequestInfo diff --git a/aiohttp/client_reqrep.py b/aiohttp/client_reqrep.py index e97c40ce0e5..43b48063c6e 100644 --- a/aiohttp/client_reqrep.py +++ b/aiohttp/client_reqrep.py @@ -72,12 +72,16 @@ RawHeaders, ) -try: +if TYPE_CHECKING: import ssl from ssl import SSLContext -except ImportError: # pragma: no cover - ssl = None # type: ignore[assignment] - SSLContext = object # type: ignore[misc,assignment] +else: + try: + import ssl + from ssl import SSLContext + except ImportError: # pragma: no cover + ssl = None # type: ignore[assignment] + SSLContext = object # type: ignore[misc,assignment] __all__ = ("ClientRequest", "ClientResponse", "RequestInfo", "Fingerprint") diff --git a/aiohttp/connector.py b/aiohttp/connector.py index 93bc2513b20..a9123f82bc0 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -60,14 +60,18 @@ ) from .resolver import DefaultResolver -try: +if TYPE_CHECKING: import ssl SSLContext = ssl.SSLContext -except ImportError: # pragma: no cover - ssl = None # type: ignore[assignment] - SSLContext = object # type: ignore[misc,assignment] - +else: + try: + import ssl + + SSLContext = ssl.SSLContext + except ImportError: # pragma: no cover + ssl = None # type: ignore[assignment] + SSLContext = object # type: ignore[misc,assignment] EMPTY_SCHEMA_SET = frozenset({""}) HTTP_SCHEMA_SET = frozenset({"http", "https"}) diff --git a/aiohttp/web.py b/aiohttp/web.py index f975b665331..d6ab6f6fad4 100644 --- a/aiohttp/web.py +++ b/aiohttp/web.py @@ -9,6 +9,7 @@ from contextlib import suppress from importlib import import_module from typing import ( + TYPE_CHECKING, Any, Awaitable, Callable, @@ -287,10 +288,13 @@ ) -try: +if TYPE_CHECKING: from ssl import SSLContext -except ImportError: # pragma: no cover - SSLContext = Any # type: ignore[misc,assignment] +else: + try: + from ssl import SSLContext + except ImportError: # pragma: no cover + SSLContext = object # type: ignore[misc,assignment] # Only display warning when using -Wdefault, -We, -X dev or similar. warnings.filterwarnings("ignore", category=NotAppKeyWarning, append=True) diff --git a/aiohttp/web_runner.py b/aiohttp/web_runner.py index f8933383435..bcfec727c84 100644 --- a/aiohttp/web_runner.py +++ b/aiohttp/web_runner.py @@ -3,7 +3,7 @@ import socket import warnings from abc import ABC, abstractmethod -from typing import Any, List, Optional, Set +from typing import TYPE_CHECKING, Any, List, Optional, Set from yarl import URL @@ -11,11 +11,13 @@ from .web_app import Application from .web_server import Server -try: +if TYPE_CHECKING: from ssl import SSLContext -except ImportError: - SSLContext = object # type: ignore[misc,assignment] - +else: + try: + from ssl import SSLContext + except ImportError: # pragma: no cover + SSLContext = object # type: ignore[misc,assignment] __all__ = ( "BaseSite", diff --git a/aiohttp/worker.py b/aiohttp/worker.py index 9b307697336..8ed121ac955 100644 --- a/aiohttp/worker.py +++ b/aiohttp/worker.py @@ -6,7 +6,7 @@ import signal import sys from types import FrameType -from typing import Any, Awaitable, Callable, Optional, Union # noqa +from typing import TYPE_CHECKING, Any, Optional from gunicorn.config import AccessLogFormat as GunicornAccessLogFormat from gunicorn.workers import base @@ -17,13 +17,18 @@ from .web_app import Application from .web_log import AccessLogger -try: +if TYPE_CHECKING: import ssl SSLContext = ssl.SSLContext -except ImportError: # pragma: no cover - ssl = None # type: ignore[assignment] - SSLContext = object # type: ignore[misc,assignment] +else: + try: + import ssl + + SSLContext = ssl.SSLContext + except ImportError: # pragma: no cover + ssl = None # type: ignore[assignment] + SSLContext = object # type: ignore[misc,assignment] __all__ = ("GunicornWebWorker", "GunicornUVLoopWebWorker") diff --git a/docs/spelling_wordlist.txt b/docs/spelling_wordlist.txt index a1f3d944584..c4e10b44987 100644 --- a/docs/spelling_wordlist.txt +++ b/docs/spelling_wordlist.txt @@ -245,6 +245,7 @@ py pydantic pyenv pyflakes +pyright pytest Pytest Quickstart From 51cdda86baead26080f774c400f81492b62022f6 Mon Sep 17 00:00:00 2001 From: Sam Bull Date: Mon, 9 Dec 2024 20:12:25 +0000 Subject: [PATCH 09/23] Add host parameter to aiohttp_server fixture (#10120) (#10121) (#10150) Co-authored-by: ChristianWBrock (cherry picked from commit 7f8e2d35ad4d0d5fed82721060bafd1bafa264b8) Co-authored-by: christianwbrock --- CHANGES/10120.feature.rst | 1 + aiohttp/pytest_plugin.py | 8 ++++++-- 2 files changed, 7 insertions(+), 2 deletions(-) create mode 100644 CHANGES/10120.feature.rst diff --git a/CHANGES/10120.feature.rst b/CHANGES/10120.feature.rst new file mode 100644 index 00000000000..98cee5650d6 --- /dev/null +++ b/CHANGES/10120.feature.rst @@ -0,0 +1 @@ +Added ``host`` parameter to ``aiohttp_server`` fixture -- by :user:`christianwbrock`. diff --git a/aiohttp/pytest_plugin.py b/aiohttp/pytest_plugin.py index 7ce60faa4a4..158fd684b7a 100644 --- a/aiohttp/pytest_plugin.py +++ b/aiohttp/pytest_plugin.py @@ -301,9 +301,13 @@ def aiohttp_server(loop: asyncio.AbstractEventLoop) -> Iterator[AiohttpServer]: servers = [] async def go( - app: Application, *, port: Optional[int] = None, **kwargs: Any + app: Application, + *, + host: str = "127.0.0.1", + port: Optional[int] = None, + **kwargs: Any, ) -> TestServer: - server = TestServer(app, port=port) + server = TestServer(app, host=host, port=port) await server.start_server(loop=loop, **kwargs) servers.append(server) return server From 7f389138e7ac33687c7c486cde2d2056f9b0f6fe Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 11 Dec 2024 15:50:34 +0100 Subject: [PATCH 10/23] [PR #10154/3f07b1a3 backport][3.11] Update StreamResponse.write annotation for strict-bytes (#10157) **This is a backport of PR #10154 as merged into master (3f07b1a38beffc350adbf51a605d27fe306de66c).** ## What do these changes do? Mypy will add a `--strict-bytes` flag. https://github.com/python/mypy/pull/18263 With that `bytearray` and `memoryview` are no longer subclasses of `bytes` and must be listed explicitly instead if they are supported. ## Are there changes in behavior for the user? -- ## Related issue number -- ## Checklist - [ ] I think the code is well written - [ ] Unit tests for the changes exist - [ ] Documentation reflects the changes - [ ] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [ ] Add a new news fragment into the `CHANGES/` folder * name it `..rst` (e.g. `588.bugfix.rst`) * if you don't have an issue number, change it to the pull request number after creating the PR * `.bugfix`: A bug fix for something the maintainers deemed an improper undesired behavior that got corrected to match pre-agreed expectations. * `.feature`: A new behavior, public APIs. That sort of stuff. * `.deprecation`: A declaration of future API removals and breaking changes in behavior. * `.breaking`: When something public is removed in a breaking way. Could be deprecated in an earlier release. * `.doc`: Notable updates to the documentation structure or build process. * `.packaging`: Notes for downstreams about unobvious side effects and tooling. Changes in the test invocation considerations and runtime assumptions. * `.contrib`: Stuff that affects the contributor experience. e.g. Running tests, building the docs, setting up the development environment. * `.misc`: Changes that are hard to assign to any of the above categories. * Make sure to use full sentences with correct case and punctuation, for example: ```rst Fixed issue with non-ascii contents in doctest text files -- by :user:`contributor-gh-handle`. ``` Use the past tense or the present tense a non-imperative mood, referring to what's changed compared to the last released version of this project. Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> --- CHANGES/10154.bugfix.rst | 1 + aiohttp/abc.py | 3 ++- aiohttp/http_writer.py | 8 ++++++-- aiohttp/web_response.py | 2 +- 4 files changed, 10 insertions(+), 4 deletions(-) create mode 100644 CHANGES/10154.bugfix.rst diff --git a/CHANGES/10154.bugfix.rst b/CHANGES/10154.bugfix.rst new file mode 100644 index 00000000000..382d9e56e6c --- /dev/null +++ b/CHANGES/10154.bugfix.rst @@ -0,0 +1 @@ +Updated :meth:`aiohttp.web.StreamResponse.write` annotation to also allow :class:`bytearray` and :class:`memoryview` as inputs -- by :user:`cdce8p`. diff --git a/aiohttp/abc.py b/aiohttp/abc.py index d6f9f782b0f..989f0a561ff 100644 --- a/aiohttp/abc.py +++ b/aiohttp/abc.py @@ -17,6 +17,7 @@ Optional, Tuple, TypedDict, + Union, ) from multidict import CIMultiDict @@ -200,7 +201,7 @@ class AbstractStreamWriter(ABC): length: Optional[int] = 0 @abstractmethod - async def write(self, chunk: bytes) -> None: + async def write(self, chunk: Union[bytes, bytearray, memoryview]) -> None: """Write chunk into stream.""" @abstractmethod diff --git a/aiohttp/http_writer.py b/aiohttp/http_writer.py index edd19ed65da..28b14f7a791 100644 --- a/aiohttp/http_writer.py +++ b/aiohttp/http_writer.py @@ -72,7 +72,7 @@ def enable_compression( ) -> None: self._compress = ZLibCompressor(encoding=encoding, strategy=strategy) - def _write(self, chunk: bytes) -> None: + def _write(self, chunk: Union[bytes, bytearray, memoryview]) -> None: size = len(chunk) self.buffer_size += size self.output_size += size @@ -93,7 +93,11 @@ def _writelines(self, chunks: Iterable[bytes]) -> None: transport.write(b"".join(chunks)) async def write( - self, chunk: bytes, *, drain: bool = True, LIMIT: int = 0x10000 + self, + chunk: Union[bytes, bytearray, memoryview], + *, + drain: bool = True, + LIMIT: int = 0x10000, ) -> None: """Writes chunk of data to a stream. diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py index cd2be24f1a3..e498a905caf 100644 --- a/aiohttp/web_response.py +++ b/aiohttp/web_response.py @@ -537,7 +537,7 @@ async def _write_headers(self) -> None: status_line = f"HTTP/{version[0]}.{version[1]} {self._status} {self._reason}" await writer.write_headers(status_line, self._headers) - async def write(self, data: bytes) -> None: + async def write(self, data: Union[bytes, bytearray, memoryview]) -> None: assert isinstance( data, (bytes, bytearray, memoryview) ), "data argument must be byte-ish (%r)" % type(data) From 5d9d83065c8ce9a4fb6548cf8d5993f793315010 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 11 Dec 2024 15:50:56 +0100 Subject: [PATCH 11/23] [PR #10154/3f07b1a3 backport][3.12] Update StreamResponse.write annotation for strict-bytes (#10158) **This is a backport of PR #10154 as merged into master (3f07b1a38beffc350adbf51a605d27fe306de66c).** ## What do these changes do? Mypy will add a `--strict-bytes` flag. https://github.com/python/mypy/pull/18263 With that `bytearray` and `memoryview` are no longer subclasses of `bytes` and must be listed explicitly instead if they are supported. ## Are there changes in behavior for the user? -- ## Related issue number -- ## Checklist - [ ] I think the code is well written - [ ] Unit tests for the changes exist - [ ] Documentation reflects the changes - [ ] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [ ] Add a new news fragment into the `CHANGES/` folder * name it `..rst` (e.g. `588.bugfix.rst`) * if you don't have an issue number, change it to the pull request number after creating the PR * `.bugfix`: A bug fix for something the maintainers deemed an improper undesired behavior that got corrected to match pre-agreed expectations. * `.feature`: A new behavior, public APIs. That sort of stuff. * `.deprecation`: A declaration of future API removals and breaking changes in behavior. * `.breaking`: When something public is removed in a breaking way. Could be deprecated in an earlier release. * `.doc`: Notable updates to the documentation structure or build process. * `.packaging`: Notes for downstreams about unobvious side effects and tooling. Changes in the test invocation considerations and runtime assumptions. * `.contrib`: Stuff that affects the contributor experience. e.g. Running tests, building the docs, setting up the development environment. * `.misc`: Changes that are hard to assign to any of the above categories. * Make sure to use full sentences with correct case and punctuation, for example: ```rst Fixed issue with non-ascii contents in doctest text files -- by :user:`contributor-gh-handle`. ``` Use the past tense or the present tense a non-imperative mood, referring to what's changed compared to the last released version of this project. Co-authored-by: Marc Mueller <30130371+cdce8p@users.noreply.github.com> --- CHANGES/10154.bugfix.rst | 1 + aiohttp/abc.py | 3 ++- aiohttp/http_writer.py | 8 ++++++-- aiohttp/web_response.py | 2 +- 4 files changed, 10 insertions(+), 4 deletions(-) create mode 100644 CHANGES/10154.bugfix.rst diff --git a/CHANGES/10154.bugfix.rst b/CHANGES/10154.bugfix.rst new file mode 100644 index 00000000000..382d9e56e6c --- /dev/null +++ b/CHANGES/10154.bugfix.rst @@ -0,0 +1 @@ +Updated :meth:`aiohttp.web.StreamResponse.write` annotation to also allow :class:`bytearray` and :class:`memoryview` as inputs -- by :user:`cdce8p`. diff --git a/aiohttp/abc.py b/aiohttp/abc.py index d6f9f782b0f..989f0a561ff 100644 --- a/aiohttp/abc.py +++ b/aiohttp/abc.py @@ -17,6 +17,7 @@ Optional, Tuple, TypedDict, + Union, ) from multidict import CIMultiDict @@ -200,7 +201,7 @@ class AbstractStreamWriter(ABC): length: Optional[int] = 0 @abstractmethod - async def write(self, chunk: bytes) -> None: + async def write(self, chunk: Union[bytes, bytearray, memoryview]) -> None: """Write chunk into stream.""" @abstractmethod diff --git a/aiohttp/http_writer.py b/aiohttp/http_writer.py index edd19ed65da..28b14f7a791 100644 --- a/aiohttp/http_writer.py +++ b/aiohttp/http_writer.py @@ -72,7 +72,7 @@ def enable_compression( ) -> None: self._compress = ZLibCompressor(encoding=encoding, strategy=strategy) - def _write(self, chunk: bytes) -> None: + def _write(self, chunk: Union[bytes, bytearray, memoryview]) -> None: size = len(chunk) self.buffer_size += size self.output_size += size @@ -93,7 +93,11 @@ def _writelines(self, chunks: Iterable[bytes]) -> None: transport.write(b"".join(chunks)) async def write( - self, chunk: bytes, *, drain: bool = True, LIMIT: int = 0x10000 + self, + chunk: Union[bytes, bytearray, memoryview], + *, + drain: bool = True, + LIMIT: int = 0x10000, ) -> None: """Writes chunk of data to a stream. diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py index cd2be24f1a3..e498a905caf 100644 --- a/aiohttp/web_response.py +++ b/aiohttp/web_response.py @@ -537,7 +537,7 @@ async def _write_headers(self) -> None: status_line = f"HTTP/{version[0]}.{version[1]} {self._status} {self._reason}" await writer.write_headers(status_line, self._headers) - async def write(self, data: bytes) -> None: + async def write(self, data: Union[bytes, bytearray, memoryview]) -> None: assert isinstance( data, (bytes, bytearray, memoryview) ), "data argument must be byte-ish (%r)" % type(data) From f38d51ca28e504cc8637093d53734abc998777ab Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Thu, 12 Dec 2024 15:38:44 +0000 Subject: [PATCH 12/23] [PR #10146/a818e51c backport][3.12] Raise `TypeError` when setting `StreamResponse.last_modified` to an unsupported type (#10161) --- CHANGES/10146.misc.rst | 1 + aiohttp/web_response.py | 3 +++ tests/test_web_response.py | 7 +++++++ 3 files changed, 11 insertions(+) create mode 100644 CHANGES/10146.misc.rst diff --git a/CHANGES/10146.misc.rst b/CHANGES/10146.misc.rst new file mode 100644 index 00000000000..bee4ef68fb3 --- /dev/null +++ b/CHANGES/10146.misc.rst @@ -0,0 +1 @@ +Setting :attr:`aiohttp.web.StreamResponse.last_modified` to an unsupported type will now raise :exc:`TypeError` instead of silently failing -- by :user:`bdraco`. diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py index e498a905caf..e2351ddf7b7 100644 --- a/aiohttp/web_response.py +++ b/aiohttp/web_response.py @@ -370,6 +370,9 @@ def last_modified( ) elif isinstance(value, str): self._headers[hdrs.LAST_MODIFIED] = value + else: + msg = f"Unsupported type for last_modified: {type(value).__name__}" + raise TypeError(msg) @property def etag(self) -> Optional[ETag]: diff --git a/tests/test_web_response.py b/tests/test_web_response.py index f4acf23f61b..24743e64635 100644 --- a/tests/test_web_response.py +++ b/tests/test_web_response.py @@ -255,6 +255,13 @@ def test_last_modified_reset() -> None: assert resp.last_modified is None +def test_last_modified_invalid_type() -> None: + resp = StreamResponse() + + with pytest.raises(TypeError, match="Unsupported type for last_modified: object"): + resp.last_modified = object() # type: ignore[assignment] + + @pytest.mark.parametrize( ["header_val", "expected"], [ From 99a373737135d924180121ca01500e479b52ba96 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Thu, 12 Dec 2024 16:42:06 +0100 Subject: [PATCH 13/23] [PR #10156/00700458 backport][3.12] Add ALPN extension to client SSL Context (#10164) **This is a backport of PR #10156 as merged into master (00700458eb7741f15861a8616dbf77a0d82dc31f).** ## What do these changes do? Add "http/1.1" ALPN extension to aiohttp client's SSL Context. ## Are there changes in behavior for the user? ## Is it a substantial burden for the maintainers to support this? ## Related issue number Fixes #10152 ## Checklist - [x] I think the code is well written - [x] Unit tests for the changes exist - [ ] Documentation reflects the changes - [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES/` folder Co-authored-by: Cycloctane --- CHANGES/10156.feature.rst | 3 +++ CONTRIBUTORS.txt | 1 + aiohttp/connector.py | 18 ++++++++++-------- tests/test_client_functional.py | 24 ++++++++++++++++++++++++ 4 files changed, 38 insertions(+), 8 deletions(-) create mode 100644 CHANGES/10156.feature.rst diff --git a/CHANGES/10156.feature.rst b/CHANGES/10156.feature.rst new file mode 100644 index 00000000000..0ff6b6b8bd8 --- /dev/null +++ b/CHANGES/10156.feature.rst @@ -0,0 +1,3 @@ +Enabled ALPN on default SSL contexts. This improves compatibility with some +proxies which don't work without this extension. +-- by :user:`Cycloctane`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 94d003a1719..035436c0426 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -366,6 +366,7 @@ William S. Wilson Ong wouter bolsterlee Xavier Halloran +Xi Rui Xiang Li Yang Zhou Yannick Koechlin diff --git a/aiohttp/connector.py b/aiohttp/connector.py index a9123f82bc0..7e0986df657 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -780,14 +780,16 @@ def _make_ssl_context(verified: bool) -> SSLContext: # No ssl support return None if verified: - return ssl.create_default_context() - sslcontext = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) - sslcontext.options |= ssl.OP_NO_SSLv2 - sslcontext.options |= ssl.OP_NO_SSLv3 - sslcontext.check_hostname = False - sslcontext.verify_mode = ssl.CERT_NONE - sslcontext.options |= ssl.OP_NO_COMPRESSION - sslcontext.set_default_verify_paths() + sslcontext = ssl.create_default_context() + else: + sslcontext = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + sslcontext.options |= ssl.OP_NO_SSLv2 + sslcontext.options |= ssl.OP_NO_SSLv3 + sslcontext.check_hostname = False + sslcontext.verify_mode = ssl.CERT_NONE + sslcontext.options |= ssl.OP_NO_COMPRESSION + sslcontext.set_default_verify_paths() + sslcontext.set_alpn_protocols(("http/1.1",)) return sslcontext diff --git a/tests/test_client_functional.py b/tests/test_client_functional.py index b34ccdb600d..05af9ae25ad 100644 --- a/tests/test_client_functional.py +++ b/tests/test_client_functional.py @@ -603,6 +603,30 @@ async def handler(request): assert txt == "Test message" +async def test_ssl_client_alpn( + aiohttp_server: AiohttpServer, + aiohttp_client: AiohttpClient, + ssl_ctx: ssl.SSLContext, +) -> None: + + async def handler(request: web.Request) -> web.Response: + assert request.transport is not None + sslobj = request.transport.get_extra_info("ssl_object") + return web.Response(text=sslobj.selected_alpn_protocol()) + + app = web.Application() + app.router.add_route("GET", "/", handler) + ssl_ctx.set_alpn_protocols(("http/1.1",)) + server = await aiohttp_server(app, ssl=ssl_ctx) + + connector = aiohttp.TCPConnector(ssl=False) + client = await aiohttp_client(server, connector=connector) + resp = await client.get("/") + assert resp.status == 200 + txt = await resp.text() + assert txt == "http/1.1" + + async def test_tcp_connector_fingerprint_ok( aiohttp_server, aiohttp_client, From 3680479df7b72d1fcef1c76abf80727553fe573f Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Thu, 12 Dec 2024 16:42:26 +0100 Subject: [PATCH 14/23] [PR #10156/00700458 backport][3.11] Add ALPN extension to client SSL Context (#10163) **This is a backport of PR #10156 as merged into master (00700458eb7741f15861a8616dbf77a0d82dc31f).** ## What do these changes do? Add "http/1.1" ALPN extension to aiohttp client's SSL Context. ## Are there changes in behavior for the user? ## Is it a substantial burden for the maintainers to support this? ## Related issue number Fixes #10152 ## Checklist - [x] I think the code is well written - [x] Unit tests for the changes exist - [ ] Documentation reflects the changes - [x] If you provide code modification, please add yourself to `CONTRIBUTORS.txt` * The format is <Name> <Surname>. * Please keep alphabetical order, the file is sorted by names. - [x] Add a new news fragment into the `CHANGES/` folder Co-authored-by: Cycloctane --- CHANGES/10156.feature.rst | 3 +++ CONTRIBUTORS.txt | 1 + aiohttp/connector.py | 18 ++++++++++-------- tests/test_client_functional.py | 24 ++++++++++++++++++++++++ 4 files changed, 38 insertions(+), 8 deletions(-) create mode 100644 CHANGES/10156.feature.rst diff --git a/CHANGES/10156.feature.rst b/CHANGES/10156.feature.rst new file mode 100644 index 00000000000..0ff6b6b8bd8 --- /dev/null +++ b/CHANGES/10156.feature.rst @@ -0,0 +1,3 @@ +Enabled ALPN on default SSL contexts. This improves compatibility with some +proxies which don't work without this extension. +-- by :user:`Cycloctane`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index ded6c463e40..5acc4de44fc 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -365,6 +365,7 @@ William S. Wilson Ong wouter bolsterlee Xavier Halloran +Xi Rui Xiang Li Yang Zhou Yannick Koechlin diff --git a/aiohttp/connector.py b/aiohttp/connector.py index a9123f82bc0..7e0986df657 100644 --- a/aiohttp/connector.py +++ b/aiohttp/connector.py @@ -780,14 +780,16 @@ def _make_ssl_context(verified: bool) -> SSLContext: # No ssl support return None if verified: - return ssl.create_default_context() - sslcontext = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) - sslcontext.options |= ssl.OP_NO_SSLv2 - sslcontext.options |= ssl.OP_NO_SSLv3 - sslcontext.check_hostname = False - sslcontext.verify_mode = ssl.CERT_NONE - sslcontext.options |= ssl.OP_NO_COMPRESSION - sslcontext.set_default_verify_paths() + sslcontext = ssl.create_default_context() + else: + sslcontext = ssl.SSLContext(ssl.PROTOCOL_TLS_CLIENT) + sslcontext.options |= ssl.OP_NO_SSLv2 + sslcontext.options |= ssl.OP_NO_SSLv3 + sslcontext.check_hostname = False + sslcontext.verify_mode = ssl.CERT_NONE + sslcontext.options |= ssl.OP_NO_COMPRESSION + sslcontext.set_default_verify_paths() + sslcontext.set_alpn_protocols(("http/1.1",)) return sslcontext diff --git a/tests/test_client_functional.py b/tests/test_client_functional.py index b34ccdb600d..05af9ae25ad 100644 --- a/tests/test_client_functional.py +++ b/tests/test_client_functional.py @@ -603,6 +603,30 @@ async def handler(request): assert txt == "Test message" +async def test_ssl_client_alpn( + aiohttp_server: AiohttpServer, + aiohttp_client: AiohttpClient, + ssl_ctx: ssl.SSLContext, +) -> None: + + async def handler(request: web.Request) -> web.Response: + assert request.transport is not None + sslobj = request.transport.get_extra_info("ssl_object") + return web.Response(text=sslobj.selected_alpn_protocol()) + + app = web.Application() + app.router.add_route("GET", "/", handler) + ssl_ctx.set_alpn_protocols(("http/1.1",)) + server = await aiohttp_server(app, ssl=ssl_ctx) + + connector = aiohttp.TCPConnector(ssl=False) + client = await aiohttp_client(server, connector=connector) + resp = await client.get("/") + assert resp.status == 200 + txt = await resp.text() + assert txt == "http/1.1" + + async def test_tcp_connector_fingerprint_ok( aiohttp_server, aiohttp_client, From c80be677c02d9bb63b4e96a3fd5b8d29bd11f0c6 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Tue, 17 Dec 2024 18:37:44 +0000 Subject: [PATCH 15/23] [PR #10151/7c12b1a9 backport][3.11] Fix infinite callback loop when time is not moving forward (#10173) Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com> Fixes #123'). --> Fixes #10149. --- CHANGES/10149.misc.rst | 4 ++++ aiohttp/web_protocol.py | 2 +- tests/test_web_functional.py | 38 ++++++++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 CHANGES/10149.misc.rst diff --git a/CHANGES/10149.misc.rst b/CHANGES/10149.misc.rst new file mode 100644 index 00000000000..61765a50fcf --- /dev/null +++ b/CHANGES/10149.misc.rst @@ -0,0 +1,4 @@ +Fixed an infinite loop that can occur when using aiohttp in combination +with `async-solipsism`_ -- by :user:`bmerry`. + +.. _async-solipsism: https://github.com/bmerry/async-solipsism diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index e8bb41abf97..3306b86bded 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -458,7 +458,7 @@ def _process_keepalive(self) -> None: loop = self._loop now = loop.time() close_time = self._next_keepalive_close_time - if now <= close_time: + if now < close_time: # Keep alive close check fired too early, reschedule self._keepalive_handle = loop.call_at(close_time, self._process_keepalive) return diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py index a3a990141a1..e4979851300 100644 --- a/tests/test_web_functional.py +++ b/tests/test_web_functional.py @@ -2324,3 +2324,41 @@ async def handler(request: web.Request) -> web.Response: # Make 2nd request which will hit the race condition. async with client.get("/") as resp: assert resp.status == 200 + + +async def test_keepalive_expires_on_time(aiohttp_client: AiohttpClient) -> None: + """Test that the keepalive handle expires on time.""" + + async def handler(request: web.Request) -> web.Response: + body = await request.read() + assert b"" == body + return web.Response(body=b"OK") + + app = web.Application() + app.router.add_route("GET", "/", handler) + + connector = aiohttp.TCPConnector(limit=1) + client = await aiohttp_client(app, connector=connector) + + loop = asyncio.get_running_loop() + now = loop.time() + + # Patch loop time so we can control when the keepalive timeout is processed + with mock.patch.object(loop, "time") as loop_time_mock: + loop_time_mock.return_value = now + resp1 = await client.get("/") + await resp1.read() + request_handler = client.server.handler.connections[0] + + # Ensure the keep alive handle is set + assert request_handler._keepalive_handle is not None + + # Set the loop time to exactly the keepalive timeout + loop_time_mock.return_value = request_handler._next_keepalive_close_time + + # sleep twice to ensure the keep alive timeout is processed + await asyncio.sleep(0) + await asyncio.sleep(0) + + # Ensure the keep alive handle expires + assert request_handler._keepalive_handle is None From 5afac5580cd40f1b20b56cc5892aa9c6125a482a Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Tue, 17 Dec 2024 18:45:53 +0000 Subject: [PATCH 16/23] [PR #10151/7c12b1a9 backport][3.12] Fix infinite callback loop when time is not moving forward (#10174) Co-authored-by: Bruce Merry <1963944+bmerry@users.noreply.github.com> Fixes #123'). --> Fixes #10149. --- CHANGES/10149.misc.rst | 4 ++++ aiohttp/web_protocol.py | 2 +- tests/test_web_functional.py | 38 ++++++++++++++++++++++++++++++++++++ 3 files changed, 43 insertions(+), 1 deletion(-) create mode 100644 CHANGES/10149.misc.rst diff --git a/CHANGES/10149.misc.rst b/CHANGES/10149.misc.rst new file mode 100644 index 00000000000..61765a50fcf --- /dev/null +++ b/CHANGES/10149.misc.rst @@ -0,0 +1,4 @@ +Fixed an infinite loop that can occur when using aiohttp in combination +with `async-solipsism`_ -- by :user:`bmerry`. + +.. _async-solipsism: https://github.com/bmerry/async-solipsism diff --git a/aiohttp/web_protocol.py b/aiohttp/web_protocol.py index e8bb41abf97..3306b86bded 100644 --- a/aiohttp/web_protocol.py +++ b/aiohttp/web_protocol.py @@ -458,7 +458,7 @@ def _process_keepalive(self) -> None: loop = self._loop now = loop.time() close_time = self._next_keepalive_close_time - if now <= close_time: + if now < close_time: # Keep alive close check fired too early, reschedule self._keepalive_handle = loop.call_at(close_time, self._process_keepalive) return diff --git a/tests/test_web_functional.py b/tests/test_web_functional.py index a3a990141a1..e4979851300 100644 --- a/tests/test_web_functional.py +++ b/tests/test_web_functional.py @@ -2324,3 +2324,41 @@ async def handler(request: web.Request) -> web.Response: # Make 2nd request which will hit the race condition. async with client.get("/") as resp: assert resp.status == 200 + + +async def test_keepalive_expires_on_time(aiohttp_client: AiohttpClient) -> None: + """Test that the keepalive handle expires on time.""" + + async def handler(request: web.Request) -> web.Response: + body = await request.read() + assert b"" == body + return web.Response(body=b"OK") + + app = web.Application() + app.router.add_route("GET", "/", handler) + + connector = aiohttp.TCPConnector(limit=1) + client = await aiohttp_client(app, connector=connector) + + loop = asyncio.get_running_loop() + now = loop.time() + + # Patch loop time so we can control when the keepalive timeout is processed + with mock.patch.object(loop, "time") as loop_time_mock: + loop_time_mock.return_value = now + resp1 = await client.get("/") + await resp1.read() + request_handler = client.server.handler.connections[0] + + # Ensure the keep alive handle is set + assert request_handler._keepalive_handle is not None + + # Set the loop time to exactly the keepalive timeout + loop_time_mock.return_value = request_handler._next_keepalive_close_time + + # sleep twice to ensure the keep alive timeout is processed + await asyncio.sleep(0) + await asyncio.sleep(0) + + # Ensure the keep alive handle expires + assert request_handler._keepalive_handle is None From 8c96a62fca08313431afa6ff56f35626c1b97e8d Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Tue, 17 Dec 2024 11:50:53 -0700 Subject: [PATCH 17/23] [PR #10093/7b5d54a backport][3.11] Use `quote_cookie` setting from ClientSession's cookiejar in `tmp_cookie_jar` (#10175) Co-authored-by: pre-commit-ci[bot] Co-authored-by: Sam Bull Co-authored-by: Cycloctane --- CHANGES/10093.bugfix.rst | 2 ++ aiohttp/abc.py | 5 +++++ aiohttp/client.py | 4 +++- aiohttp/cookiejar.py | 8 ++++++++ tests/test_client_session.py | 23 ++++++++++++++++++++--- tests/test_cookiejar.py | 1 + 6 files changed, 39 insertions(+), 4 deletions(-) create mode 100644 CHANGES/10093.bugfix.rst diff --git a/CHANGES/10093.bugfix.rst b/CHANGES/10093.bugfix.rst new file mode 100644 index 00000000000..4d7076115d9 --- /dev/null +++ b/CHANGES/10093.bugfix.rst @@ -0,0 +1,2 @@ +Update :py:meth:`~aiohttp.ClientSession.request` to reuse the ``quote_cookie`` setting from ``ClientSession._cookie_jar`` when processing cookies parameter. +-- by :user:`Cycloctane`. diff --git a/aiohttp/abc.py b/aiohttp/abc.py index 989f0a561ff..5794a9108b0 100644 --- a/aiohttp/abc.py +++ b/aiohttp/abc.py @@ -176,6 +176,11 @@ class AbstractCookieJar(Sized, IterableBase): def __init__(self, *, loop: Optional[asyncio.AbstractEventLoop] = None) -> None: self._loop = loop or asyncio.get_running_loop() + @property + @abstractmethod + def quote_cookie(self) -> bool: + """Return True if cookies should be quoted.""" + @abstractmethod def clear(self, predicate: Optional[ClearCookiePredicate] = None) -> None: """Clear all cookies if no predicate is passed.""" diff --git a/aiohttp/client.py b/aiohttp/client.py index e04a6ff989a..3b1dc08544f 100644 --- a/aiohttp/client.py +++ b/aiohttp/client.py @@ -658,7 +658,9 @@ async def _request( all_cookies = self._cookie_jar.filter_cookies(url) if cookies is not None: - tmp_cookie_jar = CookieJar() + tmp_cookie_jar = CookieJar( + quote_cookie=self._cookie_jar.quote_cookie + ) tmp_cookie_jar.update_cookies(cookies) req_cookies = tmp_cookie_jar.filter_cookies(url) if req_cookies: diff --git a/aiohttp/cookiejar.py b/aiohttp/cookiejar.py index ef04bda5ad6..f6b9a921767 100644 --- a/aiohttp/cookiejar.py +++ b/aiohttp/cookiejar.py @@ -117,6 +117,10 @@ def __init__( self._expire_heap: List[Tuple[float, Tuple[str, str, str]]] = [] self._expirations: Dict[Tuple[str, str, str], float] = {} + @property + def quote_cookie(self) -> bool: + return self._quote_cookie + def save(self, file_path: PathLike) -> None: file_path = pathlib.Path(file_path) with file_path.open(mode="wb") as f: @@ -474,6 +478,10 @@ def __iter__(self) -> "Iterator[Morsel[str]]": def __len__(self) -> int: return 0 + @property + def quote_cookie(self) -> bool: + return True + def clear(self, predicate: Optional[ClearCookiePredicate] = None) -> None: pass diff --git a/tests/test_client_session.py b/tests/test_client_session.py index 65f80b6abe9..6309c5daf2e 100644 --- a/tests/test_client_session.py +++ b/tests/test_client_session.py @@ -15,13 +15,14 @@ from yarl import URL import aiohttp -from aiohttp import client, hdrs, web +from aiohttp import CookieJar, client, hdrs, web from aiohttp.client import ClientSession from aiohttp.client_proto import ResponseHandler from aiohttp.client_reqrep import ClientRequest from aiohttp.connector import BaseConnector, Connection, TCPConnector, UnixConnector from aiohttp.helpers import DEBUG from aiohttp.http import RawResponseMessage +from aiohttp.pytest_plugin import AiohttpServer from aiohttp.test_utils import make_mocked_coro from aiohttp.tracing import Trace @@ -634,8 +635,24 @@ async def handler(request): assert resp_cookies["response"].value == "resp_value" -async def test_session_default_version(loop) -> None: - session = aiohttp.ClientSession(loop=loop) +async def test_cookies_with_not_quoted_cookie_jar( + aiohttp_server: AiohttpServer, +) -> None: + async def handler(_: web.Request) -> web.Response: + return web.Response() + + app = web.Application() + app.router.add_route("GET", "/", handler) + server = await aiohttp_server(app) + jar = CookieJar(quote_cookie=False) + cookies = {"name": "val=foobar"} + async with aiohttp.ClientSession(cookie_jar=jar) as sess: + resp = await sess.request("GET", server.make_url("/"), cookies=cookies) + assert resp.request_info.headers.get("Cookie", "") == "name=val=foobar" + + +async def test_session_default_version(loop: asyncio.AbstractEventLoop) -> None: + session = aiohttp.ClientSession() assert session.version == aiohttp.HttpVersion11 await session.close() diff --git a/tests/test_cookiejar.py b/tests/test_cookiejar.py index bdcf54fa796..0b440bc2ca6 100644 --- a/tests/test_cookiejar.py +++ b/tests/test_cookiejar.py @@ -807,6 +807,7 @@ async def make_jar(): async def test_dummy_cookie_jar() -> None: cookie = SimpleCookie("foo=bar; Domain=example.com;") dummy_jar = DummyCookieJar() + assert dummy_jar.quote_cookie is True assert len(dummy_jar) == 0 dummy_jar.update_cookies(cookie) assert len(dummy_jar) == 0 From 86e21404c6681704f45aa31401959053a1b3b0df Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Tue, 17 Dec 2024 12:11:55 -0700 Subject: [PATCH 18/23] [PR #10093/7b5d54a backport][3.12] Use `quote_cookie` setting from ClientSession's cookiejar in `tmp_cookie_jar` (#10176) Co-authored-by: pre-commit-ci[bot] Co-authored-by: Sam Bull Co-authored-by: Cycloctane --- CHANGES/10093.bugfix.rst | 2 ++ aiohttp/abc.py | 5 +++++ aiohttp/client.py | 4 +++- aiohttp/cookiejar.py | 8 ++++++++ tests/test_client_session.py | 23 ++++++++++++++++++++--- tests/test_cookiejar.py | 1 + 6 files changed, 39 insertions(+), 4 deletions(-) create mode 100644 CHANGES/10093.bugfix.rst diff --git a/CHANGES/10093.bugfix.rst b/CHANGES/10093.bugfix.rst new file mode 100644 index 00000000000..4d7076115d9 --- /dev/null +++ b/CHANGES/10093.bugfix.rst @@ -0,0 +1,2 @@ +Update :py:meth:`~aiohttp.ClientSession.request` to reuse the ``quote_cookie`` setting from ``ClientSession._cookie_jar`` when processing cookies parameter. +-- by :user:`Cycloctane`. diff --git a/aiohttp/abc.py b/aiohttp/abc.py index 989f0a561ff..5794a9108b0 100644 --- a/aiohttp/abc.py +++ b/aiohttp/abc.py @@ -176,6 +176,11 @@ class AbstractCookieJar(Sized, IterableBase): def __init__(self, *, loop: Optional[asyncio.AbstractEventLoop] = None) -> None: self._loop = loop or asyncio.get_running_loop() + @property + @abstractmethod + def quote_cookie(self) -> bool: + """Return True if cookies should be quoted.""" + @abstractmethod def clear(self, predicate: Optional[ClearCookiePredicate] = None) -> None: """Clear all cookies if no predicate is passed.""" diff --git a/aiohttp/client.py b/aiohttp/client.py index 7539310aa8a..fbf691e89d1 100644 --- a/aiohttp/client.py +++ b/aiohttp/client.py @@ -656,7 +656,9 @@ async def _request( all_cookies = self._cookie_jar.filter_cookies(url) if cookies is not None: - tmp_cookie_jar = CookieJar() + tmp_cookie_jar = CookieJar( + quote_cookie=self._cookie_jar.quote_cookie + ) tmp_cookie_jar.update_cookies(cookies) req_cookies = tmp_cookie_jar.filter_cookies(url) if req_cookies: diff --git a/aiohttp/cookiejar.py b/aiohttp/cookiejar.py index ef04bda5ad6..f6b9a921767 100644 --- a/aiohttp/cookiejar.py +++ b/aiohttp/cookiejar.py @@ -117,6 +117,10 @@ def __init__( self._expire_heap: List[Tuple[float, Tuple[str, str, str]]] = [] self._expirations: Dict[Tuple[str, str, str], float] = {} + @property + def quote_cookie(self) -> bool: + return self._quote_cookie + def save(self, file_path: PathLike) -> None: file_path = pathlib.Path(file_path) with file_path.open(mode="wb") as f: @@ -474,6 +478,10 @@ def __iter__(self) -> "Iterator[Morsel[str]]": def __len__(self) -> int: return 0 + @property + def quote_cookie(self) -> bool: + return True + def clear(self, predicate: Optional[ClearCookiePredicate] = None) -> None: pass diff --git a/tests/test_client_session.py b/tests/test_client_session.py index a2c4833b83e..548af5db551 100644 --- a/tests/test_client_session.py +++ b/tests/test_client_session.py @@ -15,13 +15,14 @@ from yarl import URL import aiohttp -from aiohttp import client, hdrs, web +from aiohttp import CookieJar, client, hdrs, web from aiohttp.client import ClientSession from aiohttp.client_proto import ResponseHandler from aiohttp.client_reqrep import ClientRequest from aiohttp.connector import BaseConnector, Connection, TCPConnector, UnixConnector from aiohttp.helpers import DEBUG from aiohttp.http import RawResponseMessage +from aiohttp.pytest_plugin import AiohttpServer from aiohttp.test_utils import make_mocked_coro from aiohttp.tracing import Trace @@ -634,8 +635,24 @@ async def handler(request): assert resp_cookies["response"].value == "resp_value" -async def test_session_default_version(loop) -> None: - session = aiohttp.ClientSession(loop=loop) +async def test_cookies_with_not_quoted_cookie_jar( + aiohttp_server: AiohttpServer, +) -> None: + async def handler(_: web.Request) -> web.Response: + return web.Response() + + app = web.Application() + app.router.add_route("GET", "/", handler) + server = await aiohttp_server(app) + jar = CookieJar(quote_cookie=False) + cookies = {"name": "val=foobar"} + async with aiohttp.ClientSession(cookie_jar=jar) as sess: + resp = await sess.request("GET", server.make_url("/"), cookies=cookies) + assert resp.request_info.headers.get("Cookie", "") == "name=val=foobar" + + +async def test_session_default_version(loop: asyncio.AbstractEventLoop) -> None: + session = aiohttp.ClientSession() assert session.version == aiohttp.HttpVersion11 await session.close() diff --git a/tests/test_cookiejar.py b/tests/test_cookiejar.py index bdcf54fa796..0b440bc2ca6 100644 --- a/tests/test_cookiejar.py +++ b/tests/test_cookiejar.py @@ -807,6 +807,7 @@ async def make_jar(): async def test_dummy_cookie_jar() -> None: cookie = SimpleCookie("foo=bar; Domain=example.com;") dummy_jar = DummyCookieJar() + assert dummy_jar.quote_cookie is True assert len(dummy_jar) == 0 dummy_jar.update_cookies(cookie) assert len(dummy_jar) == 0 From d7e4e61607fb3e16ea2e93d4044b0538bae22d26 Mon Sep 17 00:00:00 2001 From: "patchback[bot]" <45432694+patchback[bot]@users.noreply.github.com> Date: Wed, 18 Dec 2024 12:01:32 +0000 Subject: [PATCH 19/23] [PR #10172/e45c3b8e backport][3.12] Ensure Response is True even when map is empty (#10177) **This is a backport of PR #10172 as merged into master (e45c3b8ea68e879493e6593d1097155ecc1878e2).** Fixes #10119 Technically a breaking change, but I can't imagine anyone depending on this. Co-authored-by: Robin --- CHANGES/10119.bugfix.rst | 1 + aiohttp/web_response.py | 3 +++ tests/test_web_request.py | 1 + tests/test_web_response.py | 1 + 4 files changed, 6 insertions(+) create mode 100644 CHANGES/10119.bugfix.rst diff --git a/CHANGES/10119.bugfix.rst b/CHANGES/10119.bugfix.rst new file mode 100644 index 00000000000..86d2511f5b5 --- /dev/null +++ b/CHANGES/10119.bugfix.rst @@ -0,0 +1 @@ +Response is now always True, instead of using MutableMapping behaviour (False when map is empty) diff --git a/aiohttp/web_response.py b/aiohttp/web_response.py index e2351ddf7b7..99636f2de59 100644 --- a/aiohttp/web_response.py +++ b/aiohttp/web_response.py @@ -609,6 +609,9 @@ def __hash__(self) -> int: def __eq__(self, other: object) -> bool: return self is other + def __bool__(self) -> bool: + return True + class Response(StreamResponse): diff --git a/tests/test_web_request.py b/tests/test_web_request.py index c22b3b17921..6c9e3826d73 100644 --- a/tests/test_web_request.py +++ b/tests/test_web_request.py @@ -311,6 +311,7 @@ def test_match_info() -> None: def test_request_is_mutable_mapping() -> None: req = make_mocked_request("GET", "/") assert isinstance(req, MutableMapping) + assert req # even when the MutableMapping is empty, request should always be True req["key"] = "value" assert "value" == req["key"] diff --git a/tests/test_web_response.py b/tests/test_web_response.py index 24743e64635..1e65f7364b6 100644 --- a/tests/test_web_response.py +++ b/tests/test_web_response.py @@ -106,6 +106,7 @@ def test_stream_response_eq() -> None: def test_stream_response_is_mutable_mapping() -> None: resp = StreamResponse() assert isinstance(resp, collections.abc.MutableMapping) + assert resp # even when the MutableMapping is empty, response should always be True resp["key"] = "value" assert "value" == resp["key"] From 5185f932407947fbbabd31c5dc6d72abc5c54171 Mon Sep 17 00:00:00 2001 From: Javier Torres Date: Wed, 18 Dec 2024 19:28:00 +0100 Subject: [PATCH 20/23] Stream unpauses protocol before releasing connection (#10171) --- CHANGES/10169.bugfix.rst | 3 +++ CONTRIBUTORS.txt | 1 + aiohttp/streams.py | 3 +++ tests/test_flowcontrol_streams.py | 22 ++++++++++++++++++++++ 4 files changed, 29 insertions(+) create mode 100644 CHANGES/10169.bugfix.rst diff --git a/CHANGES/10169.bugfix.rst b/CHANGES/10169.bugfix.rst new file mode 100644 index 00000000000..32e06783856 --- /dev/null +++ b/CHANGES/10169.bugfix.rst @@ -0,0 +1,3 @@ +Fixed a hang where a connection previously used for a streaming +download could be returned to the pool in a paused state. +-- by :user:`javitonino`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 7a7f882c885..591142d469e 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -179,6 +179,7 @@ Jan Buchar Jan Gosmann Jarno Elonen Jashandeep Sohi +Javier Torres Jean-Baptiste Estival Jens Steinhauser Jeonghun Lee diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 82d404d617b..ca7a420c6d5 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -220,6 +220,9 @@ def feed_eof(self) -> None: self._eof_waiter = None set_result(waiter, None) + if self._protocol._reading_paused: + self._protocol.resume_reading() + for cb in self._eof_callbacks: try: cb() diff --git a/tests/test_flowcontrol_streams.py b/tests/test_flowcontrol_streams.py index a6516fec75a..9e21f786610 100644 --- a/tests/test_flowcontrol_streams.py +++ b/tests/test_flowcontrol_streams.py @@ -107,3 +107,25 @@ async def test_read_nowait(self, stream: streams.StreamReader) -> None: res = stream.read_nowait(5) assert res == b"" assert stream._protocol.resume_reading.call_count == 1 # type: ignore[attr-defined] + + async def test_resumed_on_eof(self, stream: streams.StreamReader) -> None: + stream.feed_data(b"data") + assert stream._protocol.pause_reading.call_count == 1 # type: ignore[attr-defined] + assert stream._protocol.resume_reading.call_count == 0 # type: ignore[attr-defined] + stream._protocol._reading_paused = True + + stream.feed_eof() + assert stream._protocol.resume_reading.call_count == 1 # type: ignore[attr-defined] + + +async def test_stream_reader_eof_when_full() -> None: + loop = asyncio.get_event_loop() + protocol = BaseProtocol(loop=loop) + protocol.transport = asyncio.Transport() + stream = streams.StreamReader(protocol, 1024, loop=loop) + + data_len = stream._high_water + 1 + stream.feed_data(b"0" * data_len) + assert protocol._reading_paused + stream.feed_eof() + assert not protocol._reading_paused From db56d743b2f11a8889938da4f044e73c0ad4bd30 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Wed, 18 Dec 2024 08:55:12 -1000 Subject: [PATCH 21/23] [PR #10171/5185f93 backport][3.11] Stream unpauses protocol before releasing connection (#10179) Co-authored-by: Javier Torres --- CHANGES/10169.bugfix.rst | 3 +++ CONTRIBUTORS.txt | 1 + aiohttp/streams.py | 3 +++ tests/test_flowcontrol_streams.py | 23 +++++++++++++++++++++++ 4 files changed, 30 insertions(+) create mode 100644 CHANGES/10169.bugfix.rst diff --git a/CHANGES/10169.bugfix.rst b/CHANGES/10169.bugfix.rst new file mode 100644 index 00000000000..32e06783856 --- /dev/null +++ b/CHANGES/10169.bugfix.rst @@ -0,0 +1,3 @@ +Fixed a hang where a connection previously used for a streaming +download could be returned to the pool in a paused state. +-- by :user:`javitonino`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 5acc4de44fc..589784b29cb 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -171,6 +171,7 @@ Jan Buchar Jan Gosmann Jarno Elonen Jashandeep Sohi +Javier Torres Jean-Baptiste Estival Jens Steinhauser Jeonghun Lee diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 029d577b88c..6126fb5695d 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -220,6 +220,9 @@ def feed_eof(self) -> None: self._eof_waiter = None set_result(waiter, None) + if self._protocol._reading_paused: + self._protocol.resume_reading() + for cb in self._eof_callbacks: try: cb() diff --git a/tests/test_flowcontrol_streams.py b/tests/test_flowcontrol_streams.py index 68e623b6dd7..9874cc2511e 100644 --- a/tests/test_flowcontrol_streams.py +++ b/tests/test_flowcontrol_streams.py @@ -4,6 +4,7 @@ import pytest from aiohttp import streams +from aiohttp.base_protocol import BaseProtocol @pytest.fixture @@ -112,6 +113,15 @@ async def test_read_nowait(self, stream) -> None: assert res == b"" assert stream._protocol.resume_reading.call_count == 1 # type: ignore[attr-defined] + async def test_resumed_on_eof(self, stream: streams.StreamReader) -> None: + stream.feed_data(b"data") + assert stream._protocol.pause_reading.call_count == 1 # type: ignore[attr-defined] + assert stream._protocol.resume_reading.call_count == 0 # type: ignore[attr-defined] + stream._protocol._reading_paused = True + + stream.feed_eof() + assert stream._protocol.resume_reading.call_count == 1 # type: ignore[attr-defined] + async def test_flow_control_data_queue_waiter_cancelled( buffer: streams.FlowControlDataQueue, @@ -180,3 +190,16 @@ async def test_flow_control_data_queue_read_eof( buffer.feed_eof() with pytest.raises(streams.EofStream): await buffer.read() + + +async def test_stream_reader_eof_when_full() -> None: + loop = asyncio.get_event_loop() + protocol = BaseProtocol(loop=loop) + protocol.transport = asyncio.Transport() + stream = streams.StreamReader(protocol, 1024, loop=loop) + + data_len = stream._high_water + 1 + stream.feed_data(b"0" * data_len) + assert protocol._reading_paused + stream.feed_eof() + assert not protocol._reading_paused From a3a57167c918af1d1f1323177760e3ba8abcca5c Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Wed, 18 Dec 2024 08:58:57 -1000 Subject: [PATCH 22/23] [PR #10171/5185f93 backport][3.12] Stream unpauses protocol before releasing connection (#10180) Co-authored-by: Javier Torres --- CHANGES/10169.bugfix.rst | 3 +++ CONTRIBUTORS.txt | 1 + aiohttp/streams.py | 3 +++ tests/test_flowcontrol_streams.py | 23 +++++++++++++++++++++++ 4 files changed, 30 insertions(+) create mode 100644 CHANGES/10169.bugfix.rst diff --git a/CHANGES/10169.bugfix.rst b/CHANGES/10169.bugfix.rst new file mode 100644 index 00000000000..32e06783856 --- /dev/null +++ b/CHANGES/10169.bugfix.rst @@ -0,0 +1,3 @@ +Fixed a hang where a connection previously used for a streaming +download could be returned to the pool in a paused state. +-- by :user:`javitonino`. diff --git a/CONTRIBUTORS.txt b/CONTRIBUTORS.txt index 035436c0426..930815d8b62 100644 --- a/CONTRIBUTORS.txt +++ b/CONTRIBUTORS.txt @@ -172,6 +172,7 @@ Jan Buchar Jan Gosmann Jarno Elonen Jashandeep Sohi +Javier Torres Jean-Baptiste Estival Jens Steinhauser Jeonghun Lee diff --git a/aiohttp/streams.py b/aiohttp/streams.py index 029d577b88c..6126fb5695d 100644 --- a/aiohttp/streams.py +++ b/aiohttp/streams.py @@ -220,6 +220,9 @@ def feed_eof(self) -> None: self._eof_waiter = None set_result(waiter, None) + if self._protocol._reading_paused: + self._protocol.resume_reading() + for cb in self._eof_callbacks: try: cb() diff --git a/tests/test_flowcontrol_streams.py b/tests/test_flowcontrol_streams.py index 68e623b6dd7..9874cc2511e 100644 --- a/tests/test_flowcontrol_streams.py +++ b/tests/test_flowcontrol_streams.py @@ -4,6 +4,7 @@ import pytest from aiohttp import streams +from aiohttp.base_protocol import BaseProtocol @pytest.fixture @@ -112,6 +113,15 @@ async def test_read_nowait(self, stream) -> None: assert res == b"" assert stream._protocol.resume_reading.call_count == 1 # type: ignore[attr-defined] + async def test_resumed_on_eof(self, stream: streams.StreamReader) -> None: + stream.feed_data(b"data") + assert stream._protocol.pause_reading.call_count == 1 # type: ignore[attr-defined] + assert stream._protocol.resume_reading.call_count == 0 # type: ignore[attr-defined] + stream._protocol._reading_paused = True + + stream.feed_eof() + assert stream._protocol.resume_reading.call_count == 1 # type: ignore[attr-defined] + async def test_flow_control_data_queue_waiter_cancelled( buffer: streams.FlowControlDataQueue, @@ -180,3 +190,16 @@ async def test_flow_control_data_queue_read_eof( buffer.feed_eof() with pytest.raises(streams.EofStream): await buffer.read() + + +async def test_stream_reader_eof_when_full() -> None: + loop = asyncio.get_event_loop() + protocol = BaseProtocol(loop=loop) + protocol.transport = asyncio.Transport() + stream = streams.StreamReader(protocol, 1024, loop=loop) + + data_len = stream._high_water + 1 + stream.feed_data(b"0" * data_len) + assert protocol._reading_paused + stream.feed_eof() + assert not protocol._reading_paused From 8aaaba3ec798327ab5ab52c977fb7395b56c54a4 Mon Sep 17 00:00:00 2001 From: "J. Nick Koston" Date: Wed, 18 Dec 2024 09:35:05 -1000 Subject: [PATCH 23/23] Release 3.11.11 (#10181) --- CHANGES.rst | 74 +++++++++++++++++++++++++++++++++++++++ CHANGES/10093.bugfix.rst | 2 -- CHANGES/10099.bugfix.rst | 1 - CHANGES/10149.misc.rst | 4 --- CHANGES/10154.bugfix.rst | 1 - CHANGES/10156.feature.rst | 3 -- CHANGES/10169.bugfix.rst | 3 -- aiohttp/__init__.py | 2 +- 8 files changed, 75 insertions(+), 15 deletions(-) delete mode 100644 CHANGES/10093.bugfix.rst delete mode 100644 CHANGES/10099.bugfix.rst delete mode 100644 CHANGES/10149.misc.rst delete mode 100644 CHANGES/10154.bugfix.rst delete mode 100644 CHANGES/10156.feature.rst delete mode 100644 CHANGES/10169.bugfix.rst diff --git a/CHANGES.rst b/CHANGES.rst index 586d70c9697..b07cec6a093 100644 --- a/CHANGES.rst +++ b/CHANGES.rst @@ -10,6 +10,80 @@ .. towncrier release notes start +3.11.11 (2024-12-18) +==================== + +Bug fixes +--------- + +- Updated :py:meth:`~aiohttp.ClientSession.request` to reuse the ``quote_cookie`` setting from ``ClientSession._cookie_jar`` when processing cookies parameter. + -- by :user:`Cycloctane`. + + + *Related issues and pull requests on GitHub:* + :issue:`10093`. + + + +- Fixed type of ``SSLContext`` for some static type checkers (e.g. pyright). + + + *Related issues and pull requests on GitHub:* + :issue:`10099`. + + + +- Updated :meth:`aiohttp.web.StreamResponse.write` annotation to also allow :class:`bytearray` and :class:`memoryview` as inputs -- by :user:`cdce8p`. + + + *Related issues and pull requests on GitHub:* + :issue:`10154`. + + + +- Fixed a hang where a connection previously used for a streaming + download could be returned to the pool in a paused state. + -- by :user:`javitonino`. + + + *Related issues and pull requests on GitHub:* + :issue:`10169`. + + + + +Features +-------- + +- Enabled ALPN on default SSL contexts. This improves compatibility with some + proxies which don't work without this extension. + -- by :user:`Cycloctane`. + + + *Related issues and pull requests on GitHub:* + :issue:`10156`. + + + + +Miscellaneous internal changes +------------------------------ + +- Fixed an infinite loop that can occur when using aiohttp in combination + with `async-solipsism`_ -- by :user:`bmerry`. + + .. _async-solipsism: https://github.com/bmerry/async-solipsism + + + *Related issues and pull requests on GitHub:* + :issue:`10149`. + + + + +---- + + 3.11.10 (2024-12-05) ==================== diff --git a/CHANGES/10093.bugfix.rst b/CHANGES/10093.bugfix.rst deleted file mode 100644 index 4d7076115d9..00000000000 --- a/CHANGES/10093.bugfix.rst +++ /dev/null @@ -1,2 +0,0 @@ -Update :py:meth:`~aiohttp.ClientSession.request` to reuse the ``quote_cookie`` setting from ``ClientSession._cookie_jar`` when processing cookies parameter. --- by :user:`Cycloctane`. diff --git a/CHANGES/10099.bugfix.rst b/CHANGES/10099.bugfix.rst deleted file mode 100644 index 718420a6ad5..00000000000 --- a/CHANGES/10099.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Fixed type of ``SSLContext`` for some static type checkers (e.g. pyright). diff --git a/CHANGES/10149.misc.rst b/CHANGES/10149.misc.rst deleted file mode 100644 index 61765a50fcf..00000000000 --- a/CHANGES/10149.misc.rst +++ /dev/null @@ -1,4 +0,0 @@ -Fixed an infinite loop that can occur when using aiohttp in combination -with `async-solipsism`_ -- by :user:`bmerry`. - -.. _async-solipsism: https://github.com/bmerry/async-solipsism diff --git a/CHANGES/10154.bugfix.rst b/CHANGES/10154.bugfix.rst deleted file mode 100644 index 382d9e56e6c..00000000000 --- a/CHANGES/10154.bugfix.rst +++ /dev/null @@ -1 +0,0 @@ -Updated :meth:`aiohttp.web.StreamResponse.write` annotation to also allow :class:`bytearray` and :class:`memoryview` as inputs -- by :user:`cdce8p`. diff --git a/CHANGES/10156.feature.rst b/CHANGES/10156.feature.rst deleted file mode 100644 index 0ff6b6b8bd8..00000000000 --- a/CHANGES/10156.feature.rst +++ /dev/null @@ -1,3 +0,0 @@ -Enabled ALPN on default SSL contexts. This improves compatibility with some -proxies which don't work without this extension. --- by :user:`Cycloctane`. diff --git a/CHANGES/10169.bugfix.rst b/CHANGES/10169.bugfix.rst deleted file mode 100644 index 32e06783856..00000000000 --- a/CHANGES/10169.bugfix.rst +++ /dev/null @@ -1,3 +0,0 @@ -Fixed a hang where a connection previously used for a streaming -download could be returned to the pool in a paused state. --- by :user:`javitonino`. diff --git a/aiohttp/__init__.py b/aiohttp/__init__.py index f4d732b8674..b9af3f829f7 100644 --- a/aiohttp/__init__.py +++ b/aiohttp/__init__.py @@ -1,4 +1,4 @@ -__version__ = "3.11.11.dev0" +__version__ = "3.11.11" from typing import TYPE_CHECKING, Tuple