From 20311496fe19bbe0eccf2279a7a82c4fa1699abf Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Tue, 26 Jun 2018 09:03:56 +0200 Subject: [PATCH 01/12] Align client-hints processing with Chromium's implementation This change aligns the processing of Client-Hints with the shipped Chromium implementation, by changing the following: * Clone the environment settings object's client-hints set to the request's client-hints set. * Apply CH processing to all requests, rather than only subresource requests. * Add processing of new CH headers, added in https://github.com/whatwg/fetch/pull/725 * Remove `DPR` and `Viewport-Width` from headers that are sent by default for navigation requests. It also renames "client-hints list" to "client-hints set", and changes it to be a set, to match related HTML spec changes. --- fetch.bs | 108 +++++++++++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 94 insertions(+), 14 deletions(-) diff --git a/fetch.bs b/fetch.bs index b023ee89c..e8ed7d9dc 100644 --- a/fetch.bs +++ b/fetch.bs @@ -1295,6 +1295,15 @@ to not have to set request's referrer.

This can be used to override a referrer policy associated with an environment settings object. +

A request has an associated +client-hints set, +which is a client-hints set. Unless stated +otherwise, it is the empty set. + +

This will be used to override a client-hints set associated with +an environment settings object. +[[!CLIENT-HINTS]] +

A request has an associated synchronous flag. Unless stated otherwise it is unset. @@ -2115,6 +2124,20 @@ run these steps: +

Client-hints set

+ +

HTTP Client-Hints requires further integration. More details on the +issue. +[[!CLIENT-HINTS]] + + +

A client-hints set is a +set of +Client hint tokens, each +of which is one of `DPR`, `Save-Data`, `Viewport-Width`, or +`Width`. + +

Streams

This section might be integrated into other standards, such as IDL. @@ -3062,6 +3085,10 @@ the request. origin to request's client's origin. +

  • Set request's client-hints set to be a clone + of the client-hints set of the request's + client's global object. +

  • If request's header list does not contain `Accept`, then: @@ -3107,20 +3134,73 @@ the request. HTTP/1 fetches.

  • -

    If request is a subresource request, then: - -

      -

    1. Let record be a new - fetch record consisting of - request and this instance of the - fetch algorithm. - -

    2. Append record to request's - client's - fetch group list of - fetch records. -

    - +

    If request is a navigation request, a user agent should, for each + header name (hintName) in the first column of the + following table, if request's header list + does not contain hintName, then + append + hintName/the value given in the same row on the second column, to + request's header list. + + + + +
    Name + Value +
    `Save-Data` + a suitable save-data value +
    + +

  • + +

    For each hintName of request's + client-hints set: + +

      +
    1. +

      Let value be the first matching statement, switching on hintName: + +

      +
      `DPR` +
      a suitable dpr value +
      `Save-Data` +
      a suitable save-data value +
      `Viewport-Width` +
      a suitable viewport-width value +
      `Width` +
      a suitable width value +
      `Device-Memory` +
      a suitable Device-Memory value +
      `RTT` +
      a suitable RTT value +
      `Downlink` +
      a suitable Downlink value +
      `ECT` +
      a suitable ECT value +
      + +

    2. If request's header list does not + contain hintName, a user agent should append + hintName/value to request's header list. +

    + +

    The above step should be applicable only for same-origin requests. See + issue for more details.

    + + +

  • If request is a subresource request, then: +

      +

    1. Let record be a new + fetch record consisting of + request and this instance of the + fetch algorithm. + +

    2. Append record to request's + client's + fetch group list of + fetch records. +

    +

  • If aborted, then: From b533da48ffa7cd6435158c7838816c5d89f0fb69 Mon Sep 17 00:00:00 2001 From: Eric Portis Date: Mon, 24 Sep 2018 06:50:15 -0700 Subject: [PATCH 02/12] Check Feature Policy before sending Client Hints --- fetch.bs | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fetch.bs b/fetch.bs index e8ed7d9dc..36ddae4ec 100644 --- a/fetch.bs +++ b/fetch.bs @@ -3179,6 +3179,14 @@ the request.

    a suitable ECT value +

  • If the result of running + Should + request be allowed to use feature?, + given request and hintName’s + associated + policy-controlled feature, returns false, then continue. + [[!FEATURE-POLICY]] [[!CLIENT-HINTS]] +

  • If request's header list does not contain hintName, a user agent should append hintName/value to request's header list. From 5c94e8b40021d8f3f03333444d36f073df2f2362 Mon Sep 17 00:00:00 2001 From: Eric Portis Date: Mon, 24 Sep 2018 08:26:55 -0700 Subject: [PATCH 03/12] Add Feature-Policy biblio reference --- fetch.bs | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/fetch.bs b/fetch.bs index 36ddae4ec..f5ace0e2d 100644 --- a/fetch.bs +++ b/fetch.bs @@ -78,6 +78,14 @@ url:https://tools.ietf.org/html/rfc7234#section-1.2.1;text:delta-seconds;type:df }, "OCSP": { "aliasOf": "RFC2560" + }, + "FEATURE-POLICY": { + "authors": [ + "Ian Clelland" + ], + "href": "https://wicg.github.io/feature-policy/", + "publisher": "WICG", + "title": "Feature Policy" } } From 5b7f3798bd9503c477d56cb870fc104a2b9ea364 Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Thu, 17 Jan 2019 14:27:46 +0100 Subject: [PATCH 04/12] Fix up PR from @eeeps according to IRC discussion --- fetch.bs | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/fetch.bs b/fetch.bs index f5ace0e2d..375b2c0a4 100644 --- a/fetch.bs +++ b/fetch.bs @@ -3186,24 +3186,25 @@ the request.

    `ECT`
    a suitable ECT value - -

  • If the result of running - Should + +

  • If request is a cross-origin subresource request and the result of + running Should request be allowed to use feature?, given request and hintName’s associated - policy-controlled feature, returns false, then continue. + policy-controlled feature, returns false, then skip the next steps and + continue to the next hintName. [[!FEATURE-POLICY]] [[!CLIENT-HINTS]] + + +

  • If request's header list does not contain hintName, a user agent should append hintName/value to request's header list. -

    The above step should be applicable only for same-origin requests. See - issue for more details.

    - -

  • If request is a subresource request, then:

    1. Let record be a new From 5a74cf7ce67c72614101e2f62b6bdcfab4773fa2 Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Thu, 17 Jan 2019 18:03:14 +0100 Subject: [PATCH 05/12] Remove the cross-origin condition --- fetch.bs | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fetch.bs b/fetch.bs index 375b2c0a4..7a11d638b 100644 --- a/fetch.bs +++ b/fetch.bs @@ -3187,8 +3187,7 @@ the request.

      a suitable ECT value -

    2. If request is a cross-origin subresource request and the result of - running

      If request is a subresource request and the result of running Should request be allowed to use feature?, given request and hintName’s From 3da867ba619e21c00d8942935b4fd8e00c008220 Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Thu, 17 Jan 2019 18:05:34 +0100 Subject: [PATCH 06/12] Remove vars from algo name --- fetch.bs | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fetch.bs b/fetch.bs index 7a11d638b..193240c09 100644 --- a/fetch.bs +++ b/fetch.bs @@ -3189,8 +3189,7 @@ the request.

    3. If request is a subresource request and the result of running Should - request be allowed to use feature?, - given request and hintName’s + request be allowed to use feature?, given request and hintName’s associated policy-controlled feature, returns false, then skip the next steps and continue to the next hintName. From 7b72f9a29c84885251541a4e8c28ca15b754dfac Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Fri, 18 Jan 2019 14:32:07 +0100 Subject: [PATCH 07/12] Trying to fix client-hints set linking --- fetch.bs | 14 ++++---------- 1 file changed, 4 insertions(+), 10 deletions(-) diff --git a/fetch.bs b/fetch.bs index 193240c09..b65a43ff1 100644 --- a/fetch.bs +++ b/fetch.bs @@ -2134,18 +2134,12 @@ run these steps:

      Client-hints set

      -

      HTTP Client-Hints requires further integration. More details on the -issue. -[[!CLIENT-HINTS]] - - -

      A client-hints set is a +

      A client-hints set is a set of Client hint tokens, each of which is one of `DPR`, `Save-Data`, `Viewport-Width`, or `Width`. -

      Streams

      This section might be integrated into other standards, such as IDL. @@ -3162,7 +3156,7 @@ the request.

    4. For each hintName of request's - client-hints set: + client-hints set:

      1. @@ -3186,7 +3180,7 @@ the request.
        `ECT`
        a suitable ECT value - +

      2. If request is a subresource request and the result of running Should request be allowed to use feature?, given request and hintName’s @@ -3197,7 +3191,7 @@ the request. - +

      3. If request's header list does not contain hintName, a user agent should append hintName/value to request's header list. From 8a2fed16a8785880dad8af1843d572a91fd8de7f Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Mon, 21 Jan 2019 15:09:32 +0100 Subject: [PATCH 08/12] Removed for from dfn --- fetch.bs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fetch.bs b/fetch.bs index b65a43ff1..6b3b1c7d8 100644 --- a/fetch.bs +++ b/fetch.bs @@ -2134,7 +2134,7 @@ run these steps:

        Client-hints set

        -

        A client-hints set is a +

        A client-hints set is a set of Client hint tokens, each of which is one of `DPR`, `Save-Data`, `Viewport-Width`, or From 9b53f621ea1ccad50de30dbd6b631365d9d7bce3 Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Wed, 23 Jan 2019 10:03:06 -0500 Subject: [PATCH 09/12] Sec prefix, redirect 3P handling, response density --- fetch.bs | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) diff --git a/fetch.bs b/fetch.bs index 6b3b1c7d8..191b281c0 100644 --- a/fetch.bs +++ b/fetch.bs @@ -644,6 +644,7 @@ production as

        1. Let value be header's value. +

        2. If value starts with "Sec-", return true.

        3. Byte-lowercase header's name and switch on the result: @@ -1763,6 +1764,10 @@ specified. [[!CSP]] range-requested flag, which is initially unset. +

          A response has an associated +image density, which is initially set to +zero. +

          This is used to ensure to prevent a partial response from an earlier ranged request being provided to an API that didn't make a range request. See the flag's usage for a detailed description of the attack. @@ -3191,6 +3196,7 @@ the request. +

        4. Set hintName to "Sec-" concatenated with hintName.

        5. If request's header list does not contain hintName, a user agent should append @@ -3583,6 +3589,12 @@ with a CORS flag and recursive flag, run these steps: queue a fetch-request-done task for request.

        +

      4. If request's destination is "image" and response's + header list contains + `Content-DPR`, set response's density value to the + result of parsing the header value as float. + +

      5. Queue a fetch task on request to process response for response. @@ -3946,6 +3958,20 @@ optional CORS flag and CORS-preflight flag, run these steps: not same origin with actualResponse's location URL's origin, then return a network error. +

      6. For each hintName of request's + client-hints set: +

          +

        1. Set hintName to "Sec-" concatenated with hintName. +

        2. If request's header list contains + hintName and if the result of running Should + request be allowed to use feature?, given request and hintName’s + associated + policy-controlled feature, returns false, then remove hintName from + header list. + [[!FEATURE-POLICY]] [[!CLIENT-HINTS]] +

        +

      7. If CORS flag is set and actualResponse's location URL From 7050b3e22be9f589f1b7dbe1aebdd10f6dfd666d Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Tue, 12 Feb 2019 17:20:31 +0100 Subject: [PATCH 10/12] Add hints from Tarun Bansal's PR #725 --- fetch.bs | 16 ++++++++++------ 1 file changed, 10 insertions(+), 6 deletions(-) diff --git a/fetch.bs b/fetch.bs index 191b281c0..957e17e31 100644 --- a/fetch.bs +++ b/fetch.bs @@ -1764,14 +1764,14 @@ specified. [[!CSP]] range-requested flag, which is initially unset. -

        A response has an associated -image density, which is initially set to -zero. -

        This is used to ensure to prevent a partial response from an earlier ranged request being provided to an API that didn't make a range request. See the flag's usage for a detailed description of the attack. +

        A response has an associated +image density, which is initially set to +zero. +

        A response can have an associated location URL (null, failure, or a URL). Unless specified otherwise, response has no @@ -2142,7 +2142,11 @@ run these steps:

        A client-hints set is a set of Client hint tokens, each -of which is one of `DPR`, `Save-Data`, `Viewport-Width`, or +of which is one of `DPR`, `Save-Data`, `Viewport-Width`, +`Width`, `Device-Memory`, `RTT`, `Downlink`, or +`ECT`. + + `Width`.

        Streams

        @@ -3591,7 +3595,7 @@ with a CORS flag and recursive flag, run these steps:

      8. If request's destination is "image" and response's header list contains - `Content-DPR`, set response's density value to the + `Content-DPR`, set response's image density value to the result of parsing the header value as float. From 8b1d63365f2081d5d7951ae0441a0315150494f0 Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Mon, 18 Mar 2019 19:40:35 +0100 Subject: [PATCH 11/12] Fix up sec processing --- fetch.bs | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fetch.bs b/fetch.bs index 957e17e31..20016bb2a 100644 --- a/fetch.bs +++ b/fetch.bs @@ -644,7 +644,9 @@ production as

        1. Let value be header's value. -

        2. If value starts with "Sec-", return true. + +

        3. If header's name starts with a byte-case-insensitive + match to `Sec-`, return true.

        4. Byte-lowercase header's name and switch on the result: From 50676153ca647df905e649e118474555d046b87e Mon Sep 17 00:00:00 2001 From: Yoav Weiss Date: Tue, 26 Mar 2019 10:24:21 +0100 Subject: [PATCH 12/12] s/list/set/ --- fetch.bs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/fetch.bs b/fetch.bs index 20016bb2a..d636a9611 100644 --- a/fetch.bs +++ b/fetch.bs @@ -2141,7 +2141,7 @@ run these steps:

          Client-hints set

          -

          A client-hints set is a +

          A client-hints set is a set of Client hint tokens, each of which is one of `DPR`, `Save-Data`, `Viewport-Width`,