gateway: fix car/block responses cached as deserialized ones when ?format=raw|car is missing

[lidel]

Extracting context from internal https://github.com/orgs/ipshipyard/projects/1/views/1 to create a public log why Content-Location was added in #603

Problem

• /ipfs/cid can return different bytes based on Accept header which caused bugs in the past (block response being returned instead of deserialized one etc)
  • We can't use Vary: Accept because we don't want cache key to be different for every blient/browser, we only want it to be different for block/car/ipns-record/tar/(dag-)json|cbor content types

Solution

• Old bifrost-infra had custom code that uses custom cache key if response is block or car
  • Generic HTTP stack like LB from Cloudflare may not have that capability, and if it does, it requires custom config similar to one we did in Nginx.
  • Better to solve this at gateway level, using generic HTTP, with safe defaults that "just work" with stock HTTP tools and services, without requiring additional config on each deployment

Ideas

• (A) Ensure unique URL per response type, so it does not matter if HTTP middleware is not aware of our content-types
  • Return HTTP 302 to ?format=car|raw|foo with Cache-Control: no-cache when alternative response format is requested
    • DOWNSIDE: bypasses http caches, hits backend every time (path needs to be resolved, blocks fetched/read)
• (B)
  • Include Content-Location: [...]?format=foo with non-default response
  • Implement in boxo/gateway + update gateway specs
• (C) use Vary: Accept and pay the penalty of more cache misses

Plan

• 2024-04-08 implement (B) for now, and test with popular LB/CDN provider like Cloudflare
• 2024-04-18 gateway: add Content-Location for non-default response formats #603 is ready