refactor: Redesign the protocol (Breaking Change)!

[sendya]

This pull request refactors the internal protocol constants by moving them from internal/constants to a new internal/protocol package, updates references throughout the codebase, and clarifies storage bucket type handling (especially replacing "normal" with "warm"). It also adds detailed documentation for custom internal protocol headers. These changes improve maintainability, consistency, and clarity across the project.

Protocol constants refactoring and documentation

• Moved protocol-related constants from internal/constants/global.go to internal/protocol/protocol.go, renamed the package, and updated all references across the codebase to use the new package. [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14] [15] [16] [17] [18] [19] [20] [21]
• Added a comprehensive internal/protocol/README.md documenting all custom internal protocol headers and their meanings.

Storage bucket type handling

• Changed the bucket type string from "inmemory" to "memory" in api/defined/v1/storage/indexdb.go, and updated references throughout the storage subsystem. [1] [2] [3]
• Replaced "normal" bucket type with "warm" in configuration merging and test cases to unify bucket type handling. [1] [2] [3] [4]

Storage subsystem improvements

• Renamed normalBucket to warmlBucket in storage/storage.go to reflect the updated bucket type terminology. [1] [2]
• Improved migration logic in diskBucket.touch by clarifying hit counting and using the correct context parameter. [1] [2]

Middleware and caching enhancements

• Updated caching and fill-range middleware logic to use the new protocol constants and improved header handling, including debug and trace headers. [1] [2] [3] [4] [5]

---

(References: [1] [2] [3] [4] [5]

[Copilot]

Pull request overview

This PR introduces a breaking internal refactor by moving protocol/header constants into a new internal/protocol package, updates imports/usages across middleware/plugins/tests, and standardizes storage bucket type terminology (notably normal→warm and inmemory→memory) with related storage behavior adjustments.

Changes:

• Move internal protocol constants to internal/protocol and update references throughout the codebase.
• Standardize storage bucket type strings and update tests/config merging to prefer warm (and memory).
• Adjust caching/middleware behavior to use the new protocol constants and refine storage/migration handling.

Reviewed changes

Copilot reviewed 19 out of 19 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tests/all-features/errcode/errcode_test.go	Update tests to use internal/protocol constants for headers.
storage/storage_test.go	Update bucket type test inputs to storagev1.TypeWarm.
storage/storage.go	Rename/reclassify “normal” buckets to warm handling; selector now uses warm bucket slice.
storage/builder.go	Default bucket type to warm and rewrite normal→warm during config merge.
storage/bucket/memory/memory.go	Adjust memory bucket initialization; changes how StoreType() is derived.
storage/bucket/disk/disk_test.go	Update test bucket type to TypeWarm.
storage/bucket/disk/disk_migration_test.go	Update migration tests to use TypeWarm.
storage/bucket/disk/disk.go	Clarify migration hit counting and touch signature usage.
server/middleware/caching/internal.go	Swap to internal/protocol constants; extend X-Cache composition; update internal header keys.
server/middleware/caching/caching_prefetch.go	Swap to internal/protocol constants for prefetch header handling.
server/middleware/caching/caching_fillrange.go	Swap to internal/protocol constants for fill-range header handling.
server/middleware/caching/caching.go	Swap to internal/protocol constants; adjust in-memory CRC-skip check logic.
plugin/registry.go	Use protocol.AppName for plugin naming.
plugin/purge/purge.go	Use protocol.InternalStoreUrl header for purge behavior.
pkg/e2e/e2e.go	Use protocol.InternalUpstreamAddr header for E2E routing.
metrics/request_info.go	Use protocol.ProtocolRequestIDKey for request-id extraction.
internal/protocol/protocol.go	New protocol constants package (incl. new X-FS-Mem).
internal/protocol/README.md	Add documentation for custom internal protocol headers.
api/defined/v1/storage/indexdb.go	Change TypeInMemory string from inmemory → memory.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

warmlBucket looks like a typo (extra l). Since this slice represents the warm/normal bucket layer, please rename it to something like warmBucket/warmBuckets to avoid confusion and improve readability (and keep naming consistent with hotBucket).

[Copilot]

memoryBucket.storeType is now initialized from opt.Type, which can be empty (e.g. callers of memory.New that don’t set Type) or defaulted to warm by mergeConfig. That means StoreType() may be ""/"warm" for an in-memory bucket, and code that relies on StoreType()==storage.TypeInMemory (e.g. storage layer classification / single-inmemory enforcement) will no longer recognize it as an in-memory bucket. Consider defaulting storeType to storage.TypeInMemory (or setting it based on opt.Driver=="memory" when opt.Type is empty) so in-memory buckets report a stable store-type.

[Copilot]

The X-Cache value now appends c.bucket.StoreType(). Per the storage.Bucket interface/implementations, StoreType() is the layer (warm/hot/cold/…), while Type() is the backend driver (native/memory/…). If the intent is to produce values like "HIT … from memory", this should likely use c.bucket.Type() (or adjust the wording) to avoid emitting "from warm" for an in-memory backend.

Suggested change

	resp.Header.Set(protocol.ProtocolCacheStatusKey, strings.Join([]string{c.cacheStatus.String(), "from", c.opt.Hostname, c.bucket.StoreType(), "(tavern/4.0)"}, " "))
	resp.Header.Set(protocol.ProtocolCacheStatusKey, strings.Join([]string{c.cacheStatus.String(), "from", c.opt.Hostname, c.bucket.Type(), "(tavern/4.0)"}, " "))

[Copilot]

The README examples use CacheTime: 60, but the actual header key is X-CacheTime (protocol.CacheTime). Please update the example to use the real header name so operators can copy/paste it correctly.

Suggested change

	### CacheTime
	Set force `Cache-Control` Cache time, value is seconds, like `CacheTime: 60` mean `Cache-Control: max-age=60`
	### X-CacheTime
	Set force `Cache-Control` Cache time, value is seconds, like `X-CacheTime: 60` mean `Cache-Control: max-age=60`

[Copilot]

For InternalFillRangePercent, the documentation example shows InternalFillRangePercent: 50, but the actual wire header is i-x-fp (protocol.InternalFillRangePercent). The example should use the real header name/value format to avoid misconfiguration.

Suggested change

	Set fill range percent, value is int, like `InternalFillRangePercent: 50` mean fill 50% of response
	Set fill range percent, value is int, like `i-x-fp: 50` mean fill 50% of response