Add proxy configuration support to ApiTarget and PageTarget

[Copilot]

Extends tracker targets with proxy server configuration including optional authentication credentials for Proxy-Authorization headers. Implements automatic migration for existing trackers.

Changes

• Types: Added ProxyConfig (URL + optional ProxyCredentials) with scheme/value for Proxy-Authorization header
• Targets: Added optional proxy field to ApiTarget and PageTarget
• Serialization: Extended RawApiTarget/RawPageTarget with RawProxyConfig for database storage
• Migration: V1 trackers (no proxy) automatically deserialize as V2 with proxy: None via postcard fallback
• Reqwest: Modified http_client() to return Result and configure proxy with authentication
• Playwright: Extended WebScraperContentRequest and worker to pass proxy config to browser context

Example Usage

let target = ApiTarget {
    requests: vec![TargetRequest::new(url)],
    proxy: Some(ProxyConfig {
        url: "http://proxy.example.com:8080".parse()?,
        credentials: Some(ProxyCredentials {
            scheme: "Basic".to_string(),
            value: "dXNlcjpwYXNz".to_string(),
        }),
    }),
    ..Default::default()
};

Limitations

Playwright's proxy API only supports username/password authentication. Custom auth schemes log a warning and require extraHTTPHeaders implementation (not included).

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• esm.ubuntu.com
  • Triggering command: /usr/lib/apt/methods/https /usr/lib/apt/methods/https /home/REDACTED/.rustup/toolchains/stable-x86_64-REDACTED-linux-gnu/lib/rustlib/x86_64-u�� 6ed83.rlib 64-REDACTED-linux-gnu/lib/libstd-a71a575950182156.rlib 64-REDACTED-linux-gnu/lib/libpanic_unwind-150250ca1242312f.rlib 64-REDACTED-linux-gnu/lib/libobject-00745b6f63a5acb7.rlib 64-REDACTED-linux-gnu/lib/libmemchr-ff20d9e744b5ce29.rlib 64-REDACTED-linux-gnu/lib/libaddr2line-05ff42d0bb8bc056.rlib 64-REDACTED-linux-gnu/lib/libgimli-9e30a03d9a87a46b.rlib 64-REDACTED-linux-gnu/lib/libcfg_if-ca532f729409e986.rlib 64-REDACTED-linux-gnu/lib/librustc_demangle-d3519961f658f03a.rlib 64-REDACTED-linux-gnu/lib/libstd_detect-866c9680e50cd51f.rlib 64-u�� 64-REDACTED-linux-gnu/lib/librustc_std_workspace_alloc-2fb855ef27dbee02.rlib 64-REDACTED-linux-gnu/lib/libminiz_oxide-9b8400ec28816bfd.rlib lib/rustlib/x86_-m 51b9484a6a50a8c2findmnt lib/rustlib/x86_-t 51b9484a6a50a8c2cgroup2 lib/rustlib/x86_--noheadings (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

You need to extend ApiTarget and PageTarget with proxy property. The property should include proxy server url (URL type) and optional credentials that will consist of scheme and value that will be used for Proxy-Authorization HTTP header. If proxy is specified its config should be passed to retrack-web-scraper component that later use it when creates a new playwright context (it's for PageTarget), for API target it should properly populate proxy for Reqwest HTTP client when making requests. The change should also add migration step for existing trackers (they are stored in postgress DB with a serialized target property. Without migration deserialization of existing trackers will fail. Migration is happening in migrate method of the Api struct and based on the idea is that you load tracker using previous interface, and re-save it using the latest one. Usually I've updated RawTracker to the latest interface, and created RawTrackerV1 with the previous that I used for deserialization only, but if you have better ideas how to perform that sort of migration - I'm all ears. New functionality should be covered with unit tests, there are a bunch of examples, also try to adhere to the existing patterns and conventions.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.