curl needed more event

[stangri]

Hey @aarond10, I've never seen it before but someone reported seeing a message from https://github.com/aarond10/https_dns_proxy/blob/master/src/https_client.c#L601-L604 in their logs.

Can you explain in laymans terms why/when it happens? Is it something that needs to be changed in curl when it's built?

Also, would you like a PR which fixes the message so it's more understandable?

[aarond10]

If two DNS requests arrive in quick succession, we normally rely on curl's CURLMOPT_PIPELINING option to multiplex them over the one TCP connection. This is what happens in HTTP/2 and HTTP/3. If the server (or proxy) they're talking to is using HTTP/1, then multiplexing is not supported so if a bunch of requests come in together, then we either queue them up or create multiple connections -- one per request. I imagine this must be what is going on although I'm not sure how that error occurs. There might be an off by one error somewhere. Do you have any more details on curl versions, etc I can see? CURLMOPT_MAX_TOTAL_CONNECTIONS should be set to MAX_TOTAL_CONNECTIONS when we setup curl and that error implies that curl is sending us more than that or we've somehow gotten out of sync.

…

On Tue, 10 Dec 2024 at 04:32, Stan Grishin ***@***.***> wrote: Hey @aarond10 <https://github.com/aarond10>, I've never seen it before but someone reported seeing a message from https://github.com/aarond10/https_dns_proxy/blob/master/src/https_client.c#L601-L604 in their logs. Can you explain in laymans terms why/when it happens? Is it something that needs to be changed in curl when it's built? Also, would you like a PR which fixes the message so it's more understandable? — Reply to this email directly, view it on GitHub <#178>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AABTOXXJFJ4QUCHMBDWSG6T2EXH4RAVCNFSM6AAAAABTJK54JOVHI2DSMVQWIX3LMV43ASLTON2WKOZSG4ZDONZZGEYTOMA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[stangri]

Thanks for your prompt reply @aarond10, here's the link with the OpenWrt issue with curl version and for resolver url they are using the https://dns11.quad9.net/dns-query openwrt/packages#25522 (comment)

[baranyaib90]

Issue is valid. I'm using HTTP/2 on my home server and since I have upgraded to curl 8.9.1 I see such logs. (With 8.5.0 no logs.)
So this is curl issue, I will look into it in the upcoming days.
I could also improve the fatal log message and change it to simple error.
Also the limit of 8 could be changed to configurable, but I don't think that would work-around the issue.

[baranyaib90]

I have a work around but I do not like it entirely.
I also plan to make a proper mitigation.
But this will take time, since itt happens every 1-2 day and I could bot trigger the issue more often.
Later I will open a curl ticket as well.

[baranyaib90]

I have opened 2 curl bug reports, but I'm not expecting them to fix the issues:

1. CURLOPT_MAXAGE_CONN value seems to be ignored curl/curl#15856
2. CURLMOPT_MAX_TOTAL_CONNECTIONS seems to be ignored curl/curl#15857

I'm planning to work-around and/or mitigate the issue in our code. It is feasible, I have multiple solutions already but I would like to polish them and pick the best ones...
(Draft work on my master branch if soneone wants to peek, but I guess no one does.)

[baranyaib90]

Hi, just giving life sign here: issue not forgotten, I just did not had time for it.
If everything goes well, in a few weeks I will send a larger pack of code change.

[baranyaib90]

Hi @stangri,
I forgot to tell you sooner, but since #182 has been merged, this issue is most definetly fixed in multiple ways.
So if you would uplift the OpenWRT package to latest master it should be fine.
Br, Balázs

[aarond10]

Thanks @baranyaib90 for your fixes. I'm going to close this issue now.