This repository was archived by the owner on Jan 6, 2026. It is now read-only.
Conversation
SPARK is about retrieval testing not load testing. Currently, more than 90% of retrievals fail on timeout or server errors. There is little value in checking whether we can download the entire CAR file, it's more important to know whether SPs are advertising FIL+ deals to IPNI and accept HTTP-based retrieval requests. SPARK is also not able to fetch and verify CAR files larger than 200MB, which further reduced the value of fetching full CAR files. Let's fetch the root block for the time being. - This should significantly reduce the load on "good" SPs. - It provides enough data allowing "bad" SPs to fix their setup. Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
Signed-off-by: Miroslav Bajtoš <oss@bajtos.net>
juliangruber
approved these changes
Dec 12, 2023
|
Seems reasonable to me. Depending on whether you think it adds value you could alternatively fetch either I could imagine that ipfs/specs#445 might be an interesting thing to add into the mix if we ever got that done for SPs; a shallow fetch that avoids the costly raw blocks but shows they can respond with non-trivial data. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SPARK is about retrieval testing not load testing.
Currently, more than 90% of retrievals fail on timeout or server errors. There is little value in checking whether we can download the entire CAR file; it's more important to know whether SPs are advertising FIL+ deals to IPNI and accepting HTTP-based retrieval requests.
SPARK is also currently unable to fetch and verify CAR files larger than 200MB, which further reduces the value of fetching full CAR files.
Let's fetch the root block for the time being.
/cc @rvagg @hannahhoward @willscott @gruns do you have any thoughts on this?
See also: