Test operator reports scraper

[dsariel]

Test operator reports scraper

- Connects to downstream Zuul server
- Gets tempest reports for failed jobs in an openstack-uni-jobs-periodic-integration-rhoso18.0-rhel9
  pipeline with default cutoff date of 14 days
- Creates points in Qdrant DB for every failed test/traceback

TODO (will be implemented as a separate PR):
 - Tobico tests parser
 - Add an argument to the scraper to skip the stage of connecting to the Zuul server. that will be used
   for storing test/traceback points directly from the Zuul jobs

Co-authored-by: lpiwowar lpiwowar@users.noreply.github.com

[lpiwowar]

Apologies, I did not manage to review this entirely.

But I was able to run it locally and populate the vector database, though 🎉 (I limited the scraping only to 5 jobs). The generated data looked reasonable!:)

I would need a bit more time to merge this with clear consciousness (to investigate a bit the parsing and the rest of the code a bit more carefully). But if somebody wants to merge it, especially since it is a bit urgent, I won't object it. We can refactor and polish always later.

[dsariel]

Apologies, I did not manage to review this entirely.

But I was able to run it locally and populate the vector database, though 🎉 (I limited the scraping only to 5 jobs). The generated data looked reasonable!:)

I would need a bit more time to merge this with clear consciousness (to investigate a bit the parsing and the rest of the code a bit more carefully). But if somebody wants to merge it, especially since it is a bit urgent, I won't object it. We can refactor and polish always later.

np. Meanwhile, I have added 2 additional pipelines and an additional argument to use this script from Zuul pipelines.

[EmilienM]

why do we comment code?

[EmilienM]

LGTM for now, one minor comment but we can iterate later.

[EmilienM]

Apologies, I did not manage to review this entirely.

But I was able to run it locally and populate the vector database, though 🎉 (I limited the scraping only to 5 jobs). The generated data looked reasonable!:)

I would need a bit more time to merge this with clear consciousness (to investigate a bit the parsing and the rest of the code a bit more carefully). But if somebody wants to merge it, especially since it is a bit urgent, I won't object it. We can refactor and polish always later.

I agree. It's not perfect now but we'll iterate, as we have other stuffs to merge.