report: add support for YAQL expressions in bug patterns

[ogayot]

Bug patterns have historically only supported executing regular expressions. This is ok for log files and simple strings, but when we have access to more structured data (i.e., JSON or YAML) in bug reports, having the ability to find elements in such files is very useful.

In particular, this would be beneficial for Subiquity, where we have a large number of bug reports that we currently de-duplicate manually. We want to rely more on automation for such tasks.

In addition to regular expressions, we now support YAQL expressions [1] in bug patterns. They can be specified using the following syntax:

<pattern url="https://launchpad.net/bugs/XXX">
  <re key="package">subiquity</re>
  <yaql key="ProbeData">
      [ YAQL expression ]
  </yaql>
</pattern>

The format attribute can optionally be specified and will control the parser used. Supported values are "yaml" (the default) and "json".

The YAML parser can parse JSON but is slightly slower, thus specifying format="json" is recommended for JSON files.

Patterns having <yaql> tags will be considered non-matches if the yaql python module is not installed.

https://yaql.readthedocs.io/en/latest/getting_started.html

[ogayot]

I'll need to gracefully handle the absence of the yaml module as well.

[codecov]

Codecov Report

❌ Patch coverage is 70.00000% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.44%. Comparing base (3164b24) to head (e97565f).
⚠️ Report is 113 commits behind head on main.

Files with missing lines	Patch %	Lines
apport/report.py	46.77%	29 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #485      +/-   ##
==========================================
- Coverage   83.92%   82.44%   -1.48%     
==========================================
  Files         101       99       -2     
  Lines       20455    20474      +19     
  Branches     3215     3159      -56     
==========================================
- Hits        17167    16880     -287     
- Misses       2839     3114     +275     
- Partials      449      480      +31     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[schopin-pro]

@ogayot gentle ping on this, are you planning on coming back around to it?

[ogayot]

@ogayot gentle ping on this, are you planning on coming back around to it?

I would like to, yes. Ideally sometime during the questing cycle.

Before I fix the tests though, I'd be happy to get some feedback on the general idea around this change. Is this something that sounds reasonable to have in apport upstream?