Handle and test xmllint versions that return a zero exit code even if validation fails#708
Conversation
…behavior of bad xmllint which returns a zero exit code even if the validation fails
…urn a zero exit code even if the validation fails; simply logic and update test/unit_tests/test_sdf.py accordingly
| res = validate_xml_file(file, 'suite', schema_version, logger) | ||
| if not res: | ||
| raise CCPPError(f"Invalid suite definition file, '{sdf}'") | ||
| raise CCPPError(f"Invalid suite definition file, '{file}'") |
There was a problem hiding this comment.
Unrelated bug fix uncovered while working on this PR
|
@climbfuji I'm a bit confused about how this new test helps the issue. I thought your intention was to run a test on a known bad XML, and if that returns a zero exit code, use that to determine that the xmllint version is bad? With this test it seems like we're just creating a test that will never fail with a non-zero exit code, unless I'm misunderstanding something. |
No, the intention is to create a fake xmllint version that behaves like the "bad" version that @gold2718 somehow ended up with and then make sure that this version can also be used and produces the correct results. The fake xmllint version is the newly added xmllint wrapper. |
dustinswales
left a comment
There was a problem hiding this comment.
These changes look good to me. Thanks!
|
@climbfuji I assume we want to wait for @gold2718 to review this one before merging? |
Yes, this is only five days old. Thanks for checking. |
gold2718
left a comment
There was a problem hiding this comment.
This looks good (i.e., way better than my version), thanks!
Handle and test xmllint versions that return a zero exit code even if validation fails
scripts/parse_tools/xml_tools.py, handlexmllintversions that return a zero exit code even if the validation fails. Simply logic, remove unused code, and updatetest/unit_tests/test_sdf.pyaccordinglytest/unit_tests/xmllint_wrapper/xmllintwrapper that emulates the behavior of "bad"xmllintwhich returns a zero exit code even if the validation fails, and test in GitHub actionsNotes.
call_commandwas only used in one place (validate XML files) and contained overly complicated logic to deal with optional arguments that were unnecessary. With the need to parse stdout/stderr from the subprocess in thevalidate_xmlfunction, there would have been even more code duplication and unnecessary logic.xmllintbecame a hard requirement, therefore we can remove logic that allows silent failures of the XML validation step whenxmllintcannot be found (silent failures in general are a bad idea imo).User interface changes?: No
Fixes #700
Testing:
unit tests:
pytestis now run in GitHub actions with both "good" and "bad" xmllint versionsmanual testing: ran Python
docteston the updatedxml_tools.pyfile, ranpytestlocally on my laptop with "good" and "bad" xmllint