-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[fix][test] Made ProtobufSchemaTest.testParsingInfoProperty order-independent #24807
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[fix][test] Made ProtobufSchemaTest.testParsingInfoProperty order-independent #24807
Conversation
2e8176c to
7003e52
Compare
lhotari
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
353d0be to
ebc185d
Compare
|
I apologize, I forgot to enable the CI checks on my local fork after creating it. I fixed the formatting issues but the code failed some other CI checks on my fork so I'm looking into that right now. I will update this once I've gotten all of the CI checks to pass on my fork. |
c8176ad to
ae0b534
Compare
ae0b534 to
7ad6d73
Compare
|
@lhotari All checks passed on my fork: LucasEby#2 |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## master #24807 +/- ##
============================================
- Coverage 74.34% 74.24% -0.10%
+ Complexity 33842 33769 -73
============================================
Files 1912 1912
Lines 149072 149072
Branches 17299 17299
============================================
- Hits 110833 110684 -149
- Misses 29440 29555 +115
- Partials 8799 8833 +34
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
…ependent (apache#24807) (cherry picked from commit 2dc4dcb) (cherry picked from commit e169f8d)
…ependent (apache#24807) (cherry picked from commit 2dc4dcb) (cherry picked from commit e169f8d)
Fixes #24806
Motivation
In ProtobufSchemaTest.testParsingInfoProperty, the json's contents and the PARSING_INFO field's contents do not have a deterministic order but the hardcoded string assertion assumes a deterministic order.
Serializing a map to a JSON with new ObjectMapper().writeValueAsString(schema.getSchemaInfo().getProperties()) can change the order of the keys since attribute order is not guaranteed in JSON. Additionally the inner JSON string order in PARSING_INFO can vary for the same reason. The ordering in both cases can change due to different environments producing the contents in different orders despite the logical contents being the same. Since the test compares the raw strings/trees "as-is", harmless re-ordering could flip the test from pass to fail despite the data being semantically the same.
Modifications
We no longer compare raw strings/trees "as-is". Instead, we compare a json structure that we derive. When we normalize the properties, we copy every key/value from the properties map so nothing at the top level is ignored. For __PARSING_INFO__ only we parse every string value into an ArrayNode, sort the array deterministically by (number, name) to remove element-order nondeterminism, and then store it back as a JSON array (instead of a string). For all other keys we keep their string values as-is.
When performing the Assert.assertEquals assertion, it is performed with the JsonNode equals method which is an important subtlety. This is because it compares field names and values not memory addresses. Additionally field order is not considered. This is the reason why we treated the __PARSING_INFO__ field differently before because this assertion is sensitive to the order of the contents of __PARSING_INFO__ (it does an as-is comparison instead of allowing different orders).
This change keeps the spirit of the original test while eliminating failures caused solely by allowed (but previously unexpected) reordering.
Verifying this change
This change is already covered by existing tests, such as
org.apache.pulsar.client.impl.schema.ProtobufSchemaTest#testParsingInfoProperty.Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: LucasEby#2