Skip to content

Add fixes for complex type schemas, handle inline structs and others#1

Merged
wirelessr merged 5 commits intowirelessr:mainfrom
zenc0ded:dan/schema_compat
Mar 11, 2025
Merged

Add fixes for complex type schemas, handle inline structs and others#1
wirelessr merged 5 commits intowirelessr:mainfrom
zenc0ded:dan/schema_compat

Conversation

@zenc0ded
Copy link
Contributor

Added following fixes/functionality:

  • Support inline structs, these are transparent in the Avro itself but need to pull in the fields to the parent record.
  • for supported complex schema types (records, maps, arrays) embed the full schema as required by spec - this also allows for nested records in non-omitempty cases.
  • Added tag parity for bson.
  • For multiple occurrences of a record type need to state the previously defined 'name' as the type. Keep a simple map for this. Im sure it could be done in a better way.
  • Added some options to use the go field names as the Avro names, rather than what is defined in json/bson. Probably not widely desired functionality but I need it and so tried to hack it in without breaking too much other stuff.

Tried to keep things backwards compatible where possible but for the complex record types its definitely changed the schema... I'm unsure what the actual use cases were for these previous schema in reflect_test.go but they weren't 'valid' according to avro-tools or hamba/avro. I believe they now are so should be a positive step for anyone wanting this functionality.

- For complex types the type field needs to be a fully defined schema,
  rather than the name of the complex type such as 'array', 'map'.
- Add test case for inline structs with json tag.
Add a couple of options:
- one to prefer the go field name over any given json/bson name
- another to add fields even if there are no json/bson tags
@zenc0ded zenc0ded marked this pull request as draft February 23, 2025 05:45
@zenc0ded zenc0ded closed this Feb 23, 2025
@zenc0ded
Copy link
Contributor Author

oops! Hadn't meant to raise against upstream, feel free to take anything useful from here if it helps!

@wirelessr
Copy link
Owner

I feel this PR is great, and I will take more time to read it seriously.

@wirelessr wirelessr reopened this Feb 27, 2025
@wirelessr wirelessr marked this pull request as ready for review February 27, 2025 10:06
@wirelessr wirelessr merged commit 4b7508f into wirelessr:main Mar 11, 2025
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants