unnamed_type error with primary type schema

[adjivas]

Hello,

Do is it possible to register a primary type' schema with Avrora?

A primary type schema is by example that:

{
    "type": "string",
    "name": "myvalue"
}

When I try to save it with the register_schema_by_name function:

Avrora.Utils.Registrar.register_schema_by_name("myschema-value")

This error happen:

13:02:04.414 [debug] reading schema `myschema-value` from the file /app/priv/schemas/myproject/myschema-value.avsc
{:error, :unnamed_type}

[Strech]

Hi @adjivas 👋🏼

Initially Avrora was designed to work with Record schemas which allows you to have nested schemas of complex types. But there was no reason to register a primary type (also can't find that definition in Avro specs) yet.

Probably that type can be registered as a part of another Record schema.

[adjivas]

Hi @Strech, sorry for the late reply.
There you can found a official list and a documented example of Primitive Types from the Avro specs:

{"type": "string"}

Primitive Types exists to the side of Complex Types.

[Strech]

Sorry for a long pause, I will take a look @adjivas and drop a message of what we can do about it

[Strech]

I've re-read the documentation (also check the erlavro source code). And the specification says https://avro.apache.org/docs/1.11.1/specification/#primitive-types

Primitive types have no specified attributes.
Primitive type names are also defined type names. Thus, for example, the schema “string” is equivalent to: {"type": "string"}

Then I checked the erlavro and their parsing mechanism

iex(1)> :avro_json_decoder.decode_schema(~s({"type":"string","name":"MyString"}), allow_bad_references: true)   
{:avro_primitive_type, "string", []}

And the result is exactly as stated in the specification, primitive types can't have any attributes thus I don't see any in the output.

Here is another answer on that topic: https://stackoverflow.com/questions/66210730/aliases-for-primitive-types-in-avro

TL;DR You can't reference primitive type by alias (our new name perse)

If you have further question, feel free to drop them here

[rewritten]

It is true that a primitive type cannot have a name, but unnamed types can be used in registries, the clearest example is a union (see https://docs.confluent.io/platform/current/schema-registry/fundamentals/serdes-develop/serdes-avro.html#multiple-event-types-in-the-same-topic).

If a registry has a schema that, instead of being a record, is a union (that is, in JSON, an array of other types, typically records), and Avrora is asked to decode a message marked with that id, then it will fail to transform the downloaded JSON into a valid schema to decode, instead returning {:error, :unnamed_type}.

@Strech do you agree that this should be opened back?

I might give it a try in that case.

[Strech]

@rewritten could you please provide an example schema you've mentioned? Maybe it's indeed an issue. It also could be that it's just a case so rare/broken that it makes no sense to put effort.

[rewritten]

Sorry I did not notice the response (not my main work account).

Another very common case is decoding message keys. Usually they are strings or numbers, but still avro-encoded in the message payload. In that case the schema will be just "string".

In case of unions, the schema is an array:

[
  {"type": "record", "name": "foo", "fields": [{"name": "foo_field", "type": "string"}]},
  {"type": "record", "name": "bar", "fields": [{"name": "bar_field", "type": "string"}]}
]

Currently, only schemas that are JSON objects with a "name" field at their root are supported (i.e., records, enums, and surprisingly fixeds), instead unions and basic types fail.

[Strech]

Thanks for the additional details. I still need to wrap my head around it, but it feels like an issue now. It would be cool to have a basic failing example, like - this is the schema, here is it in the file, here is how I encode/decode, here is an error.

Because then I can iterate over it until we have a working solution. I will come to this issue right after decoding the logical types issue.