Format proposal feedback #1
Description
Hi, I'd like to give my 2c regarding the work-in-progress format proposal.
My main concern is regarding the adoptability and future-proofness. One idea that is good to keep in mind is that there is an "implicit" goal that I believe applies for all standards, that is to aim for wide adoption and having it made general enough to not get obsolete as tech evolves.
getting all inference frontends and trainers to agree on this format.
An important actor is not considered, namely the research groups that release novel models. I believe supporting them is crucial for the standard to succeed.
A specific use case I had in mind while reviewing is that a research group comes up with a new paper that involves a novel model pipeline - for example let's say it's still some kind of diffusion model but with a completely new pipeline, custom tokenizer etc. And they heard about the initiative and want to use this new standard format to release it. What will happen..? They can't, because the proposed format requires a known base-type to be specified in the metadata. So they just release it in some non-standard way - then either it will never adopt the OMI standard, or the standard has to be updated to a new version each time, which is terrible.
Here is the problematic part in the proposal:
"pipeline": {
// The type comes from a pre-defined set of supported pipelines.
"type": "SDXL",My point is that the standard should not restrict "support" for specific known pipelines and instead allow backends to support or not support the pipeline described in the metadata.
Now, I don't have specific suggestions on how to solve this, but I have two directions in mind. 1) describe the pipeline fully, which might be overkill. 2) Allow the type field to hold custom names instead of pre-defined known names (If you feel like this list of known models is essential then I suggest it to be specified in a separate standard, but I see no point).
The same concern comes up with the prediction_type field btw. (Also if you think about it, currently it makes no sense to include this field since the pipeline's base_type already implies it, right? But there would be a place for it pipeline types were not predefined!)
I have some smaller nitpicks too but those can wait until the proposal is finalized. Let me know if it makes sense what I'm writing, glad to see the initiative off to a good start!