Can we simplify Ion 1.1? #397
Description
TLDR; Ion 1.1, when considering the costs and benefits from a business perspective, does not seem to be a feasible product, and the specification itself seems to be at the root of key performance issues. Can we simplify Ion 1.1, or skip Ion 1.1 and create a simpler Ion 1.2?
Attempts to implement an Ion 1.1 reader in ion-java have run into issues with performance of macro evaluation. I (and others) have tried multiple different strategies for implementing a macro evaluator that fits in the current IonReader APIs. We have also tried implementation strategies that are unconstrained by the existing IonReader API. In all of our attempts, we have not been able to get Ion 1.1 with macros to reach the read throughput of Ion 1.0 despite multiple people working on this for over a year.
The read throughput is a significant problem—we don't think anyone will actually use Ion 1.1 macros if it results in a 40% or more drop in read throughput. Furthermore, the complexity of the specification makes it difficult and time-consuming to iterate on different implementation ideas. This is anecdotal, but I have made a few different Ion 1.0 binary readers in the last few years, and the time for me to do that is usually on the order of 1-2 days to get something reasonably performant. Attempting to implement the current Ion 1.1 spec takes significantly longer (several weeks) just to get to simple module management and basic template macro evaluation, and there's a long tail of several months to complete all of the system macros and special forms.
For a long time, we thought that the problem was in the way we were implementing it in ion-java because the ion-rust experimental 1.1 implementation benchmarks seemed to prove that there as a high performance ceiling. However, we recently realized that the measurements we were using were not up-to-date and did not accurately reflect the current Ion 1.1 specification; ion-rust suffers from similar performance issues as ion-java.
Rather than throw away Ion 1.1 entirely, I attempted an experiment with a simplified Ion templates implementation that's closer to RFC: Ion Templates.
Not only did it take only about 1 week to implement (without even vibe-coding any of it), prior to any performance optimization, the read throughput was comparable to Ion 1.0 read throughput. Cleaning up the obvious bits of lazy/bad, slow code resulted in cases where the simplified templates were ~15% better read throughput that Ion 1.0 and 100-200% better read throughput than our best attempts at implementing the current Ion 1.1 specification.
In my experiment, key changes compared to the current Ion 1.1 specification include:
- No system macros or special forms
- Templates must produce exactly one value
- Arguments must be 0 or 1 value
- Macro/template signatures are no explicitly declared. They are derived from the location of placeholders in the template body, and the Arguments must be used exactly once, in the order they are declared
- Templates cannot invoke or call other templates
- Templates cannot produce system directives. (They can produce certain no-op system values, for backwards compatibility with Ion 1.0)
- No more Template Definition Language, so that macros are simple templates without any computational power. (There's a new syntactical construct to represent placeholders in a template.)
Even with these constraints, I believe that we could still provide significant value to 99% of current Ion users.
So, do we pivot to a simplified templates project? If we do, is it a simplification of Ion 1.1, or do we give up on Ion 1.1 and create a new Ion 1.2?