Offsets for attribute keys and values

[not-my-profile]

I want to use Djot for an application where I want to programmatically update certain attribute values so having the parser provide offsets for attribute keys and values would be really useful.

[hellux]

This is sort of possible with the current version (0.6.0). When using the
Parser::into_offset_iter, one can obtain the offset of each event, and the
attributes are part of the Start event for block containers and part of the
End event for inline containers. E.g

{.block}
para{.inline}

yields

Start(Paragraph, {.block}) "{.block}\n" 0..9   <-- block attributes
Start(Span, {.inline}) "" 9..9
Str("para") "para" 9..13
End(Span) "{.inline}" 13..22                   <-- inline attributes
End(Paragraph) "" 23..23

We cannot determine where each key and value is but we can know where the full
attribute set is. It is therefore possible to modify the Attributes of the
container and then just replace the original attributes in the source with the
newly modified one. We can use the Debug implementation of
Attributes which outputs itself in valid djot format.

One caveat is that the spans may contain more than just the attributes. For
block attributes there may be some trailing newline and start of the container,
and for inline attributes it may start with the end of the container, e.g.

{a=1 b=2}
> [quote]{b=2 c=3}

yields

Start(Blockquote, {a="1" b="2"}) "{a=1 b=2}\n>" 0..11   <-- extra \n>
Start(Paragraph, {}) "" 12..12
Start(Span, {b="2" c="3"}) "[" 12..13
Str("quote") "quote" 13..18
End(Span) "]{b=2 c=3}" 18..28                           <-- extra [
End(Paragraph) "" 28..28
End(Blockquote) "" 28..28

We can find the block attributes by stripping out everything after the last '}'
and we can find the inline attributes by stripping out everything before the
first '{'.

With that we can now modify and overwrite the attributes in the original djot
source:

let src = concat!(
    "{a=1 b=2}\n",
    "> [quote]{b=2 c=3}\n", //
);

let mut out = String::new();
let mut pos = 0;
let mut modify_attrs = |span_attrs: std::ops::Range<usize>, mut attrs: jotdown::Attributes| {
    let mut modified = false;
    for (k, v) in &mut attrs {
        if k.key() == Some("b") {
            *v = "modified".into();
            modified = true;
        }
    }
    if modified {
        out.push_str(&src[pos..span_attrs.start]);
        out.push_str(&format!("{attrs:?}"));
        pos = span_attrs.end;
    }
};

let mut inline_attrs = Vec::new();
for (e, span) in jotdown::Parser::new(src).into_offset_iter() {
    match e {
        jotdown::Event::Start(c, attrs) if c.is_block() => {
            let span_attrs = span.start
                ..(span.start + src[span.clone()].rfind('}').map_or(0, |p| p + 1));
            modify_attrs(span_attrs, attrs);
        }
        jotdown::Event::Start(_, attrs) => inline_attrs.push(attrs),
        jotdown::Event::End(c) if !c.is_block() => {
            let span_attrs =
                (span.start + src[span.clone()].find('{').unwrap_or(span.len()))..span.end;
            modify_attrs(span_attrs, inline_attrs.pop().unwrap());
        }
        _ => {}
    }
}
out.push_str(&src[pos..]);

assert_eq!(
    out,
    concat!(
        "{a=\"1\" b=\"modified\"}\n",
        "> [quote]{b=\"modified\" c=\"3\"}\n", //
    )
);

Some drawbacks:

• All values normalize to always use quotes, so neighboring values may be
  modified. Consequtive sets (e.g. {.a}{.b}) will also be merged into a single
  one.

• This is not a completely general solution, it will break for e.g. multi-line
  attributes within an inner block. E.g

  > word{b=2 c="multiple
  > lines"}
  

  which yields

  Start(Blockquote, {}) ">" 0..1
  Start(Paragraph, {}) "" 2..2
  Start(Span, {b="2" c="multiple lines"}) "" 2..2
  Str("word") "word" 2..6
  End(Span) "{b=2 c=\"multiple\n> lines\"}" 6..32          <-- inbetween >
  End(Paragraph) "" 33..33
  End(Blockquote) "" 33..33
  

  This has attributes that span multiple lines and the above method would fail
  to maintain the inbetween blockquote. However, this would fail even if we had
  access to the span of every attribute value.

[not-my-profile]

Thanks for the quick answer!

This also doesn't account for attributes transferred from reference definitions, e.g:

{title=foo}
[ref]: /url

[ref][]

But I guess that's alright for a start.

However, this would fail even if we had access to the span of every attribute value.

Yeah after creating the issue I also realized that attribute values in Djot can be made up of multiple spans (due to the class attribute syntax and multi-line attributes). I'll revisit this issue if I end up needing more accurate spans.

[hellux]

This also doesn't account for attributes transferred from reference definitions

Yeah, true. A single `Attributes` set can also be the union of those from the link and those from the definition, a span for each attribute pair (or per key and value) could solve this though. I guess the drawback would be a more cluttered API and some minor overhead when unused.

Yeah after creating the issue I also realized that attribute values in Djot can be made up of multiple spans (due to the class attribute syntax and multi-line attributes).

The individual class attributes are actually preserved by the `Attributes` object, they are only merged when using the `Attributes::unique_pairs` method.