Skip to content

Iterator refactor#20

Merged
OverkillGuy merged 12 commits intomasterfrom
iterator-refactor
Apr 1, 2021
Merged

Iterator refactor#20
OverkillGuy merged 12 commits intomasterfrom
iterator-refactor

Conversation

@OverkillGuy
Copy link
Owner

New (unused) iterators for simplifying encoding,

BufferedIterator wraps the read N bytes issue fine (based on PR #1), and ChunkIterator supposed to return a full chunk string on next(). This works, but the base64 is not integrated yet.

Unit Tests are RED on the ChunkIterator, due to comparing byte array to string.

Open question is how to integrate base64 conversion inside the process, when the base64::EncoderWriter approach is a Writer trait, not a Reader trait, so I can't make a oneliner pipeline cleanly, it seems.

@danieleades
Copy link
Contributor

Open question is how to integrate base64 conversion inside the process, when the base64::EncoderWriter approach is a Writer trait, not a Reader trait, so I can't make a oneliner pipeline cleanly, it seems.

you can either wrap a Reader with base64::read::DecoderReader or wrap a Writer with base64::read::EncoderWriter depending on where in your pipeline you want to do the conversion

@OverkillGuy
Copy link
Owner Author

OverkillGuy commented Mar 21, 2021

you can either wrap a Reader with base64::read::DecoderReader or wrap a Writer with base64::read::EncoderWriter depending on where in your pipeline you want to do the conversion

Yeah, but I'm annoyed because I want a Reader to do encoding but this lib doesn't have it.

Looking around I see another lib: base64-stream has ToBase64Reader, which fits more what I want, despite less downloads... To be continued...

EDIT: I want to clarify, I do like the Iterator approach to generating chunks, which means I want to stay on the side of Reader-conversion.

This was referenced Mar 21, 2021
@OverkillGuy
Copy link
Owner Author

Since the actual bugfix was covered elsewhere, but the refactor is still useful alone, I opened #21 to specifically address this refactor.

Inspired by / stolen from PR #1

Not called ChunkIterator because a more specific "chunk iterator" will
be created next for converting text into chunk string content (base64
incl. prefix)
Does not do base64 yet
Assume readable text string is the input (likely base64-encoded
binary) instead of random binary to base64 encode ourselves
Was hardcoded to 10, now done properly.

We calculate it as naive total / readsize, rather than including the
overhead in calculation, which would lead to counterintuitive results:
When reading N bytes from a 2N stream, you expect 2 read operations,
not 3 (overhead of chunk prefix = 8 bytes per chunk)
@OverkillGuy OverkillGuy marked this pull request as ready for review March 28, 2021 12:23
YAML optional dep was never used in master, only experimented with.
Using feedback from PR #24 to make this code better.
"Panic message not a string litteral" solved now
@OverkillGuy OverkillGuy merged commit 332c845 into master Apr 1, 2021
@OverkillGuy OverkillGuy deleted the iterator-refactor branch April 1, 2021 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants