detokenization parallelization#37

Open

NickNickGo wants to merge 17 commits intomicrosoft:mainfrom

NickNickGo:detokenization

Commits on Sep 8, 2020

detokenization parallelization
NickNickGo
committed
minor changes
NickNickGo
committed

Commits on Sep 9, 2020

adding arguments to Postprocess
NickNickGo
committed
adding arguments to Postprocess
NickNickGo
committed
updating throughput
JiushengChen
committed

Commits on Sep 10, 2020

Ensuring in-order writes
NickNickGo
committed
minor comments
NickNickGo
committed

Commits on Sep 12, 2020

linting checks
NickNickGo
committed

Commits on Sep 16, 2020

formatting changes
NickNickGo
committed

Commits on Sep 25, 2020

Multi-worker preprocess and fetch
NickNickGo
committed

Commits on Sep 29, 2020

nitpicks
NickNickGo
committed

Commits on Oct 30, 2020

argument description added
NickNickGo
committed

Commits on Nov 11, 2020

benchmarks updated
NickNickGo
committed
readme Updated
NickNickGo
committed
Merge branch 'main' into detokenization
NickNickGo
authored

Commits on Nov 12, 2020

readme update
NickNickGo
committed
Merge branch 'detokenization' of https://github.com/NickNickGo/fastseq into detokenization
NickNickGo
committed