Byte-pair encoding by hand on a toy vocabulary of eight words.
What I worked on
Foundations
How language models work
– (1h 20m)
Fictional notes on tokenization
Text
Byte-pair encoding by hand on a toy vocabulary of eight words.
– (1h 20m)
Fictional notes on tokenization
Text