Compared token counts of the same paragraph in three languages.
What I worked on
Foundations
How language models work
– (2h 15m)
Fictional notes on tokenization
Text
Compared token counts of the same paragraph in three languages.
– (2h 15m)
Fictional notes on tokenization
Text