Matthew Hernandez 6f12edb0cc Fix issue: 731 by resolving semantic error (#738)		пре 4 месеци
..
README.md	a22d612be6 Bonus material: extending tokenizers (#496)	пре 10 месеци
bpe-from-scratch.ipynb	6f12edb0cc Fix issue: 731 by resolving semantic error (#738)	пре 4 месеци
tests.py	6f12edb0cc Fix issue: 731 by resolving semantic error (#738)	пре 4 месеци

Byte Pair Encoding (BPE) Tokenizer From Scratch

огледало од https://github.com/br8bit/LLMs-from-scratch.git

bpe-from-scratch.ipynb contains optional (bonus) code that explains and shows how the BPE tokenizer works under the hood.