GitBross Home Explore Help
Register Sign In
sol_87yf6cwp
/
LLMs-from-scratch
mirror of https://github.com/br8bit/LLMs-from-scratch.git
1
0
Fork 0
Files Issues 0 Wiki
Tree: 861c296312
Branches Tags
LLMs-from-scrat...
/
ch04
/
01_main-chapter-code
rasbt 861c296312 add imports and version on top 1 year ago
..
figures f057156181 use smaller number of tokens to emphasize next token prediction goal 1 year ago
README.md 3a5fc79b38 add and update readme files 1 year ago
ch04.ipynb 861c296312 add imports and version on top 1 year ago
exercise-solutions.ipynb e0df4df433 add dropout for embedding layers 1 year ago
gpt.py da33ce8054 remove redundant unsqueeze in mask 1 year ago
previous_chapters.py da33ce8054 remove redundant unsqueeze in mask 1 year ago

README.md

Chapter 4: Implementing a GPT model from Scratch To Generate Text

  • ch04.ipynb contains all the code as it appears in the chapter
  • previous_chapters.py is a Python module that contains the MultiHeadAttention module from the previous chapter, which we import in ch04.ipynb to create the GPT model
  • gpt.py is a standalone Python script file with the code that we implemented thus far, including the GPT model we coded in this chapter