GitHub - HardikJain02/GPT2: GPT2 [125M] from Scratch

Check Task Implementations [PDF] for detailed implementation of each task.

Each task can be run directly on colab, except ddp and fsdp training code as they need more than a single gpu. [no need for any dependencies]

Name		Name	Last commit message	Last commit date
Latest commit History 5 Commits
Q_A Assignment [Hardik Jain] .pdf		Q_A Assignment [Hardik Jain] .pdf
README.md		README.md
Task Implementations [Hardik Jain].pdf		Task Implementations [Hardik Jain].pdf
Task1.ipynb		Task1.ipynb
Task2GQA.ipynb		Task2GQA.ipynb
Task2Rotary.ipynb		Task2Rotary.ipynb
Task2Sliding.ipynb		Task2Sliding.ipynb
Task3.ipynb		Task3.ipynb

Provide feedback