How To Build A GPT

Click here for the feature article “I Built A GPT”

Pre-requisites 

• “Introduction to matrices”, Sal Khan (Khan Academy)

• “Matrix multiplication (part 1)”, Sal Khan (Khan Academy)

• “Essential Matrix Algebra for Neural Networks, Clearly Explained!!!”, Josh Starmer (StatQuest)

• “Newton’s Method”, PatrickJMT

• "Gradient Descent, Step-by-Step”, Josh Starmer (StatQuest)

• “Stochastic Gradient Descent, Clearly Explained!!!”, Josh Starmer (StatQuest)

Understanding The Concepts 

• “But what is a GPT? Visual intro to transformers”, Grant Sanderson (3Blue1Brown) 

• “Attention in transformers, visually explained”, Grant Sanderson (3Blue1Brown) 

• "How might LLMs store facts”, Grant Sanderson (3Blue1Brown) 

Down the Rabbit Hole 

• [For Python beginners] “Create a Large Language Model from Scratch with Python – Tutorial”, Elliot Artledge (FreeCodeCamp)

• [For Python users] “Let’s build GPT: from scratch, in code, spelled out”, Andrej Karpathy

Digging Deeper

• “Neural Networks and Neural Language Models”, Natalie Parde

• “50 Days of Deep Learning”, Misra Turp