Mini LLM From Scratch
Transformer-based chatbot built using pure NumPy and manual backprop
A tiny Transformer-based conversational model built from the ground up using exclusively NumPy. Implements multi-head self-attention, feed-forward layers, manual matrix backpropagation, and positional encodings without PyTorch or external deep learning frameworks.
- Custom multi-head scaled dot-product self-attention mechanism
- Exact mathematical backpropagation for all linear and attention projection layers
- Custom vocabulary tokenizer, cross-entropy loss, and temperature sampling inference loop
