LLM Systems Lab ↗In progressA GPT built from scratch in PyTorch, with experiments on how model size and attention affect runtime and memory.A learning project, still in progress.GitHub ↗In progress · PyTorch×LLM Systems LabMy learning project: building a GPT and measuring how its implementation affects time and memory.What’s thereA GPT implementation, training and text generation, GPT-2 weight loading, and synthetic benchmarks. The experiments compare model depth, memory use, and attention implementations.What’s nextKV caching and inference benchmarks. The work is unfinished, and I’m still exploring the design and measurements.View on GitHub ↗