Nanochat

Open-source blueprint for training a Claude Code-like agent in pure JAX on TPUs, detailing model sizes, cost, data/engineering steps, and Constitutional AI alignment.

A dense, data-driven diary of training a GPT-2 small base model from scratch on a single RTX 3090, detailing data prep, tokenization, sequence handling, speed tests, checkpoints, validation, and comparisons to OpenAI weights, with conclusions about feasibility and gaps to achieve a true Chinchilla-optimal train.