--- title: Neural Network from Scratch url: https://rone.dev/projects/neural-network-from-scratch description: MNIST at 97.9% validation accuracy, with every layer and gradient written in NumPy. language: en-US updated: 2026-09-27 translations: id: https://rone.dev/id/projects/neural-network-from-scratch --- # Neural Network from Scratch MNIST at 97.9% validation accuracy, with every layer and gradient written in NumPy. [Projects](https://rone.dev/projects) / [AI](https://rone.dev/projects/category/ai) Research · Started 2025 · 0 GitHub stars · 0 Forks A feed-forward network assembled from parts rather than imported: dense and dropout layers, six activation functions, five loss functions, and mini-batch training with momentum and early stopping — all in NumPy. PyTorch appears exactly once in the repository, to download MNIST; nothing in the model touches it. ## Numbers * **97.9%** Validation accuracy * **150** Epochs * **0** Framework dependencies ## Project facts | | | | -------- | -------------------------------------------------------------------- | | Status | Research | | Category | AI | | Started | 2025 | | Role | Research and implementation | | Stack | Python, NumPy, Matplotlib, Tkinter | | Licence | CC0-1.0 | | Links | [Source](https://github.com/ridwaanhall/Neural-Network-from-Scratch) | ## Overview Calling a framework’s fit method teaches you the API. Writing the backward pass teaches you why the API has the shape it does. This is the second one: forward and backward for every layer, the chain rule applied by hand, and gradients that are wrong in obvious ways until they are not. The recorded run reaches 98.9% training and 97.9% validation accuracy on MNIST after 150 epochs — close enough to a framework baseline to show the mathematics is right, which is the only claim a project like this should make. ## What is actually implemented Dense layers with a choice of weight initialisation, dropout that knows the difference between training and inference, six activations, five losses including Huber and the numerically stable softmax-with-cross-entropy pairing, mini-batch shuffling, momentum, and early stopping on validation loss. The layers share one small interface — forward, backward, update — so a network is a list of things that satisfy it. That constraint is what keeps the code readable at this size, and it is the same constraint every framework arrives at. ## A harder test than the test split A drawing canvas ships alongside the training code: pick a saved model, draw a digit with the mouse, and see the prediction and its confidence. This matters more than it looks. MNIST digits are centred, size-normalised and drawn with a specific stroke weight; a mouse-drawn digit is none of those. The gap between a held-out score and what happens on genuinely new input is the single most useful thing a beginner can be shown, and it is much more convincing to draw it yourself than to read about distribution shift. A command-line draft helper trained on 128 heroes of synergy and counter data. Draw a character; four convolutional models read it back, in Latin or Arabic.