microDINOv3

A recreation of the DINOv3 self-supervised Vision Transformer training pipeline in pure Python, with no external ML framework underneath it — the autograd engine and the transformer are both hand-written.

Built with: Pure Python, Computer Vision

GitHub: RyanKim17920/microDINOv3

• Hand-written reverse-mode autograd and ViT, with zero external ML framework dependencies
• Student-teacher EMA distillation with the DINO and iBOT objectives, plus KoLeo regularization
• Axial RoPE position encoding with register tokens
• Gram anchoring, the piece DINOv3 adds to keep dense features from degrading over long training

February – April 2026