Turn Any Idea Into a Story.
Engineered in C++17.
Nano-Mind is a zero-dependency GPT-2-style language model running on native CPU vector intrinsics. Built completely from scratch to deliver ultra-smooth streaming generation with zero expensive GPU infrastructure.
1,000+
Active Users
82.3 tok/s
Median Speed
97%
Success Rate
$0 GPU
Pure CPU Power
Serving 1,000+ Users Across the World
Nano-Mind has served over 1,600+ real generation requests on Hugging Face Spaces free-tier infrastructure, proving that custom CPU systems engineering can rival GPU speeds with zero cloud spend.
Generate a Story in Real-Time
Type an imaginative opening line or choose an idea below. Watch tokens stream directly from our compiled C++ engine.
How I Built Nano-Mind from Scratch
No PyTorch runtime. No ONNX. No black-box frameworks. Here is how the custom C++17 engine was engineered from the mathematical foundations to production.
C++17 Transformer Math
Implemented the complete GPT-2 transformer architecture in pure C++17: multi-head self-attention, layer normalization, exact GELU activation via tanh approximation, and stochastic top-k sampling.
Zero Deep Learning FrameworksAVX2 & FMA SIMD Acceleration
Hand-optimized core matrix-vector multiplications using 256-bit x86 vector registers with Fused Multiply-Add
(_mm256_fmadd_ps), calculating 8 single-precision float operations per clock cycle.
Autoregressive KV-Cache
Designed pre-allocated Key-Value caches across all layers to store past token representations. This eliminates O(n²) attention recalculations, turning autoregressive token decoding into constant O(1) steps.
O(1) Step DecodingZero-Copy Binary Model Loader
Engineered a proprietary binary model layout (model.bin) loaded directly via pointer arithmetic.
Weights are mapped directly into contiguous RAM buffers, eliminating expensive memory duplication.
OpenMP Multi-Core Parallelism
Distributed multi-head attention loops and feedforward projection passes across multiple CPU cores using OpenMP pragmas, achieving near-linear scaling on modern multi-threaded CPUs.
Parallel Multi-Head AttentionLive SSE Streaming Backend
Created a lightweight server layer that bridges OpenAI's 50,257-token BPE vocabulary via tiktoken
and pushes generated tokens live to the browser over Server-Sent Events with under 40ms TTFT.
End-to-End Inference Flow
The complete journey of your prompt from browser click to streaming story.
1. Browser Client
Sends JSON payload (prompt, max_tokens, temperature, top_k) to /generate endpoint.
2. Server Dispatcher & Tiktoken BPE
Tokenizes user prompt into GPT-2 token IDs using Byte-Pair Encoding and opens SSE stream.
3. Compiled C++17 Inference Engine
Runs forward pass through mapped weights with AVX2 SIMD intrinsics & OpenMP CPU threads.
4. Server-Sent Events (SSE) Stream
Decodes generated token IDs into text chunks and streams data: {"token": "..."} live.
5. Real-Time Browser Rendering
Renders each token live on the client canvas with blinking cursor and performance metrics.
Live Hugging Face Spaces Benchmark
Measured strictly on free-tier cpu-basic infrastructure under real production conditions.
https://not-omega-nanomind.hf.space using 20 sequential runs with 3 warmup cycles.
Throughput and latency include full network round-trip, Hugging Face reverse-proxy routing, BPE tokenization, C++ engine execution, and SSE chunk decoding.
Sumeet Khatri
Hey there! I am Sumeet, creator of Nano-Mind. I specialize in building custom, zero-dependency deep learning inference engines, hardware-accelerated computational kernels (AVX2, FMA, SIMD intrinsics, OpenMP), and high-throughput production backends. I am currently actively looking for full-time and contract opportunities in AI Systems, Performance Optimization, and Core Infrastructure. Let’s connect!
Watch the Engine in Action
See Nano-Mind run real-time inference directly in the terminal and browser, showing prompt tokenization, AVX2 SIMD forward pass execution, and streaming output.
Watch on YouTube