AI
Inkling 2 Mini: Looped MoE Architecture, Independent Benchmarks, and Pricing Trade-Offs
Thinking Machines Lab released Inkling 2 Mini on November 20, 2026 under Apache 2.0—a 26.4B-total, 3.4B-active looped MoE model that packs 48 layer passes per token into a 20 GB NVFP4/MXFP8 checkpoint. Independent tests show it ties Qwen3.8-27B on the Artificial Analysis Intelligence Index at one-sixth the task cost, though verbosity and runaway loops require strict turn caps.
9 min read
Inkling 2 Mini: Looped MoE Architecture, Independent Benchmarks, and Pricing Trade-Offs
Executive Summary
On November 20, 2026, Thinking Machines Lab released Inkling 2 Mini, an open-weight multimodal (text and image input, text output) model distributed under the Apache 2.0 license (Model Card, Hugging Face). Rather than scaling parameter count for local hardware, Inkling 2 Mini uses a looped Mixture-of-Experts (MoE) transformer with 26.4 billion total parameters and 3.4 billion active parameters per token, running its middle 12 layers three times to reach 5.4 billion effective weight activations per token in an approximately 20 GB (19,526,019,168-byte) NVFP4/MXFP8 checkpoint (Kingy AI, vLLM Recipes).
Independent evaluations published on November 20–21, 2026 highlight a sharp cost-versus-reliability trade-off:
- Competitive Compact Intelligence at Lower Task Cost: Inkling 2 Mini scores 20 on the Artificial Analysis Intelligence Index v5.0.1—tying
Qwen3.8-27B(xhigh) andNanbeige4.3-33B—while averaging $0.59 and 9.0 minutes per task on Tinker's$0.15/$0.50per 1M token rates versus$3.53and38.2minutes forQwen3.8-27B. - Verbosity and Runaway-Loop Risk: In Artificial Analysis's run, Inkling 2 Mini emitted
161Moutput tokens (nearly double the85.7Mpeer median), and in Plotly's Data Analytics Benchmark, a single 585-query runaway loop pushed its total run cost to $4.09—more than doubleInkling 2 Small's $1.81. - Context Window & Reliability Caveats: While the model card advertises up to 1M tokens, Tinker's hosted sampling and training tiers expose 64K and 256K windows (OrcaRouter). Both the model card and independent testers warn of confident factual hallucinations and unconfirmed destructive commands without sandboxing.
Architecture & How It Works
Inkling 2 Mini pairs aggressive expert pruning with weight-shared looping and native 4-bit training so a 26.4B model fits comfortably on 32 GB+ Apple Silicon Macs or single-GPU workstations (vLLM Recipes, OrcaRouter):
| Specification | Verified Value | Source |
|---|---|---|
| Architecture | Looped Mixture-of-Experts (MoE) transformer + vision encoder | Model Card / Kingy AI |
| Parameters | 26.4B total; 3.4B active; 5.4B effective activations/token |
Kingy AI |
| Layer Structure | 24 stored layers (48 layer applications/token via 3x core loop); 8 of 128 experts/layer |
vLLM Recipes / Kingy AI |
| Precision & Size | ~20 GB checkpoint (20 layers NVFP4, final 4 layers MXFP8); DSpark speculator |
vLLM Recipes / Kingy AI |
| Modalities | Text and image inputs; text output | Model Card |
| Context Window | Up to 1M tokens (Model Card / AA); 64K / 256K tiers on Tinker |
OrcaRouter |
| License & Release Date | Apache 2.0 — Released November 20, 2026 |
Model Card |
How the Three-Pass Looped Core Executes
According to vLLM's architecture breakdown, the 24 physical layers split into a prelude (layers 0–5), a loop core (layers 6–17), and a coda (layers 18–23). Instead of loading 48 distinct layers into VRAM, every token traverses the 12 core layers three times:
[Input Tokens / Vision Embeddings]
│
▼
Prelude (Layers 0–5)
│
▼
┌──────────────────────────────────────────────────────┐
│ Loop Core (Layers 6–17) × 3 Passes │
│ • 8 of 128 experts active per layer │
│ • Pass 2 & 3 merge prior attention-residual blocks │
│ via gated rank-64 controller │
│ • Separate KV & conv-state cache per pass (48 total)│
└──────────────────────────────────────────────────────┘
│
▼
Coda (Layers 18–23, MXFP8 on final 4 layers)
│
▼
[Output Token / DSpark Speculator]
Because each pass maintains its own KV cache and convolutional-state cache (48 caches total), prefix caching and CUDA graphs remain compatible in vLLM, while SGLang added day-one support (PR #51926) with NVFP4 grouped-GEMM paths (vLLM Recipes, OrcaRouter). Thinking Machines built the weights by pruning two-thirds of experts (keeping 128 of 384), healing on 1T distilled tokens, and running SFT/RL over 2.2T tokens in 2,000 steps—trained end-to-end in 4-bit rather than quantized post-hoc.
# Local quickstart via Ollama (text + image input; requires >= 32 GB RAM/VRAM recommended)
ollama run inkling-2-mini
# Hosted Tinker API model IDs (64K default vs 256K PEFT context tier)
# Standard sampling/training: thinkingmachines/Inkling-2-Mini
# Extended 256K tier: thinkingmachines/Inkling-2-Mini:peft:262144
Benchmarks, Cost & Latency Analysis
Vendor Claims vs. Independent Evaluations
Thinking Machines' launch benchmarks highlight strong agentic and software-engineering scores for a 20 GB model: 48.2% on DeepSWE v1.1 (vs. 42.2% for Qwen3.8-27B), 77.9% on the 51-task SWE-bench Pro v2 Hard subset (vs. 73.8%), 79.0% on BrowseComp with compaction (vs. 61.6%), 85.3% on DeepSearchQA, and 82.7% on MCP Atlas—while trailing Qwen3.8-27B on Terminal-Bench v2.1 (71.6% vs. 73.0%) and scoring 46.4% on MRCR v2 long-context coreference (Kingy AI, OrcaRouter).
Independent leaderboards and hands-on runs clarify where Inkling 2 Mini excels and where it lags:
| Model | License / Type | AA Intelligence Index v5.0.1 | Vals Index v2.2 | API Price (In / Out per 1M) | AA Task Cost & Time |
|---|---|---|---|---|---|
| Inkling 2 Mini | Apache 2.0 (Open) |
20 |
46.46% ± 1.11 |
$0.15 / $0.50 (Tinker); $0.10 / $0.40 (OpenRouter) |
$0.59 / 9.0 min |
| Qwen3.8-27B | Open Weight | 20 (xhigh) |
— | $0.50 / $3.00 (list); $0.33 / $2.40 (OrcaRouter) |
$3.53 / 38.2 min |
| Inkling 2 Small | Open Weight | 31 |
56.08% |
$0.30 / $1.00 (Tinker) |
$1.18 / — |
| DeepSeek V4.1 Flash 1017 | Open Weight | 33 (max) |
61.26% (max) |
— | $0.60 / — |
| Claude Opus 5.5 | Proprietary | 59 (max) |
70.14% (max) |
$4.00 / $20.00 |
— |
| GPT-6.2 Astra | Proprietary | 62 (max) |
67.66% (max) |
$8.00 / $40.00 |
$4.00 / — |
Sources: Artificial Analysis, Kingy AI, BenchLM, Plotly, eesel AI.
Three independent telemetry findings stand out:
- Hosted Accuracy & Speed: On Artificial Analysis, Thinking Machines' hosted endpoint delivers 280 output tokens/sec with a 9.72-second time to first answer token. In Kingy AI's 12-fixture hosted test (JSON, SQLite, and screenshot reading on Runpod with vLLM), Inkling 2 Mini passed 36/36 attempts—matching
Qwen4-Max(36/36) and outperformingQwen3.8-27B(30/36strict,32/36withLIKEallowed). - Data Analytics & Runaway-Loop Caveat: On Plotly's 43-question Data Analytics Benchmark, Inkling 2 Mini scored 36/43 (84%), outperforming
Muse Glimmer 30B(35) andGemma 4 31B(26) among U.S. open-weight models small enough for a 64 GB Mac mini, while trailingQwen 3.8 27B(38) andInkling 2 Small(39). However, it entered a 585-query loop over 48 minutes on one hidden-pattern question—consuming179Mtokens (8xInkling 2 Small) and driving total cost to $4.09 vs. Small's $1.81. - 24 GB Mac Reality Check: Thinking Machines recommends 32 GB+ unified memory on Apple Silicon or 32 GB+ VRAM on GPUs. When Kingy AI tested a quantized
9.14 GBUnslothUD-Q2_K_XLGGUF on a 24 GB M4 Pro MacBook Pro (llama.cpp b12588), short 30-token prompts ran at a median 11.6 tok/s and a16,103-token needle retrieval succeeded in 292.1 seconds, but the13.63 GBQ3 build failed memory-pressure checks and a 240-second Python coding run timed out with4.00 GiBof new swap-outs.
Production Deployment & Trade-Offs
When to Use Inkling 2 Mini vs. Alternatives
- Use Inkling 2 Mini locally when you need an
Apache 2.0multimodal model that fits on a32 GB–64 GBApple Silicon Mac or single32 GBGPU (~16–20 GBweights) for offline code assistance, structured extraction, or screenshot parsing. - Choose
DeepSeek V4.1 Flash 1017orInkling 2 Smallfor hosted APIs: On Artificial Analysis,DeepSeek V4.1 Flash 1017scores33(13 points higher than Mini) for $0.60/task—virtually identical to Mini's$0.59/task(Kingy AI). Note also that Thinking Machines' serverless inference beta coversInkling 2 Small, notInkling 2 Mini, leaving Tinker sampling ($0.15/$0.50per 1M;$0.75–$1.50/1M for training) and Prime Intellect via OpenRouter ($0.10/$0.40) as the primary hosted routes (OrcaRouter, Plotly). - Use frontier models (
Claude Opus 5.5,GPT-6.2 Astra,Gemini 4 Neon) for factual precision or complex multi-step engineering where Mini's53.9–59.1estimated coding/agentic scores and factual hallucination rate are unacceptable (BenchLM, eesel AI).
Operational Gotchas
- Enforce Turn & Token Caps: Because Inkling 2 Mini averages
~1.9xthe output tokens of peers and can loop during debugging or SQL exploration, configure hardmax_tokensand agent turn-count budgets in production harnesses. - Sandbox File & Shell Tools: The official model card explicitly warns that Mini can hallucinate, report outcomes inaccurately, and delete or overwrite files without confirmation—always run it inside disposable checkouts or isolated containers.
- Verify Runtime Kernel Support: Looped-transformer support in
SGLang,vLLM, andllama.cppshipped alongside the November 20 release; pin tested container tags and benchmark throughput on your own hardware before routing live traffic.
Frequently Asked Questions
What are Inkling 2 Mini's parameter count and license?
Inkling 2 Mini is a looped MoE model with 26.4B total parameters, 3.4B active parameters per token, and 5.4B effective weight activations per token across three core passes, released on November 20, 2026 under the Apache 2.0 license (Model Card, Kingy AI).
How much does Inkling 2 Mini cost via API?
On Thinking Machines' Tinker platform, sampling costs $0.15 per 1M input tokens ($0.015 cached) and $0.50 per 1M output tokens, with fine-tuning at $0.75/1M (64K) and $1.50/1M (256K) (Kingy AI). On OpenRouter, Prime Intellect serves it at $0.10/1M input and $0.40/1M output (Plotly).
Can Inkling 2 Mini run on a 24 GB Mac?
Only with heavy quantization (such as a 9.14 GB Q2 GGUF) and tight context limits; Thinking Machines officially recommends at least 32 GB of unified memory on Apple Silicon or 32 GB VRAM on GPUs, and a 64 GB Mac mini comfortably fits 4-bit weights (~16 GB) (Kingy AI, Plotly).
Does Inkling 2 Mini actually support a 1M-token context window?
The model card and Artificial Analysis document up to 1M tokens for inference, whereas Tinker's hosted model list exposes 64K and 256K context tiers—and long-context retrieval scores are mixed (46.4% on MRCR v2) (OrcaRouter).
Why did Inkling 2 Mini cost more than Inkling 2 Small on Plotly's benchmark?
Despite lower per-token rates, a single runaway loop of 585 queries on one hidden-pattern question consumed 179M tokens (8x Inkling 2 Small), pushing Mini's total run cost to $4.09 versus Small's $1.81 (Plotly).
Sources
- Thinking Machines Lab — Inkling 2 Mini Model Card (Nov 20, 2026): https://thinkingmachines.ai/model-card/inkling-2-mini/
- Hugging Face — Inkling 2 Mini, Cutting-edge Performance for your Local Setup (Nov 21, 2026): https://huggingface.co/blog/inkling2mini
- vLLM Recipes — thinkingmachines/Inkling-2-Mini Architecture & Cache Guide (Nov 21, 2026): https://recipes.vllm.ai/thinkingmachines/Inkling-2-Mini
- Artificial Analysis — Inkling 2 Mini Release & Intelligence, Performance & Price Analysis (Nov 21, 2026): https://artificialanalysis.ai/articles/inkling-2-mini
- Kingy AI — Inkling 2 Mini vs. open-weight and frontier AI models (Nov 20, 2026): https://kingy.ai/blog/inkling-2-mini/
- Plotly — Inkling 2 Mini Benchmark Results for Data Analysis (Nov 21, 2026): https://plotly.com/blog/inkling-2-mini-plotly-data-analytics-bench/
- OrcaRouter — Inkling 2 Mini: $0.59 a Task at Qwen3.8-27B's Score (Nov 21, 2026): https://www.orcarouter.ai/blog/inkling-2-mini-cost-per-task
- BenchLM — Claude Opus 5.5 vs Inkling 2 Mini: Benchmarks & Cost (Nov 21, 2026): https://benchlm.ai/compare/claude-opus-5-5-vs-inkling-2-mini
- eesel AI — Inkling 2 Mini: specs, price, and speed vs Inkling 2 Small (Nov 21, 2026): https://www.eesel.ai/blog/inkling-2-mini