LLM Engineering
How LLM applications get built and shipped: inference and serving, structured output, token formats, evaluation, and the libraries powering production language-model systems.
157 articles
LLM Engineering
HyperQwen: 1,000 tok/s on a Single RTX 3090 Through Aggressive Recurrent-State Speculation
LLM Engineering
oMLX: Why This Mac Inference Server Caches to Your SSD (And Why That's Brilliant)
LLM Engineering
Qwen3.8-27B on a Single RTX 3090: How Calibrated Quantization and Speculative Decoding Hit 1,000 tok/s
LLM Engineering
METATRON: Building a Penetration Testing Assistant with Local LLMs and Zero Cloud Dependencies
LLM Engineering
Switchyard: Composable LLM Routing Without the Gateway Lock-In
LLM Engineering
Running a 2.78-Trillion-Parameter Model on a Laptop: Inside WASTE's NVMe-Streaming Architecture
LLM Engineering
Running 35B MoE Models on 8GB GPUs: How vvllm-ms Caches Experts in System RAM
LLM Engineering
Terminal-Bench: Why Evaluating LLM Agents on Real Terminal Tasks Is Harder Than You Think
LLM Engineering
OSWorld-V2: The GUI Agent Benchmark That Hides Its Answers
LLM Engineering
Running a 2.78-Trillion-Parameter Model on a Laptop: Inside WASTE's NVMe-Streaming Engine
LLM Engineering
Running 753B MoE Models on Consumer GPUs: Hand-Written SASS Kernels for 2-Bit Experts
LLM Engineering
LLM Checker: Hardware-Aware Model Selection for Local Inference
LLM Engineering
Fighting LLM Hallucinations in 2026: llm-council vs claude-octopus vs LLM-Check
LLM Engineering
Running Large LLMs on Limited Hardware in 2026: AirLLM vs llm-checker
LLM Engineering
The Fragile Economics of Free AI: Mapping the Great Inference Subsidy Wars
LLM Engineering
Shard: Proving LLM Inference Can Work Across Scattered GPUs and Terrible Internet
LLM Engineering
Nanocoder: The Terminal Coding Agent That Lets You Switch Models Mid-Conversation
LLM Engineering
ds4: The SSD-Streaming Inference Engine That Treats Your Mac's NVMe Like RAM
LLM Engineering
Harness-1: Training Search Agents with State Externalization
LLM Engineering
SichGate Methodology: When Healthcare CISOs Need to Red-Team 4-Bit Llama Without Hiring Offensive Security
LLM Engineering
ModelRegression: Building a Daily LLM Benchmark That Tests What Developers Actually Use
LLM Engineering
Inside AI Product Bench: Why Two LLMs Disagree on Half Their Product Recommendations
LLM Engineering
Neuromod-LLM: Treating Language Models Like Brains on Drugs
LLM Engineering