Nov 8, 2026 • 10 min read
Resolve vLLM block manager allocation failures, KV cache memory fragmentation, and preemption thrashing under high concurrency with verified production configs.
#vllm#pagedattention#kv-cache#gpu#inference#performance
Nov 8, 2026 • 10 min read
Deploy 50+ fine-tuned LoRA adapters on a single vLLM instance. Learn dynamic GPU adapter swapping, Punica batched GEMM kernels, and latency benchmarks.
#vllm#lora#llm-serving#multi-tenant#gpu#inference
Sep 13, 2026 • 8 min read
Fix vLLM CUDA OOM errors without sacrificing throughput. Covers max-model-len, gpu_memory_utilization, KV cache tuning, chunked prefill, and multi-GPU tensor parallelism with real configs.
#vllm#cuda#gpu#inference#production#troubleshooting#llm-serving
Sep 13, 2026 • 6 min read
Run the full DeepSeek R1 671B parameter model on consumer hardware using KTransformers CPU offloading. Real VRAM requirements, quantization choices, throughput benchmarks, and production limitations.
#deepseek#ktransformers#ollama#cpu-offloading#llm-serving#quantization#benchmark