Self-host LLM cost: 300 tok/s of DeepSeek-V4.1-Flash

This note is not the architecture. The architecture — one entry point, quotas, the switch between a vendor and your GPUs — is in Models that stay inside. This is only arithmetic: one concrete model, one concrete SLO, September 2026 list prices, and the invoice that falls out.

The architecture: models that stay inside. If legal has not yet said the text cannot go to a SaaS, read that one first and do not buy silicon.

The usual search is “self host LLM cost”, “vLLM vs OpenAI cost”, “cuánto cuesta un H100”. It almost always assumes Llama-3-70B on two GPUs, like 2024. On 10 September 2026 DeepSeek shipped V4.1-Flash. That working account is dead. What follows is the one that replaced it.

300 tok/s is an SLO, not a full cluster

Three hundred output tokens per second is what a busy internal product wants to feel: an agent that answers without anyone counting the ellipsis. It is not “fill the GPUs”. A node that fits this model, batched, sits in the thousands of tok/s. At 300, VRAM is full and the SMs are bored.

DeepSeek’s public API sits around 190–214 tok/s per stream. Baseten publishes about 289. Three hundred is slightly above what DeepSeek will sell you as a service. Self-hosted, 300 tok/s per user wants Blackwell, DSpark and low concurrency — not more H100s. As aggregate node capacity, 300 is trivial. You still pay for the whole node.

The working account uses 300 tok/s of output, around the clock, 730 hours a month. That is 788 million output tokens. An agentic 4:1 input:output mix — the prompt is the transcript, the tools, the RAG — adds 3.15 billion input tokens. A 1:3 chat mix would make the API even cheaper against the node. We use the mix that brings the two invoices closest. They still do not touch.

The model is cheap to run and expensive to house

The card is deepseek-ai/DeepSeek-V4.1-Flash . MIT licence. API: deepseek-flash. 552B MoE backbone, 8B active on prefill, 16B on decode, plus 196.6B of Engram tables (~183 GiB in FP8) and a ~14B DSpark drafter. Checkpoint: 510 GB across 48 shards, experts in MXFP4, the rest in MXFP8. Context: 1,048,576 tokens. KV: 890 B per token — a million-token prompt is under 1 GB of global KV.

The “cheap to run” story is true: 8B and 16B active. The VRAM story is not. Every expert and both Engram tables have to reside. There is no GGUF, no Ollama, no desktop shortcut. The checkpoint is already quantized. What you buy is not compute. It is room.

vLLM’s recipe puts the floor at 614 GB with 1.2× headroom. That is why one 80 GB H100, or two, or even eight, is not the SKU. Eight H100s are 640 GB: 26 GB over the floor, and in no production recipe. Four H200s are 564 GB: under it. The Hopper path that exists is 8× H200 (1,128 GB). The path vLLM and Dynamo have verified is a 4-GPU Blackwell tray.

The weights set capacity, not the KV. A 1M context fits; a million concurrent tokens do not. --language-model-only is in every day-one recipe: vision is in the checkpoint, not in the serve that is tested.

The node, always on

List prices, CoreWeave, on-demand, September 2026. 730 hours. Vendor dollars, euros at $1 = €0.92 — the same FX as the 29% of your AWS bill. Spanish VAT (21%) sits on top and does not move the ratios. Spot exists and is not how you serve a latency SLO: they preempt the agent mid-sentence.

The H100 row is there so you see the trick: it costs almost the same as the H200 and does not work. “How much does an H100 cost” is the wrong question. The question is which node fits the checkpoint. On Hopper, 8× H200. On Blackwell, a four-GPU tray. There is no two-GPU SKU on this card.

A business-hours calendar does not turn the node off. 8 h × 22 days at 300 tok/s is 190 million output tokens — a quarter of the 24/7 account — and the API lands around $230. The node is still €33,900, because the product will not take a twenty-minute cold start at 08:59. Autoscaling a 510 GB serve to zero is not a saving. It is an incident.

The same 788 million outputs, on APIs

DeepSeek, official pricing. Off-peak: $0.15 / 1M input on a miss, $0.003 on a hit, $0.60 output. Peak: double. Peak is Monday to Friday, 01:00–04:00 and 06:00–10:00 UTC — seven hours per weekday, ~21% of the week, and it lands on Beijing hours, not on a European support queue. The whole weekend is off-peak.

GPT-5.6 Terra and Sol, OpenAI pricing . Sonnet 5 and Opus 5, Anthropic pricing. OpenAI EU data-residency endpoints: +10% on models from 5 March 2026. 4:1 mix, all misses, 730 h.

The line people put on the slide — “GPT-4o / Claude versus our H100s” — is Opus 5 versus an H200 node: €32,600 against €33,900. A theatrical tie. Against DeepSeek Flash, the same node is forty times the API. Against Sonnet 5, two and a half times. The tie-break is not the logo. It is whether the tokens may leave.

Cache: if half of DeepSeek input is a hit, off-peak drops to about $714 (~€657). The node does not notice. The system prefix that repeats on every agent call is exactly the discount DeepSeek already sells you at $0.003 / 1M. Self-hosted, that cache is KV on your GPUs, and at 300 tok/s of one stream it is not even the limit.

Where the node wins

Off-peak, 4:1, misses: each million output tokens on DeepSeek costs 4 × $0.15 + $0.60 = $1.20. The $36,821/month node breaks even around 11,700 tok/s sustained, all month. At 300 tok/s you are at 2.6% of that line. The GPUs are paid. They are not busy.

The node actually wins in three places, and only there:

• The tokens cannot leave. Residency, secrecy, a contract that does not accept “the vendor does not train on your data”. Then €33,900 a month is the price of compliance, not the price of tokens.
• You sustain ~12,000 tok/s for real, measured, not on a one-user million-token bench. Then silicon beats DeepSeek on money. Measure before you sign.
• The per-user SLO sits above what the public API delivers (~214 tok/s) and you can prove it on Blackwell, DSpark and a small batch. That is product, not FinOps.

Buying the node “in case traffic rises” is how waste becomes a twelve-month contract — the same trap as Reserved Instances on idle nodes. If traffic rises, DeepSeek charges you more. If it does not, you have not bought €33,900 of headroom.

Serving it (text, 8× H200)

vLLM 0.30.0 or the vllm/vllm-openai:deepseekv41-flash-0909 tagged image. There is no pip wheel for this checkpoint on day one. First load: VLLM_ENGINE_READY_TIMEOUT_S=3600 . Copy the exact speculative-config JSON from the vLLM recipe for your image. Thinking defaults on at effort 50; a small max_tokens looks like an empty reply.

docker run --gpus all --privileged --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e VLLM_ENGINE_READY_TIMEOUT_S=3600 \
  vllm/vllm-openai:deepseekv41-flash-0909 \
  deepseek-ai/DeepSeek-V4.1-Flash \
  --tensor-parallel-size 8 \
  --language-model-only \
  --tokenizer-mode deepseek_v41 \
  --tool-call-parser deepseek_v41 --enable-auto-tool-choice \
  --reasoning-parser deepseek_v41 \
  --gpu-memory-utilization 0.9 \
  --max-model-len 262144

On GB200, TP4 on a four-GPU tray, not TP8. Cache the weights on a volume or in an OCI registry; downloading 510 GB on every start turns a restart into a twenty-minute outage — the same contract we already use for any large artifact. Sampling: temperature 1.0, top_p 0.95, a generous max_tokens (256K) if you let the model think.

The rest — keys, quotas, the switch so product never talks to port 8000 — is not invented here. It lives in the gateway. That note is how you operate it. This is what the silicon costs once you have decided the text does not leave.

What the €33,900 figure does not include

• The person. Operating vLLM is not one more Deployment. A quarter-FTE of a senior platform engineer in Spain already exceeds the DeepSeek bill at 300 tok/s. The arithmetic for that hire is in hire a platform engineer, or outsource it.
• Network and disk. 510 GB of weights, a registry that serves them in the same AZ, and the fact that an imagePullPolicy: Always on a Friday is a twenty-minute incident.
• The real idle. At 300 tok/s the node is at 2.6% of its break-even. Flexera’s cloud waste rose this year because of AI; do not donate that rise to a cut, and do not call a node nobody saturates “the product” either.
• Buying the H200s. Colo and capex change the shape of the invoice, not the VRAM floor. You still need eight Hopper or four Blackwell, plus power, plus somebody who turns them off when the product is not using them — and this product does not turn them off.

Where this breaks in practice

• Planning two H100s because “70B fitted”. V4.1-Flash is not a 70B. It is a 552B MoE with tables you cannot leave at home.
• Treating 300 tok/s as cluster throughput and concluding “we have spare silicon, cut to two GPUs”. The cut leaves you without the checkpoint. The slack you can see is latency, not VRAM.
• Signing a one-year commitment on the node the same week legal has not yet said the text cannot leave. The commitment turns a hypothesis into a mortgage.
• Putting Ollama on the slide. There is no build. The first 510 GB pull from Hugging Face on a node with no cache is the first incident.
• Measuring the SLO with max_tokens=64 and thinking on, and declaring the model “does not answer”. It answers. It is thinking. Raise max_tokens, or turn thinking off for the low-latency path.

Do not hire us if

The list is explicit. If you are on it, the email is surplus.

• You want two H100s in the rack and Llama-3-70B. That working account is 2024. This note does not revive it.
• You want the node to come out cheaper than DeepSeek at 300 tok/s. It does not. We will not torture the ratios until it does.
• Legal has not said the text cannot leave, and the argument is “just in case”. Just in case is calling DeepSeek, with a cap on the gateway.
• The problem is that three teams have three keys and nobody can attribute the bill. That is not silicon. It is the gateway.

If you already have the node, the checkpoint cached, a measured tok/s SLO and an owner who can turn it off, do not call us to confirm the FX. If the slide says “a pair of H100s” and the card is V4.1-Flash, that is a conversation.

Send us the model, the tok/s SLO, and whether the text may leave. Within 24h you get back whether the node wins, whether the API wins, or whether the problem is the gateway and not the GPUs. Sometimes the answer is: call DeepSeek, and do not call us. hola@pikas.systems