Self-hosting LLMs for GDPR: when it is required and what it costs

Self-hosting a language model is often justified with data protection, but for most B2B text workloads it is not required. The GDPR does not forbid third-party processors. It asks for a contract with each processor, a lawful basis for any transfer and, for risky processing, a documented risk assessment. An EU-hosted API with a data processing agreement can meet those requirements. Self-hosting is the right answer when personal data must stay inside a network you control, or when a contract or a regulator rules out any processor you cannot audit. Then the GPU bill is the easy part to budget – the operations are where the money goes.

This article checks the figures on 10 October 2026, using only pages I could open. It covers when self-hosting is required, how much GPU memory open-weight models need at each precision, what EU-based GPUs cost, what vLLM delivers, a break-even per million tokens against EU-hosted APIs, and the costs that appear after go-live.

When self-hosting is actually required

The question is not whether a model can run on your hardware, but whether personal data has to stay there. Three situations push me towards a self-hosted model:

• A contract forbids it. Some client contracts and internal policies rule out sub-processors you do not control, or require that data never leaves a defined network.
• Your impact assessment says no. The data protection impact assessment for the use case finds the risk of a processor chain too high, for example for health data or employee records.
• No suitable provider exists. No EU-hosted provider offers the model on the terms you need, or the system must run offline, on a plant floor or at the edge.

Everything else is a processor question, and an EU-hosted API answers it. Article 28 requires a contract that sets out the subject matter and duration of the processing, its nature and purpose, the types of personal data and data subjects, and the controller's obligations and rights. It also requires prior written authorisation for sub-processors, specific or general, with the same obligations passed down to them. Article 44 makes any transfer to a third country subject to the conditions of the regulation's transfer chapter. For US companies certified under the EU-US Data Privacy Framework, the Commission's adequacy decision of 10 July 2023 provides the basis for the transfer.

Self-hosting does not settle the model question either. The EDPB's Opinion 28/2024, adopted on 17 December 2024, says that AI models trained with personal data cannot in all cases be considered anonymous. Anonymity claims therefore have to be assessed case by case, so ask the provider for documentation on the training data before you rely on such a claim.

Model sizes and GPU memory with quantisation

Weight memory is simple arithmetic: parameters times bits per weight, divided by eight. A 70-billion-parameter model needs about 140 GB at 16 bits, 70 GB at 8 bits and 35 GB at 4 bits (my calculation). That excludes the KV cache, which grows with context length and concurrent requests, and the runtime's own overhead. By default vLLM may use 92 per cent of GPU memory (gpu_memory_utilization, 0.92 in the current docs), and the KV cache gets what the weights leave.

Two lessons from the table. On one 80 GB card, a 70B model at 8 bits leaves only a few gigabytes for the cache once vLLM has taken its share (my calculation), so plan for two cards or a 96 GB card. And the model you can run depends on the memory you can rent, so check memory before you read a benchmark.

GPU options and prices on 10 October 2026

These are the prices I could read on provider pages, with their dates. Where a provider gives a monthly figure, I quote it; where it does not, I calculate at 730 hours a month and say so.

Hetzner describes its GPU servers as GDPR compliant and located in Europe. Its matrix page lists RTX PRO 6000 Blackwell cards with 96 GB of GDDR7 in the GEX63 and GEX131. Its monthly prices appear only in the configurator, which I could not read, so the table leaves them out. Check them there before you compare.

Two things to watch. Scaleway raised its H100 price from €2.73 to €2.868 an hour on 1 June 2026, so older comparison pages are out of date. OVHcloud displays its prices in dollars, so convert them on the day you compare them with the euro rows.

Throughput with vLLM: the numbers and how to measure them

vLLM's own benchmarks report relative gains rather than absolute tokens per second. In its September 2024 post on v0.6.0, the team reported 2.7 times the throughput of v0.5.3 for Llama 3 8B on one H100, and 1.8 times for Llama 3 70B on four H100s, on the ShareGPT dataset with 500 prompts. For the server I would start with vLLM; my vLLM review covers the trade-offs.

Absolute numbers depend on the model, on prompt and output lengths, and on the speed each user should get. The SemiAnalysis InferenceX benchmark shows that trade-off for Llama 3.3 70B on H100. Per GPU it reaches about 1,850 tokens per second when each user gets 38 tokens per second, about 1,170 at 58 tokens per second per user, and about 880 at 78. These three values are interpolated from the page's data points (my reading), and the benchmark uses 8K/1K sequence lengths. The page does not state the precision or the serving engine consistently, so I treat the numbers as an order of magnitude.

# serve gpt-oss-120b on one 80 GB GPU (model card)
vllm serve openai/gpt-oss-120b

# a 70B model at bf16 needs two 80 GB GPUs (my calculation)
vllm serve meta-llama/Llama-3.3-70B-Instruct --tensor-parallel-size 2

# from a second terminal, send 500 random prompts at 4 requests per second
vllm bench serve --model openai/gpt-oss-120b --dataset-name random --num-prompts 500 --request-rate 4

Measure your own numbers with vllm bench serve. Its defaults send 1,000 random prompts with an infinite request rate, so set the prompt count and the rate yourself, and use prompts that look like yours where you can. Then sweep the rate and record time to first token and time per output token at each step. The --goodput option counts only the requests that meet your latency targets, which is the number that matters for a user-facing feature.

Break-even per million tokens

The formula fits on one line. Cost per million tokens equals the monthly GPU cost divided by the tokens produced that month, times one million. Tokens per month equal tokens per second, times 2,628,000 seconds (730 hours), times the share of time the GPU is busy. I treat each token served as one billable token at the API's output price, which keeps the comparison simple (my method, not a benchmark result).

Cost per million tokens for one rented H100 at €2,094 a month, using the InferenceX rates (my calculation):

Against the EU APIs, Llama 3.3 70B is the closer case. OVHcloud charges €0.67 per million tokens and Scaleway €0.90. At 58 tokens per second per user, one rented H100 would have to sustain about 1,190 tokens per second around the clock to match OVHcloud (my calculation). The benchmark gives 1,171 at that speed, so it cannot get there. Against Scaleway's €0.90, the break-even is about 885 tokens per second, roughly three-quarters of that benchmark rate.

gpt-oss-120b sets a higher bar. The API charges €0.60 per million output tokens at Scaleway and €0.40 at OVHcloud. One rented H100 at €2,094 a month therefore needs an average of about 1,330 output tokens per second to match Scaleway, and about 1,990 to match OVHcloud (my calculation). I found no verified H100 figure for gpt-oss-120b, so measure it before you decide. Input tokens are cheaper again, at €0.15 and €0.08 per million.

Operating costs come on top. Each €1,000 a month of operations cost raises the gpt-oss-120b break-even at €0.60 per million tokens by about 1.7 billion output tokens a month, or roughly 630 tokens per second around the clock (my calculation). So price alone rarely justifies self-hosting when an EU-hosted API already offers the model. The case appears when the hardware is already yours, when the workload keeps a GPU busy all month, or when no EU provider offers the model you need.

On-premise hardware: the card is cheap, the power and the licence are not

A card you buy looks cheap next to a rented one. NVIDIA priced the GeForce RTX 5090 at $1,999 at its launch, with 32 GB of GDDR7 and a total graphics power of 575 W. Spread over 36 months, the card costs about $56 a month (my calculation). Run flat out, 24 hours a day, the card alone draws about 420 kWh a month (575 W times 730 hours, my calculation), which at an example assumption of 30 ct/kWh is about €126 a month (my calculation, not a quoted tariff). Over three years the power alone comes to about €4,500, more than twice the $1,999 launch price.

The costs that show up after go-live

Once the service is live, the GPU rent is usually the smallest line. The rest is labour and risk. The planning figures in the table are my own assumptions, not measured data, so replace them with your team's numbers.

What I would do first

1. Write down the data and the rules. List which data classes reach the model and which contracts or DPIA conclusions apply. If none rules out a processor, start with an EU API that has a DPA.
2. Get the hosting facts in writing. Ask for the hosting region, the sub-processor list and the retention settings. Mistral's pricing page links a DPA but does not say where its API runs, so that answer has to come from the contract.
3. Pick the smallest model that passes your evals. Then check its memory at the precision you can actually serve, not at the precision on the model card.
4. Benchmark on your own prompts. Run vllm bench serve with your real prompt lengths and a latency target before you commit to any hardware.
5. Redo the break-even with your numbers. Use your measured tokens per second, your real utilisation and your operations budget. If it does not clear, stay on the API and keep self-hosting written down as a fallback.
6. Budget for operations before go-live. Set the eval suite, the on-call rota and the update policy first. The GPU is the easy part.

For the data side of the pipeline, read my notes on EU data residency, region controls and zero retention and on PII redaction. For the regression set, see LLM evals for product features.

Sources

1. openai/gpt-oss-120b model card (Hugging Face)
2. mistralai/Mistral-Small-3.2-24B-Instruct-2506 model card (Hugging Face)
3. meta-llama/Llama-3.3-70B-Instruct model card (Hugging Face)
4. mistralai/Mistral-Large-3-675B-Instruct-2512 model card (Hugging Face)
5. vLLM blog: v0.6.0 performance update (September 2024)
6. vLLM documentation: engine arguments
7. vLLM documentation: vllm bench serve
8. SemiAnalysis InferenceX: Llama 3.3 70B on H100 vs H200
9. Scaleway GPU instances pricing
10. Scaleway blog: a transparent update on Scaleway pricing (27 April 2026)
11. Scaleway Generative APIs pricing
12. OVHcloud public cloud price list
13. OVHcloud AI Endpoints
14. OVHcloud AI Endpoints model catalogue
15. Hetzner dedicated GPU servers
16. NVIDIA newsroom: GeForce RTX 50 Series launch
17. NVIDIA GeForce RTX 5090
18. NVIDIA GeForce software licence
19. GDPR Article 28: processor
20. GDPR Article 44: general principle for transfers
21. European Commission: EU-US data transfers (Data Privacy Framework)
22. EDPB Opinion 28/2024 on AI models (adopted 17 December 2024)
23. Mistral AI pricing