Sovereign AI: Taking Ownership of your AI Stack

Executive summary

Sovereign AI is becoming a strategic priority as companies rethink how much control they have over the models, infrastructure, software harnesses, and data that power their AI systems. Rather than relying fully on third-party platforms, organizations can now combine open-weight models, private or sovereign compute, and self-hosted orchestration tools to create AI systems that are more portable, secure, compliant, and resilient. The key message: AI sovereignty is not all-or-nothing, but a series of architectural choices that determine whether AI becomes a vendor dependency or a private, permanent business asset.

Sovereign AI?

Until very recently, AI systems (particularly those employing LLMs) followed a predictable architecture that relied heavily on 3rd-party resources. Recently, however, the conversation around “Sovereign AI” has been growing rapidly, with many starting to recognize the risks of 3rd-party control over their strategically important systems.

Simply, AI sovereignty is the degree to which an individual or organization owns and controls the various parts of their “AI Stack”, which can largely be broken down into Models, Compute, Harness, and Data. In this blog, we will aim to shed light on what is possible today across the first three of these pillars at varying degrees of sovereignty.

1. The Brains: Frontier Open Weights

Being the headline component of any LLM based system, the model in use is an obvious place to start when considering the sovereignty of your AI system. Luckily, the open-weights ecosystem of models has been maturing steadily for a long time and there are lots of high performance players to consider.

Mistral

• The Flagships: Mistral Small 4 (119B MoE) and the voice-native Voxtral TTS.
• The Architecture Secret: Merges reasoning, multimodal vision, and agentic coding into a single, efficient 128-expert MoE architecture on an Apache 2.0 license, activating roughly 6B parameters per token.
• Where They Excel: High-speed function calling, real-time local audio/voice interfaces, and developer coding pipelines.
• The Use Case: Built for multimodal actions. supports agents that can visually parse a layout, write frontend code, and verbally explain deployment steps using high-fidelity text-to-speech completely offline.

DeepSeek

• The Flagships: DeepSeek-V4 (and the reasoning-heavy DeepSeek-R1).
• The Architecture Secret: A 1.6-trillion parameter Mixture-of-Experts (MoE) array with three native reasoning modes (Non-Thinking, Think High, Think Max)
• Where They Excel: High-throughput data pipelines, complex programming, and long-context information retrieval.
• The Use Case: software engineering and codebase refactoring. The Think Max mode uses internal self-verification loops for advanced coding logic on a local workstation.

Llama (Meta)

• The Flagships: Llama 4 Maverick and the long-context Llama 4 Scout.
• The Architecture Secret: Mixture-of-Experts Architecture optimised for large context windows
• Where They Excel: Autonomous agent workflows, native processing of large codebases, and sweeping legal/research corpora without truncation.
• The Use Case: Suitable for long-horizon agent loops, enabling agents to scan multi-thousand-file software architectures, map dependencies, and trace bugs in a single turn without complex vector databases.

Gemma (Google)

• The Flagships: Gemma 4 (31B Dense / 26B MoE) and the laptop-optimized Gemma 4 (12B Unified).
• The Architecture Secret: native step-by-step thinking mode, a 256K context window, encoder-free multimodal architecture that processes audio, video, and image signals within the core LLM backbone.
• Where They Excel: Long-context agentic workflows, on-device audio transcription/voice-editing, complex multimodal data processing, and edge deployments.
• The Use Case: Gemma 4 eliminates the VRAM overhead of separate vision/audio encoders. The 12B Unified model allows running multimodal, voice-driven local agents that execute scripts and review visual data directly on a standard 16GB consumer laptop.

Qwen (Alibaba)

• The Flagships: Qwen 3.6 Plus (for open-weights deployment)
• The Architecture Secret: Optimized for long-horizon execution in the "Agent Era." It generalizes across foreign runtime harnesses (OpenClaw, Claude Code) without behavioral degradation.
• Where They Excel: Multi-lingual enterprise agent swarms, long-running office productivity automation, and tool-use precision.
• The Use Case: Suitable for systems that coordinate dozens of cross-functional sub-agents executing hundreds of sequential local tool-calls over multiple hours, offering agent-first execution capabilities for a local workspace.

Phi (Microsoft)

• The Flagship: Phi-4-mini (3.8B).
• The Architecture Secret: Relies on highly filtered synthetic data and textbook-grade logic strings, offering a 128K context window and strong mathematical capabilities in a compact footprint.
• Where They Excel: Extreme-edge computing, mobile app integration, completely offline mobile tasks, and low-latency loops.
• The Use Case: Optimized for mobile applications requiring text formatting, complex script handling, or mathematical logic to run 100% offline on a user's phone without relying on an external network or excessive battery drain. Phi-4-mini is an architecture used for this.
  

When considering which of these models to use, the best way to get started is to try several options and see which best fits your solution. The best place to see all the available models is huggingface.co, which is something like a ‘GitHub’ for AI models. Additionally, the “Model Gardens” from each of the key hyperscalers provide something of a marketplace for models (Azure Foundry Models, GCP Agent Platform, and AWS Bedrock). Here you will find both proprietary and open-weight models, though they typically offer only the flagship options from key model providers.

One of the key things you will notice when browsing the options is that each is usually available in a range of parameter sizes. For example, gemma4 is available in 2B, 4B, 12B, and 31B (the B standing for Billions of parameters).

Parameters represent the total number of internal connections in a model, directly dictating its depth of knowledge and reasoning capabilities. Models with higher parameter counts tend to have better logic and problem-solving abilities but require more GPU memory to run. (We include a table summarising Model Size to VRAM Req. relationship in the hardware section below)

The size of the model (number of parameters) that you will choose will largely come down to a trade-off between the desired capabilities of the model in your system and the available hardware on which to run the model, which presents a nice segue to the next pillar of sovereignty in your AI stack. 

2. Hardware

Once you have selected the best model for your needs, you need some hardware on which to run it. The least “sovereign” approach would be to opt for one of the Model-Hosting-as-a-Service products available from the Big 3 hyperscaler vendors. The below table offers a comparison of the key offerings from the Big 3.

These services offer all the benefits that have made the cloud at large such an attractive offer over the last couple of decades: high availability, redundancy, reliability, security, and scalability - all whilst requiring a much lower maintenance overhead compared to managing one’s own on-prem compute resources integrated in your existing cloud stack/bill.

However, the ownership of the platform is very much out of your hands, keeping you beholden to the vendors and their whims. Pricing and feature sets could change at any time. Not very Sovereign.

Luckily, running open-weight large language models locally is more accessible than one might think.

On the cheapest end of the scale, older graphics cards can be bought quite cheap and provide reasonable performance with the smaller quantizations. The GTX 1060 with 6GB of VRAM is available second hand for as little as €70.

For example, Gemma4 E4B running on a GTX 1060 6GB achieved an output speed of ~13.5 token/s and an input speed of ~510 tokens/s. This is not really suitable for the most advanced precision-sensitive cases or for real time chat applications, but its multi-modal and reasoning capabilities could make it a strong option for agentic applications that run in the background (i.e: Second Brain/ personal wiki assistant).

Ultimately, this setup represents the floor of what is currently possible with cheap, self-owned hardware, and the sky is the limit with prices for NVidia H100 cards reaching in excess of €25k.

Representing a middle ground between the pricey-poles, Apple systems have recently been a popular choice for Local LLM enthusiasts. Their System-On-A-Chip designs and specifically Unified memory for CPU and GPU tasks (with the option to utilise the SSD for even more memory if needed) make them an ideal choice for those looking for Local LLM work-horses.

Windows die-hards will also be encouraged to hear that NVidia are working on their own platform for Laptops and Mini-PCs, the RTX Spark. Announced in June of 2026, the new platform promises to bring agentic workloads to Windows personal computers and could make Windows machines competitive to Mac for AI tasks.

While the RTX Spark targets consumer Windows devices, the DGX Spark is a Linux-powered desktop mini-supercomputer. It utilizes the same GB10 Grace Blackwell chip and 128GB of unified memory, but trades mobility for maximum thermal headroom and unthrottled performance. This creates a completely air-gapped, plug-and-play sandbox for developers to safely run and fine-tune heavy workloads or models up to 70B parameters entirely offline.

Whatever your budget, the key calculation to make is how much VRAM you will need to support the types of models you are looking to run. The table below breaks down the requirements for a range of Model Sizes. Q4 and Q8 are levels of “Quantization”, a reduction of the model’s precision which improves memory performance at the cost of a little bit of ‘intelligence’.