The German NIS2 implementation act has been in force since 6 December 2025 and is intended to strengthen cybersecurity. It raises the level of cybersecurity in sectors whose failure would endanger supply, and for the first time it places direct obligations on management. For distribution grid operators, municipal utilities and energy suppliers this means one thing in concrete terms: under § 38 BSIG, management is personally liable for failures in cybersecurity, and breaches can be sanctioned with fines of up to ten million euros or two per cent of worldwide annual turnover.
The energy industry is one of the sectors in which “state of the art” security measures are mandatory. In that situation, it is worth looking at a practice that has long been taking place in almost every company but rarely appears in a risk assessment: the informal use of publicly available AI services.
Let us look at a few everyday examples. An employee copies an excerpt from a grid calculation report into ChatGPT to have a summary written. A caseworker has a chatbot draft a customer email and pastes personal data into it. Both happen very often as a quick shortcut, without anyone keeping a record, and frequently without any bad intent. What still passed as pragmatic self-help last year is, under NIS2, an uncontrolled outflow of data to an unassessed service provider — and therefore a liability risk for which management must answer personally in the case of culpable breaches.
The obvious response, simply banning the use of AI, does not work in practice. Where a tool provides real benefit, it will be used, if necessary via a private smartphone and therefore outside any control. The sensible question is therefore not whether AI should be used in the company, but which architecture that use runs through. This is exactly where LLM clusters and LLM gateways come in. What is the difference?
What is possible today: the local LLM cluster
An immediately available solution is a locally operated language model. Concretely, that is a machine standing in your own data centre or office, hosting a model instance that is addressed directly. All requests and all data end up there and nowhere else. For a grid operator whose most sensitive requests contain operational data from grid control or personal customer data, this solves the core of the problem: that data does not leave the building. Local LLMs are usually less capable, however, because they are typically trained on far less data and have fewer resources available overall than the commercial providers from the United States.
A large language model (LLM) is an AI system that has learned from large volumes of text how to process and produce language. It answers questions, summarises or drafts texts by statistically completing the most probable next word.
Locally operated models are good enough for many everyday tasks, but for complex requests the large models hosted outside Europe or outside the company are clearly superior. Anyone who wants maximum data sovereignty pays for it by forgoing the peak performance of the strongest models. For a considerable share of internal AI use that trade-off is acceptable, and the gain in control is immediate.
Governance gateways: available, but not the end of the story
Between purely local operation and the unregulated use of external services, a product category of its own has established itself over the past two years: so-called LLM or AI gateways. These tools are commercially available, in part open source, and they deliver what counts from a compliance perspective. They detect and filter sensitive content such as personal data at the gateway level, before a request reaches an external provider; they log every request in a traceable way; and they can be operated inside your own network boundary, so that regulated data traffic does not pass through a third-party cloud service.
That addresses the problem of “shadow AI” — uncontrolled AI use bypassing IT and management — head-on. They should not be overestimated, though. By design, these gateways are built to forward requests to external providers in a controlled way, filtering and logging them along the way, not to establish maximum sovereignty. And they are industry-neutral. They do not come with a representation of the specific regulatory logic of the energy sector, such as the requirements of the IT security catalogue. They are a solid foundation, not a turnkey answer for an operator of critical infrastructure.
The next step: the intelligent, sovereign gateway
The genuinely interesting approach goes one step further and combines both. Technically, this gateway would itself be a local language model that checks every request in real time and decides how it is handled. If a request contains sensitive information that must not leave the building, it is answered locally. If the request is uncritical but complex, the gateway forwards it to a stronger external model. The filter therefore decides on two criteria at once, data content and complexity, and gets the best of both worlds: data sovereignty where it is needed, and peak performance where it is unproblematic.
At this point, honesty calls for caution. The individual building blocks exist, and research into such sensitivity- and complexity-based routing between local and external models is currently very active. A finished, hardened product that brings exactly this logic to the regulatory requirements of a German energy supplier is not yet available off the shelf. Anyone who wants to build it is still entering new territory today.
What an LLM gateway does technically
An LLM gateway is not an additional security layer laid on top of existing processes. It is the point at which AI requests from across the company are consolidated, checked and logged before they leave a system. The difference between a corporate account with an AI provider and a real gateway lies in four functions.
Data classification before the prompt
Before a request is forwarded at all, the gateway checks what it contains. Operational data from grid control, personal customer data and technically uncritical requests are distinguished. This classification governs everything that follows and prevents sensitive content from reaching an external service unnoticed. It is the technical equivalent of the risk assessment that NIS2 requires anyway, applied to every single transaction instead of once a year on paper.
Model routing based on classification
On the basis of the classification, the gateway decides which model answers a request: the local model in your own data centre for anything that must not leave the building, an external model for uncritical but demanding tasks. The decision therefore sits in the architecture rather than in the discretion of individual employees. That is the real difference from a usage policy which exists on paper but binds no one in day-to-day work.
Access control and logging
Every request is attributed to a person, a role and a point in time, and is logged traceably. Role-based permissions additionally limit who may use which models and which data classes at all. For an audit or a notification under NIS2, this is the decisive point: only the log makes it possible to state which data went to which model and when. Without a gateway, a gap remains here that cannot be closed after the fact.
Supply chain security
An external language model is a service provider like any other and has to be assessed accordingly: place of processing, contractual basis, treatment of inputs as training data, subcontractors used. NIS2 explicitly requires supply chain security, yet for AI services it is rarely documented so far. The gateway is the place where such an assessment takes technical effect, because only approved providers are reachable at all.
On top of that come practical hurdles that should not be played down. Routing all AI traffic through a local gateway means additional latency, it requires well-considered network administration, and one loophole remains: what if an employee bypasses the internal solution and uses a private phone after all? This shows that even the most sophisticated gateway does not solve the problem by technical means alone. Without accompanying organisational rules, clear usage policies and an offering good enough that no one feels the need to circumvent it, every technical architecture remains incomplete.
LLM gateway: not a parallel world to existing obligations
For grid operators this is not a new, additional compliance field but the continuation of a requirement that has long existed. Anyone operating an electricity or gas grid must already maintain a certified information security management system in line with ISO/IEC 27001 and 27019. The legal basis for this has changed with the NIS2 implementation act: § 5c EnWG replaces the previous § 11 (1a) and (1b) EnWG. According to the authority, the existing IT security catalogues of the Federal Network Agency from 2015 and 2018 continue to apply until the revised catalogue under the new legal framework is published.
An LLM gateway is therefore not the invention of a new problem in order to sell a solution for it. It is the consistent application of principles already anchored in an information security management system (ISMS) to a channel that has so far flown under the radar. Data classification, access control, auditability and supplier assessment are not AI-specific inventions but core components of any sound security concept. All that is new is that they now have to apply to dealing with language models as well.
In closing: LLM use will grow, gateways and clusters help
The use of AI in day-to-day work will increase. The only question is whether it takes place within a controlled architecture or in a grey area that cannot be explained in the event of an audit. Those who settle the architecture now gain room for manoeuvre instead of having to retrofit later under time pressure and possibly after an incident. The effort of building such a structure is manageable. The effort of explaining its absence after the fact is not.
About control-f. control-f GmbH is a values-driven AI company based in Konstanz. Since 2022, the data boutique has been building big data platforms for industrial telemetry data and helping companies in German-speaking Europe make complex data landscapes usable. Its clients include large corporations and mid-sized companies from plant engineering, automotive and the energy industry. Managing directors Simon Deussen (machine learning engineer and founder) and Daniel Tremer (formerly Specialist Data Science & AI Projects at Porsche AG) focus on building stable data architectures as the foundation for analytics, software solutions and AI applications such as predictive maintenance.
This article provides an overview and does not replace legal advice in individual cases.
Kategorie / Category: Blogposts