Last month I sat across from a managing partner of a mid-sized Dublin practice who had just finished pasting a client's witness statement into ChatGPT to get a "quick summary". He didn't see the problem. The statement contained the home address of a vulnerable witness in a family law matter. It now lives, in some form, on a US server he has no contract with, no DPA from, and no audit trail into. That single act — repeated daily across Irish law firms — is the reason I built what I built, and it's the reason a SaaS chatbot is the wrong shape of tool for the job a solicitor actually needs to do.
The data sovereignty problem isn't theoretical for Irish solicitors
Solicitors in Ireland operate under a layered set of obligations that don't fit neatly into the terms of service of a generative AI vendor. You have the Solicitors Acts and the Law Society's professional conduct guidance, the GDPR and the Data Protection Act 2018, the Legal Services Regulation Act, and — for anyone touching corporate or financial work — the AML directives transposed through the Criminal Justice (Money Laundering and Terrorist Financing) Act. Layered on top of all of this is the EU AI Act, which classifies a meaningful slice of legal work (administration of justice, evaluating evidence) as high-risk.
The practical effect is that when a junior associate uploads a draft contract to a public LLM, the firm has potentially: transferred personal data to a third country without an adequate basis, breached client confidentiality under section 26 of the Solicitors Act framework, lost legal professional privilege over the content (the moment a third party sees it), and triggered an AI Act obligation around logging and human oversight that the SaaS provider cannot help them meet.
SaaS vendors will tell you they have SCCs, ISO 27001, and a SOC 2 report. That's all true and all insufficient. The Schrems II decision didn't go away. The question is not "is this provider secure" — it's "do I, as the data controller, have the supplementary measures in place to demonstrate I retained control of the data?" When the model weights, the prompt logs, the embedding index and the inference compute all sit in another jurisdiction, the honest answer is no.
What "on-premise" actually means in 2024 — and what it doesn't
There's a lot of muddle around the term. On-premise does not mean "we have a server in the cupboard running a 7B model from Hugging Face". That's a science project, not a production legal tool. It also doesn't mean "private cloud" — running an OpenAI deployment in your Azure tenant is still a SaaS architecture, just with a different billing model and a slightly better DPA. The training data flows, the safety filters, and the model updates are still controlled by a third party.
What I mean by on-premise is a stack with these properties:
- The model weights live on hardware you control. Either physically in the firm's office, or in an Irish-jurisdiction colocation with a single-tenant lease. The weights are open-weight (Llama, Mistral, Qwen, or fine-tunes thereof) so there is no licence drift and no surprise deprecation.
- Inference is air-gapped from the public internet by default. Outbound traffic is denied at the firewall except for an allow-list. The model never reaches out to a vendor API for "context" or "tool use" unless you explicitly route it.
- The retrieval index — the part that actually matters — is built from your matter files, your precedents, your closed cases. It never leaves the box.
- Logging is local, immutable, and yours. Every prompt, every retrieval, every output, every user. You can satisfy an AI Act audit, a Law Society inspection, or a subject access request without phoning a vendor.
The hardware footprint for a small-to-mid Irish practice is genuinely modest. A single workstation-class GPU box (one or two consumer-grade cards in the 24–48GB VRAM range) will comfortably serve a 30-fee-earner firm doing document drafting, summarisation, due diligence review, and precedent search. You don't need a datacentre. You need a locked room and a UPS.
Retrieval is the actual product, not the model
This is the single most misunderstood thing in legal AI. Solicitors don't need a smarter model — GPT-4-class capability has been available in open weights for some time now and the gap to frontier is narrowing for the kind of bounded reasoning legal work demands. What they need is a model that knows their firm: the way the senior partner phrases an indemnity clause, the standard form for a Section 96 letter, the firm's house style on chronologies, the closed-matter precedent that solved a near-identical problem in 2019.
None of that lives in a public model. It lives in the firm's iManage or NetDocuments or — let's be honest — the shared drive that everyone pretends is properly organised. The job of an on-premise legal AI is to ingest that corpus, build a retrieval layer over it, and use the model as a reasoning engine on top of retrieved context rather than its training memory.
Technically this means a few things that vendors gloss over. You need a chunking strategy that respects legal document structure — you cannot blindly split a contract on 512 tokens and expect clause cross-references to survive. You need hybrid retrieval (dense embeddings plus BM25 plus metadata filters on matter, client, jurisdiction, date) because pure vector search loses to keyword search whenever a solicitor types a specific section number or party name. You need a permissions model that mirrors the firm's existing matter-level access controls, because the worst possible failure mode is the AI cheerfully retrieving privileged material across a Chinese wall.
This is what we've built into the Intelligence Brain for legal practices — and it's also why a generic chatbot, however clever, cannot do this job. The chatbot doesn't know what a Chinese wall is, doesn't know which matters you're conflicted out of, and has no concept of what "the firm's standard form" means.
Audit trails, hallucination control, and the duty of care
The duty of care a solicitor owes is not reduced by the use of an AI tool. If anything, the Law Society's emerging guidance — and the AI Act's high-risk requirements — push the obligation higher: you are now responsible for the AI's output as if you wrote it yourself, plus you must demonstrate human oversight.
That demonstration is impossible with a chatbot. There is no record of what the model retrieved, what it ignored, what version it was, or what the temperature setting was on the day. With an on-premise system you can give a junior every output with a citations panel showing the exact source paragraphs from the firm's own documents. You can require a sign-off step before any AI-drafted text leaves the building. You can run nightly evaluations against a held-out set of golden questions to detect model drift. You can show a regulator a complete chain of custody from input to output to reviewer.
Hallucination is the other piece. The genuinely effective mitigation is not "use a bigger model" — it's refuse-to-answer-without-evidence as a system-level constraint. If the retrieval layer returns nothing above a confidence threshold, the system says "I don't have anything in the firm's materials on this point" rather than confecting a plausible answer. That behaviour is straightforward to enforce when you control the stack. It's impossible to enforce on a public chatbot.
The economics work better than partners expect
The objection I hear most is cost. The assumption is that on-premise must be more expensive than a per-seat SaaS subscription. For a five-person firm, fine — SaaS wins on day one. For anything larger, the maths inverts quickly.
A per-fee-earner SaaS legal AI subscription, multiplied across a 40-person firm, multiplied by twelve months, multiplied by the inevitable annual price rises — this is not a small line item. An on-premise deployment is a capital cost (hardware) plus an integration cost (getting it talking to your DMS and matter system) plus ongoing maintenance. The capital cost amortises. The marginal cost of the 41st user is essentially zero.
More importantly, the on-premise system can do work the SaaS tool cannot — because you can put genuinely sensitive material into it. That widens the surface area of what AI can usefully automate in the firm, which is where the real return sits. Summarising public-domain case law is a parlour trick. Triaging an inbound bundle of discovery against the firm's existing matter knowledge is leverage.
Where to start this week
If you're a managing partner reading this on a Tuesday morning, here's a concrete first step: don't buy anything yet. Spend an hour writing down the five tasks your fee-earners do most often that involve reading a document and producing another document. That list is your specification. Then audit — quietly — what tools your team are already using to do those tasks, including the personal ChatGPT accounts they haven't told you about. The gap between those two lists is the size of your problem and also the size of your opportunity. Once you have that, you can have a sensible conversation about what shape of system actually fits your firm — and you can read more about how I think about this on the Intelligence Brain overview. The worst thing you can do is nothing; the second worst is to sign a three-year SaaS contract before you've understood what sovereignty means for your practice.