On-premise AI for SMEs in Spain: complete guide 2026
What an SME needs to run artificial intelligence on its own servers: hardware, open models, integration with your systems, costs, timelines and regulation.
Published Updated
Where it can run
In your office
Your own server inside your network.
Dedicated server in Spain
In a data centre, with no hardware to maintain.
European private cloud
Isolated infrastructure in the EU.
Concepts
What "on-premise AI" means
"On-premise" means "on the premises": strictly, the software runs on hardware in your office. In practice the term also covers a model running on a dedicated server or a private cloud you control. The opposite is the public cloud or SaaS, where the provider manages the infrastructure. In AI, it means the language model that processes text and generates answers is installed on your servers, and your data is processed inside your environment.
- On-premise
- On your premises. The model runs on infrastructure you control, and you decide whether it has internet access or not.
- LLM
- Language model. The engine that understands and writes text. Some open models can be downloaded and installed on your own server.
- RAG
- Search over your documents. Before answering, the AI looks up the relevant passages of your documentation and can show where each fact comes from.
- Fine-tuning
- Model tuning. Retraining the model on your own examples to adapt its style or vocabulary. For most SMEs, RAG and good instructions are enough.
- Inference
- Running the model. Every time the model answers. It is what uses the GPU and what you need to size.
Why
Why a Spanish SME should consider on-premise AI
The four most common reasons to run AI in-house instead of buying it as a service.
Protection of business data
Client data, proposals, projects and financial figures are processed on your infrastructure. You don't depend on an external AI provider's terms of service to know what happens to them.GDPR and AI Act: demonstrable control
If you don't send personal data to an AI provider, you have fewer processors and fewer transfers to justify. And if your use case is high-risk under the AI Act, controlling the infrastructure makes it easier to provide the logging and human oversight those systems will require.Predictable long-term cost
AI subscriptions are usually charged per user per month and grow with your team. On-premise, the main cost is the infrastructure and its maintenance, which depends less on the number of users.Customisation: an AI that knows your business
With RAG it works on your internal documentation; with integrations, on your CRM or ERP; and, if needed, fine-tuning adapts its vocabulary to your sector.
Cloud AI (SaaS)
- Where the model runs
- On the provider's servers
- Where your data is processed
- Outside your network, under the provider's terms
- How you pay
- Per user or per usage, every month
- Which model you use
- Whatever the provider offers
- Without internet access
- It doesn't work
On-premise AI
- Where the model runs
- On your server or the environment you choose
- Where your data is processed
- Inside your infrastructure
- How you pay
- Infrastructure, deployment and maintenance
- Which model you use
- An open model you choose and can swap
- Without internet access
- It can run on an isolated network
The cloud is still a good option for tasks without sensitive data, and many SMEs combine both.
Regulation, as of November 2026
- GDPR
It applies to any company that processes personal data, whatever its size. For the most serious infringements, fines can reach €20 million or 4% of total worldwide annual turnover of the preceding year, whichever is higher.
- AI Act
Regulation (EU) 2024/1689, in force since August 2024 and applied in stages: prohibitions and AI literacy since February 2025, general-purpose models since August 2025 and, as a general rule, the rest since August 2026. The Digital Omnibus on AI (Regulation (EU) 2026/1744, in force since July 2026) postponed the obligations for high-risk systems to 2 December 2027 (Annex III) and 2 August 2028 (Annex I products), recast AI literacy as a duty to support its promotion and added two prohibitions that apply from 2 December 2026.
- AEPD · AESIA
In Spain, the AEPD supervises the processing of personal data and has published guidance on AI; AESIA is the state agency created to supervise artificial intelligence. The draft Organic Law on the good use and governance of artificial intelligence, sent to Parliament in May 2026, designates it as the main supervisory authority.
Looking for the strategic why (sovereignty, traceability and governance) rather than the how? We cover it in private, governed AI for businesses.
Resources
Does my SME have the resources for this?
This is the question we get asked most. The short answer: it depends on the use case, but probably yes. The starting point is not a supercomputer but a well-sized server with the right model.
Hardware requirements
One GPU with 16–24 GB of memory
Small models, around 7 to 20 billion parameters, quantised
Good for
Internal assistant, summaries and questions over documents for a small team.
A 48–80 GB professional GPU, or several GPUs
Mid-size models, around 24 to 35 billion, or quantised large ones
Good for
More users at once, long documents and better answers on complex tasks.
A server with several data-centre GPUs
Large models, 70 billion and up, and mixture-of-experts models
Good for
Many concurrent users, agents and high-volume automations.
General guidance. GPU memory is the main limit: a model quantised to 4 bits takes a little over half a gigabyte per billion parameters, plus memory for the context. We size it during the assessment with your number of users and your documents.
Personnel requirements
You don't need a team of data scientists. You need three roles, and you probably already have two.
Someone to manage the server
It can be the same person who looks after your systems or your website today.
A business owner
Decides which use cases go first and checks that the answers are useful.
A technology partner for the deployment
At Galileo Studio we choose the model, deploy it, connect it to your systems and train your team.
Models
Common open language models in 2026
The open model ecosystem has matured a lot. These are common families in on-premise deployments; each has several sizes and versions that are updated often.
For most SMEs, a small or mid-size model well connected to their documents is enough. Large models are justified when the task is complex or quality is critical.
Llama
Meta
One of the most widespread families, with broad tool support. Meta's own licence, with conditions of use: Llama 4's, for example, does not grant its multimodal models to companies with their principal place of business in the EU.
Mistral
Mistral AI · France
European models with a good quality-to-speed ratio; several are released under the Apache 2.0 licence.
Qwen
Alibaba
A wide range of sizes, strong at code and in several languages; many under the Apache 2.0 licence.
Gemma
Google
Compact models designed for modest hardware. Earlier versions use Google's own terms; Gemma 4 is released under the Apache 2.0 licence.
gpt-oss
OpenAI
Open-weight models under the Apache 2.0 licence; the smallest is designed to run with about 16 GB of memory.
Phi
Microsoft
Small models for machines with limited resources.
ALIA
Barcelona Supercomputing Center
A public Spanish family, trained with special attention to Spanish and Spain's co-official languages.
Before using a model in production, check its licence: not all of them allow the same uses. On the Spanish model, read what ALIA is and what it brings to AI sovereignty.
Architecture
How it connects to your systems
An on-premise installation is not just the model. It has four layers, and all of them run inside your infrastructure.
01Interface
A team chat, similar in use to ChatGPT, or AI inside the tools you already use.
Open WebUI · OpenAI-compatible API
02Orchestration and RAG
Searches your documents, applies permissions and prepares the request to the model.
LangChain · LlamaIndex · n8n
03Inference engine
Runs the model on the GPU and serves several users at once.
vLLM · Ollama · llama.cpp
04Your data
Documents, databases and business systems, connected read-only unless you decide otherwise.
SharePoint · CRM · ERP · Qdrant · pgvector
Security and permissions
- Sign-in with company accounts: Microsoft Entra ID, Google Workspace or LDAP.
- Permissions per person, team and folder: the AI only uses the documents each user can see.
- Usage log: what was asked, when and by whom.
- Option to run on a network with no internet access.
Maintenance and operations
- Model and software updates when they bring improvements, tested before going to production.
- Backups of the configuration and the document indexes.
- Monitoring of usage and GPU load to know when to scale.
Use cases
Use cases for Spanish SMEs
Tasks an on-premise AI can take on across sectors. In all of them, information is processed inside your infrastructure.
Manuals that answer
Technical manuals and maintenance procedures searchable in natural language, connected to plant documentation, plus support for training new technicians.
Illustrative example
What are the steps to change the filter on line 3?
Stop the line, depressurise the circuit and replace the cartridge. Check the seal before restarting.
Source: maintenance-manual-L3.pdf · section 4.2
Processed on your server · no internet access
Questions on regulation
Summaries of regulatory changes and drafts of client communications. Connected to your document base, it answers citing the source document.
Contracts and briefs
Contract analysis, search across internal documentation and brief preparation, without sending confidential information to external servers.
Proposals and reports
Drafts of proposals, status reports, market analyses and meeting summaries. Connected to the CRM, it prepares personalised drafts for each client.
Clinical documentation
Help drafting reports and finding information, with health data inside your perimeter and always reviewed by the professional. The AI does not diagnose or decide. Health data is a special category under the GDPR: before starting, we review the legal basis and safeguards with you.
Deployment
The deployment process
This is how a first use case usually goes, from the first meeting to production. Timelines are indicative: they depend on the case, your systems and whether the hardware is already available.
Week 1
Assessment
We pick the priority use case, review your existing infrastructure and define the right model and hardware.
Weeks 2–3
Deployment
We install the model and the interface, connect your documents and systems, and set up access and permissions.
Week 4
Pilot
A small group tests it with real cases. We adjust instructions, permissions and parameters based on what we see.
Week 5 onwards
Production
Roll-out to the rest of the team, documentation, training and support, with knowledge transfer to your technical team.
Costs
How much on-premise AI costs
There is no single price: the cost depends on the number of users, the model size and whether you already have a server. These are the items worth comparing with a cloud subscription.
Infrastructure
Buying a GPU server or renting a dedicated one. It is the item that varies most with the model and the users.Deployment
Assessment, deployment, integrations and pilot. Paid once per use case.Operations
Maintenance, updates, energy or data-centre fees, and support.Versus the cloud
Subscriptions are paid per user or per usage every month. With more users and more usage, your own infrastructure tends to pay off; with little usage, the cloud can be cheaper.
During the assessment we calculate the break-even point with your real figures. If your use case is not clear yet, start with our AI consulting for SMEs.
Technical FAQ
Frequently asked questions from SMEs
The technical questions we hear most before installing AI on your own servers.
Have another technical question? Write to us
01How many users can it serve at once?
It depends on the model, the GPU and the inference engine. A server with a single GPU can serve a small team with a compact model; for more users you add another GPU or deploy more instances. We measure it during the pilot with your real usage.
02Does it work without internet access?
Yes. Once installed, the model doesn't need the internet to work, so it can run on an isolated network. You only need a connection, if you choose, to download updates.
03Can I use my own documents as a knowledge base?
Yes, and it is usually the most valuable use case. With RAG, the AI searches your PDFs, Word documents or databases before answering, and can show which document each answer comes from. Access permissions are respected.
04What server do I need to get started?
For a first use case with a small team, a server with one GPU with 16 to 24 GB of memory is usually enough. If you'd rather not buy hardware, you can rent a dedicated server in a data centre in Spain.
05Do I need to retrain the model on my data?
Usually not. Connecting the model to your documents with RAG and giving it good instructions covers most cases. Fine-tuning is for when you need a very specific style or vocabulary.
06Can it be combined with cloud models?
Yes. A hybrid setup is common: sensitive data is processed by the local model, and tasks without confidential information can go to a cloud service, with clear rules on what goes where.
07Is on-premise AI legal in Spain?
Yes. Installing and using open models on your servers is legal as long as you respect their licence and the regulations that apply to you. Under the GDPR, processing data on your own infrastructure avoids sending it to an external AI provider, although you still need a legal basis, to inform data subjects and security measures. The AI Act regulates uses by risk level, not where the model runs.
08Who takes care of maintenance?
We agree it with you: we can take care of it or train your technical team to run it. The usual tasks are updating the model and software, reviewing usage and adding capacity when needed.
Conclusion
Conclusion: where to start
On-premise AI for SMEs is no longer reserved for large companies. With today's open models, an SME can have a first AI assistant in production within a few weeks, depending on the case and whether the hardware is already available, with data inside its infrastructure and no per-user licences from an AI provider: the cost lies in infrastructure, deployment and maintenance.
The first step is choosing which task, among those that take up your team's time today, the AI would take on. From there, the path is short.