Galileo Studio
Technical guide · On-premise AI

On-premise AI for SMEs in Spain: complete guide 2026

What an SME needs to run artificial intelligence on its own servers: hardware, open models, integration with your systems, costs, timelines and regulation.

Published Updated

Where it can run

  • In your office

    Your own server inside your network.

  • Dedicated server in Spain

    In a data centre, with no hardware to maintain.

  • European private cloud

    Isolated infrastructure in the EU.

Concepts

What "on-premise AI" means

"On-premise" means "on the premises": strictly, the software runs on hardware in your office. In practice the term also covers a model running on a dedicated server or a private cloud you control. The opposite is the public cloud or SaaS, where the provider manages the infrastructure. In AI, it means the language model that processes text and generates answers is installed on your servers, and your data is processed inside your environment.

On-premise
On your premises. The model runs on infrastructure you control, and you decide whether it has internet access or not.
LLM
Language model. The engine that understands and writes text. Some open models can be downloaded and installed on your own server.
RAG
Search over your documents. Before answering, the AI looks up the relevant passages of your documentation and can show where each fact comes from.
Fine-tuning
Model tuning. Retraining the model on your own examples to adapt its style or vocabulary. For most SMEs, RAG and good instructions are enough.
Inference
Running the model. Every time the model answers. It is what uses the GPU and what you need to size.

Why

Why a Spanish SME should consider on-premise AI

The four most common reasons to run AI in-house instead of buying it as a service.

  • Protection of business data

    Client data, proposals, projects and financial figures are processed on your infrastructure. You don't depend on an external AI provider's terms of service to know what happens to them.
  • GDPR and AI Act: demonstrable control

    If you don't send personal data to an AI provider, you have fewer processors and fewer transfers to justify. And if your use case is high-risk under the AI Act, controlling the infrastructure makes it easier to provide the logging and human oversight those systems will require.
  • Predictable long-term cost

    AI subscriptions are usually charged per user per month and grow with your team. On-premise, the main cost is the infrastructure and its maintenance, which depends less on the number of users.
  • Customisation: an AI that knows your business

    With RAG it works on your internal documentation; with integrations, on your CRM or ERP; and, if needed, fine-tuning adapts its vocabulary to your sector.

Cloud AI (SaaS)

Where the model runs
On the provider's servers
Where your data is processed
Outside your network, under the provider's terms
How you pay
Per user or per usage, every month
Which model you use
Whatever the provider offers
Without internet access
It doesn't work

On-premise AI

Where the model runs
On your server or the environment you choose
Where your data is processed
Inside your infrastructure
How you pay
Infrastructure, deployment and maintenance
Which model you use
An open model you choose and can swap
Without internet access
It can run on an isolated network

The cloud is still a good option for tasks without sensitive data, and many SMEs combine both.

Regulation, as of November 2026

  • GDPR

    It applies to any company that processes personal data, whatever its size. For the most serious infringements, fines can reach €20 million or 4% of total worldwide annual turnover of the preceding year, whichever is higher.

  • AI Act

    Regulation (EU) 2024/1689, in force since August 2024 and applied in stages: prohibitions and AI literacy since February 2025, general-purpose models since August 2025 and, as a general rule, the rest since August 2026. The Digital Omnibus on AI (Regulation (EU) 2026/1744, in force since July 2026) postponed the obligations for high-risk systems to 2 December 2027 (Annex III) and 2 August 2028 (Annex I products), recast AI literacy as a duty to support its promotion and added two prohibitions that apply from 2 December 2026.

  • AEPD · AESIA

    In Spain, the AEPD supervises the processing of personal data and has published guidance on AI; AESIA is the state agency created to supervise artificial intelligence. The draft Organic Law on the good use and governance of artificial intelligence, sent to Parliament in May 2026, designates it as the main supervisory authority.

Looking for the strategic why (sovereignty, traceability and governance) rather than the how? We cover it in private, governed AI for businesses.

Resources

Does my SME have the resources for this?

This is the question we get asked most. The short answer: it depends on the use case, but probably yes. The starting point is not a supercomputer but a well-sized server with the right model.

Hardware requirements

Small team

One GPU with 16–24 GB of memory

Small models, around 7 to 20 billion parameters, quantised

Good for

Internal assistant, summaries and questions over documents for a small team.

Several departments

A 48–80 GB professional GPU, or several GPUs

Mid-size models, around 24 to 35 billion, or quantised large ones

Good for

More users at once, long documents and better answers on complex tasks.

Heavy use

A server with several data-centre GPUs

Large models, 70 billion and up, and mixture-of-experts models

Good for

Many concurrent users, agents and high-volume automations.

General guidance. GPU memory is the main limit: a model quantised to 4 bits takes a little over half a gigabyte per billion parameters, plus memory for the context. We size it during the assessment with your number of users and your documents.

Personnel requirements

You don't need a team of data scientists. You need three roles, and you probably already have two.

  • Someone to manage the server

    It can be the same person who looks after your systems or your website today.

  • A business owner

    Decides which use cases go first and checks that the answers are useful.

  • A technology partner for the deployment

    At Galileo Studio we choose the model, deploy it, connect it to your systems and train your team.

Models

Common open language models in 2026

The open model ecosystem has matured a lot. These are common families in on-premise deployments; each has several sizes and versions that are updated often.

For most SMEs, a small or mid-size model well connected to their documents is enough. Large models are justified when the task is complex or quality is critical.

  • Llama

    Meta

    One of the most widespread families, with broad tool support. Meta's own licence, with conditions of use: Llama 4's, for example, does not grant its multimodal models to companies with their principal place of business in the EU.

  • Mistral

    Mistral AI · France

    European models with a good quality-to-speed ratio; several are released under the Apache 2.0 licence.

  • Qwen

    Alibaba

    A wide range of sizes, strong at code and in several languages; many under the Apache 2.0 licence.

  • Gemma

    Google

    Compact models designed for modest hardware. Earlier versions use Google's own terms; Gemma 4 is released under the Apache 2.0 licence.

  • gpt-oss

    OpenAI

    Open-weight models under the Apache 2.0 licence; the smallest is designed to run with about 16 GB of memory.

  • Phi

    Microsoft

    Small models for machines with limited resources.

  • ALIA

    Barcelona Supercomputing Center

    A public Spanish family, trained with special attention to Spanish and Spain's co-official languages.

Before using a model in production, check its licence: not all of them allow the same uses. On the Spanish model, read what ALIA is and what it brings to AI sovereignty.

Architecture

How it connects to your systems

An on-premise installation is not just the model. It has four layers, and all of them run inside your infrastructure.

  1. 01Interface

    A team chat, similar in use to ChatGPT, or AI inside the tools you already use.

    Open WebUI · OpenAI-compatible API

  2. 02Orchestration and RAG

    Searches your documents, applies permissions and prepares the request to the model.

    LangChain · LlamaIndex · n8n

  3. 03Inference engine

    Runs the model on the GPU and serves several users at once.

    vLLM · Ollama · llama.cpp

  4. 04Your data

    Documents, databases and business systems, connected read-only unless you decide otherwise.

    SharePoint · CRM · ERP · Qdrant · pgvector

Security and permissions

  • Sign-in with company accounts: Microsoft Entra ID, Google Workspace or LDAP.
  • Permissions per person, team and folder: the AI only uses the documents each user can see.
  • Usage log: what was asked, when and by whom.
  • Option to run on a network with no internet access.

Maintenance and operations

  • Model and software updates when they bring improvements, tested before going to production.
  • Backups of the configuration and the document indexes.
  • Monitoring of usage and GPU load to know when to scale.

Use cases

Use cases for Spanish SMEs

Tasks an on-premise AI can take on across sectors. In all of them, information is processed inside your infrastructure.

Industrial companies

Manuals that answer

Technical manuals and maintenance procedures searchable in natural language, connected to plant documentation, plus support for training new technicians.

Illustrative example

What are the steps to change the filter on line 3?

Stop the line, depressurise the circuit and replace the cartridge. Check the seal before restarting.

Source: maintenance-manual-L3.pdf · section 4.2

Processed on your server · no internet access

Accounting and tax firms

Questions on regulation

Summaries of regulatory changes and drafts of client communications. Connected to your document base, it answers citing the source document.

Law firms and notaries

Contracts and briefs

Contract analysis, search across internal documentation and brief preparation, without sending confidential information to external servers.

Agencies and consultancies

Proposals and reports

Drafts of proposals, status reports, market analyses and meeting summaries. Connected to the CRM, it prepares personalised drafts for each client.

Private clinics

Clinical documentation

Help drafting reports and finding information, with health data inside your perimeter and always reviewed by the professional. The AI does not diagnose or decide. Health data is a special category under the GDPR: before starting, we review the legal basis and safeguards with you.

Deployment

The deployment process

This is how a first use case usually goes, from the first meeting to production. Timelines are indicative: they depend on the case, your systems and whether the hardware is already available.

  1. Week 1

    Assessment

    We pick the priority use case, review your existing infrastructure and define the right model and hardware.

  2. Weeks 2–3

    Deployment

    We install the model and the interface, connect your documents and systems, and set up access and permissions.

  3. Week 4

    Pilot

    A small group tests it with real cases. We adjust instructions, permissions and parameters based on what we see.

  4. Week 5 onwards

    Production

    Roll-out to the rest of the team, documentation, training and support, with knowledge transfer to your technical team.

Costs

How much on-premise AI costs

There is no single price: the cost depends on the number of users, the model size and whether you already have a server. These are the items worth comparing with a cloud subscription.

  • Infrastructure

    Buying a GPU server or renting a dedicated one. It is the item that varies most with the model and the users.
  • Deployment

    Assessment, deployment, integrations and pilot. Paid once per use case.
  • Operations

    Maintenance, updates, energy or data-centre fees, and support.
  • Versus the cloud

    Subscriptions are paid per user or per usage every month. With more users and more usage, your own infrastructure tends to pay off; with little usage, the cloud can be cheaper.

During the assessment we calculate the break-even point with your real figures. If your use case is not clear yet, start with our AI consulting for SMEs.

Technical FAQ

Frequently asked questions from SMEs

The technical questions we hear most before installing AI on your own servers.

Have another technical question? Write to us

01How many users can it serve at once?

It depends on the model, the GPU and the inference engine. A server with a single GPU can serve a small team with a compact model; for more users you add another GPU or deploy more instances. We measure it during the pilot with your real usage.

02Does it work without internet access?

Yes. Once installed, the model doesn't need the internet to work, so it can run on an isolated network. You only need a connection, if you choose, to download updates.

03Can I use my own documents as a knowledge base?

Yes, and it is usually the most valuable use case. With RAG, the AI searches your PDFs, Word documents or databases before answering, and can show which document each answer comes from. Access permissions are respected.

04What server do I need to get started?

For a first use case with a small team, a server with one GPU with 16 to 24 GB of memory is usually enough. If you'd rather not buy hardware, you can rent a dedicated server in a data centre in Spain.

05Do I need to retrain the model on my data?

Usually not. Connecting the model to your documents with RAG and giving it good instructions covers most cases. Fine-tuning is for when you need a very specific style or vocabulary.

06Can it be combined with cloud models?

Yes. A hybrid setup is common: sensitive data is processed by the local model, and tasks without confidential information can go to a cloud service, with clear rules on what goes where.

07Is on-premise AI legal in Spain?

Yes. Installing and using open models on your servers is legal as long as you respect their licence and the regulations that apply to you. Under the GDPR, processing data on your own infrastructure avoids sending it to an external AI provider, although you still need a legal basis, to inform data subjects and security measures. The AI Act regulates uses by risk level, not where the model runs.

08Who takes care of maintenance?

We agree it with you: we can take care of it or train your technical team to run it. The usual tasks are updating the model and software, reviewing usage and adding capacity when needed.

Conclusion

Conclusion: where to start

On-premise AI for SMEs is no longer reserved for large companies. With today's open models, an SME can have a first AI assistant in production within a few weeks, depending on the case and whether the hardware is already available, with data inside its infrastructure and no per-user licences from an AI provider: the cost lies in infrastructure, deployment and maintenance.

The first step is choosing which task, among those that take up your team's time today, the AI would take on. From there, the path is short.