Why AI infrastructure matters #
Choosing your AI infrastructure is not a technical decision — it is a business decision that impacts privacy, compliance, costs and technological independence.
The problem with cloud AI services #
When your company uses ChatGPT, Gemini, Copilot or other cloud AI services:
- Data passes through US servers (or at least beyond your control)
- Your database schemas, employee queries and uploaded documents are processed by third parties
- Costs scale with usage: more users, more tokens, more spending — with no predictable ceiling
- You depend on a single vendor: if OpenAI changes prices or terms, you have no immediate alternatives
- Compliance is your responsibility: GDPR and the AI Act apply to you as the user, not to the provider
When private AI infrastructure makes sense #
Private AI is not for everyone. It makes sense when:
- You handle personal data (customers, employees, patients)
- You process confidential data (contracts, strategies, intellectual property)
- You operate in regulated sectors (healthcare, finance, public administration)
- You want predictable costs independent of usage volume
- You need a complete audit trail for compliance
- You want to avoid vendor lock-in with a single AI provider
The three options: cloud, on-premise, hybrid #
Option 1: Public cloud (ChatGPT, Gemini, Azure AI) #
| Pros | Cons |
|---|---|
| No initial setup | Data on third-party servers |
| Always up to date | Per-token/per-user costs that grow |
| Unlimited scalability | Vendor lock-in |
| Complex compliance | |
| No control over models |
Suited for: personal use, brainstorming, non-sensitive content.
Option 2: On-premise (servers in your data centre) #
| Pros | Cons |
|---|---|
| Maximum control over data | Initial hardware investment |
| Fixed, predictable cost | Requires management expertise |
| Native compliance | Scalability limited by hardware |
| No vendor lock-in | Updates are your responsibility |
| Complete audit trail |
Suited for: healthcare, ultra-sensitive data, companies with existing IT infrastructure.
Option 3: Managed European cloud #
| Pros | Cons |
|---|---|
| Data in EU data centres | Less control than full on-premise |
| Simplified management | Recurring cost |
| Flexible scalability | Provider dependency (but European) |
| Easier GDPR compliance | |
| No hardware investment |
Suited for: SMEs without a dedicated IT team, companies that want privacy without complexity.
Evaluation criteria #
When choosing a private AI infrastructure, evaluate these 8 criteria:
1. Data localisation #
Where does data physically reside during processing? “European cloud” is not enough — verify:
- Which country the data centre is located in
- Whether the provider is subject to the US Cloud Act (even European subsidiaries of US companies are)
- Whether data transits outside the EU, even temporarily
2. Supported models #
A modern AI infrastructure must support multiple models:
- Open-source models (DeepSeek, LLaMA, Mistral, Qwen) for total privacy
- Commercial models via API (Claude, GPT) for specific use cases
- The ability to fine-tune on your own data
Vendor lock-in on a single model is a risk: models evolve rapidly and today’s best may not be tomorrow’s.
3. RAG capabilities (Retrieval Augmented Generation) #
To use AI with your business documents you need a RAG system that:
- Indexes documents in multiple formats (PDF, Word, Excel, email)
- Searches for relevant passages for each query
- Generates answers with source citations
- Respects document access permissions
4. Security and encryption #
Verify:
- End-to-end encryption (data in transit and at rest)
- Authentication and authorisation (who can access what)
- Logging and audit trail
- Protection against prompt injection and jailbreak
5. Compliance #
The infrastructure should facilitate — not complicate — compliance:
- GDPR: no extra-EU transfers, clear legal basis
- AI Act: transparency, human oversight, documentation
- Sector regulations: ISO 13485 for healthcare, ISO 27001 for information security
6. Scalability #
The infrastructure must grow with your needs:
- From a few users to hundreds
- From one use case to many
- From one model to several specialised models
7. Total cost of ownership (TCO) #
Do not just look at the initial price. Calculate the total cost of ownership:
| Item | Public cloud | On-premise | Managed EU cloud |
|---|---|---|---|
| Setup | Low | High | Medium |
| Monthly cost | Variable (tokens) | Fixed | Fixed |
| Scaling | Linear with usage | Step (new HW) | Flexible |
| Management | None | In-house | Included |
| 3-year cost (100 users) | High and unpredictable | Medium | Medium |
8. Technological independence #
How tied are you to the vendor?
- Can you switch models without rebuilding everything?
- Are your data and configurations portable?
- Does the vendor use open standards?
PRISMA: the private AI stack from HTX #
PRISMA (Private Intelligence Stack for Modular AI) is the private AI infrastructure we built at HTX to solve exactly these problems.
What PRISMA includes #
- Model orchestration: support for DeepSeek, LLaMA, Mistral, Qwen and commercial models via API
- Enterprise RAG: document indexing with source citations
- Security: end-to-end encryption, authentication, audit trail
- Monitoring: usage dashboards, performance metrics, alerting
- API: integration with your existing systems
Products running on PRISMA #
| Product | Function | Details |
|---|---|---|
| ORCA | Private business chatbot | Chat, documents, web search — an alternative to ChatGPT |
| MANTA | Text-to-SQL | Query databases in natural language |
| KOI | Clinical classification | Decision support for anaesthesiology (RUO) |
Where PRISMA operates #
PRISMA can be deployed:
- On-premise: in your data centre or server room
- BIC FVG Data Centre: the certified incubator of the Friuli Venezia Giulia region, with dedicated infrastructure and redundant connectivity
- TriesteValley HPC: high-performance computing cluster with NVIDIA GPUs, for intensive workloads
How to get started #
You do not need a large investment to begin. The typical path:
- Free assessment (1 call): we analyse your use cases and recommend the right configuration
- Pilot (2-4 weeks): we install PRISMA with one product (ORCA or MANTA) for a small group
- ROI measurement: after the pilot, we measure the results together
- Scaling: we extend to more users and use cases based on the results
Contact us for an assessment →
This article was written by the HTX team — Human Technology eXcellence. We design private artificial intelligence systems for healthcare and industry, from our data centre in Trieste.
Frequently asked questions #
What is a private AI infrastructure?
A private AI infrastructure runs artificial intelligence models on servers under your control — on-premise in your data centre or in a dedicated European cloud. Data is never sent to third-party servers such as OpenAI or Google. Full GDPR and AI Act compliance.
How much does a private AI infrastructure cost?
Costs vary depending on the configuration. A basic setup with shared GPUs for an SME starts from a few hundred euros per month. The advantage over cloud services is that the cost is fixed and predictable, with no per-token or per-user charges that grow with usage.
Can I use open-source models in my infrastructure?
Yes. Models such as DeepSeek, LLaMA, Mistral and Qwen are open-source and can be run on-premise without licence fees. The performance of the best open-source models is comparable to commercial models for most business use cases.
On-premise or European cloud: which should I choose?
It depends on your requirements. On-premise offers maximum control but requires management expertise. A European cloud (EU data centre with contractual guarantees) simplifies management while maintaining GDPR compliance. For healthcare and the most sensitive data, on-premise is the recommended choice.
What is PRISMA by HTX?
PRISMA (Private Intelligence Stack for Modular AI) is the private AI infrastructure built by HTX. It integrates model orchestration, RAG, security and monitoring into a single stack. It supports on-premise and European cloud deployment. It is the foundation on which ORCA, MANTA and KOI run.