Can different models be used side by side?
Yes. Model routing is one of the main reasons for an AI operating layer: each task can use the model that fits its quality, cost and data protection requirements.
AI Operating System · Architecture
Reference Architecture of an AI Operating System
How the layers of an organisation-wide AI layer fit together and how a request flows from a user or system to a model and back.
The layers
Access layer
Chat interfaces, APIs and integrations through which people and systems use AI.
Control plane
Identity, policies, approvals, logging and cost tracking applied to every request.
Orchestration
Agents and workflows that split tasks, call tools and keep state.
Model layer
Access to several models with routing by task, data class and cost.
Knowledge layer
Indexed documents and data sources with permission-aware retrieval.
How a request flows
- A user asks a question or a system triggers a task
- The control plane checks identity and applicable policies
- Orchestration decides which agent or workflow handles it
- Relevant knowledge is retrieved within the user's permissions
- A suitable model is selected and called
- Actions on systems require the defined approvals
- Everything is logged for audit and evaluation
Integration with enterprise systems
Connectors to CRM, ERP, ticketing, collaboration and storage systems should be built once in the layer and reused by all agents. Write actions deserve stricter controls than read access.
Deployment options
Cloud, private cloud and on-premises components can be mixed. Data classes often determine where processing may happen: public data with any approved model, confidential data only with models in a controlled environment.
Frequently asked questions
Where is company knowledge stored?
Usually it stays in the source systems. The knowledge layer indexes it and enforces existing permissions during retrieval.