Bedrock, SageMaker AI or OpenShift AI? Choose the layer by constraint.
A map of six AI architecture layers, from ready-made services to your own GPUs, with the questions, diagrams and cost numbers I use in client reviews and in my lecture on AI architectures at Hochschule Burgenland.
- 6 layers explained
- Decision tool
- Break-even numbers
- Sources linked
Managed or self-managed is the SaaS, PaaS, IaaS question again.
Every AI system sits on a layer. The higher the layer, the more the provider operates for you; the lower, the more control you get and the more your team has to run[2].
Pick the layer by constraint, not by preference: where the data may go, how much you need to customise, how steady the load is, how fast it must answer, and which skills you have.
Hybrid is not a layer. It is what happens when one constraint, usually data residency, forces part of the system onto a lower layer. On list prices, managed services win for almost everyone[3][4]; residency and control, not price, are what justify running your own.
Five questions, one suggested layer.
Runs in your browser, nothing is sent. The suggestion follows the same logic I use at the start of an architecture review.
Which layer fits your use case?
Five questions. The suggestion is a starting point for the review, not a verdict.
1. What does the AI need to do?
2. Where may the data go?
3. How much customisation?
4. How steady and large is the load?
5. Who would operate it?
Six layers, who operates what.
Examples on AWS first, equivalents on other clouds where they exist. Service names change; the layers do not.
| Layer | Name | Examples | What you operate |
|---|---|---|---|
| L0 | Consumer AI product | Claude, Microsoft 365 Copilot, Gemini in Workspace, Notion AI | The tenant: sign-in, retention, connectors |
| L1 | Ready-made capability | Amazon Textract, Transcribe, Rekognition, Comprehend, Bedrock Data Automation | Your process calls an API and gets structured data back |
| L2 | Managed model | Amazon Bedrock with Knowledge Bases, Guardrails and AgentCore; Microsoft Foundry; Vertex AI | Model choice, prompts, retrieval, guardrails, evaluation |
| L3 | Managed platform | Amazon SageMaker AI; Azure Machine Learning; Vertex AI | Your data, code, training jobs and endpoints |
| L4 | Your own platform | Red Hat OpenShift AI on ROSA, ARO or on-premises; Kubernetes with vLLM and KServe | The runtime, upgrades, scaling, failures |
| L5 | Infrastructure | GPU instances such as EC2 G6e, AWS Outposts, own racks; FSx for NetApp ONTAP as storage bridge | The machines, networks and storage |
Ready-made capability or managed model.
The most common decision. Amazon Textract returns fixed fields for one page synchronously and up to 3,000 pages asynchronously[9]; a model on Bedrock follows a prompt and needs evaluation.
Rent the platform or hold the platform.
SageMaker AI runs the ML lifecycle for you and bills four meters: notebooks, training, hosting and extras such as MLflow[7]. OpenShift AI gives you the same lifecycle on a platform you run, on ROSA, ARO or on-premises[11][12].
Four ways to serve a model, four ways to pay.
Real-time endpoints bill while they exist; serverless bills per request; asynchronous scales to zero if configured; batch bills only while it runs. The payload and time limits decide as much as the price[8].
Route by data classification.
Amazon Bedrock does not run on AWS Outposts; what reaches on-premises data are agents and tools[15][16]. The practical pattern is routing: a managed model in an EU Region for data that may leave, a self-hosted model in the building for data that may not.
When does your own GPU pay off?
List prices, us-east-1, checked August 2026[3][4][5]. The break-even is a utilisation requirement, not a capability limit: one L40S GPU can serve an 8B model faster than this rate[6], the question is whether your traffic keeps it busy all month.
| Model | Amazon Bedrock | Self-hosted on EC2 | Break-even per month |
|---|---|---|---|
| Llama 3.1 8B Instruct | USD 0.22 per 1M tokens | g6e.xlarge, USD 1.861/h = USD 1,358.53 per 730-h month | 6.18 bn tokens (about 2,350 tokens/s around the clock) |
| Llama 3.3 70B Instruct | USD 0.72 per 1M tokens | g6e.12xlarge, USD 10.49/h on demand | 10.6 bn tokens |
| Llama 3.3 70B Instruct | USD 0.72 per 1M tokens | g6e.12xlarge, USD 6.61/h with 1-year reservation | 6.7 bn tokens |
Derived from the cited list prices. Excludes storage, data transfer, engineers and the platform itself, which all push the break-even higher. Try your own numbers in the self-hosting vs Bedrock calculator.
A hybrid architecture in five steps.
The review ends with a diagram anyone in the room can read. Two boxes, the data where it lives, the services you can name, arrows labelled with what they carry, and the reasons under the picture[19].
Get the layer decision in writing.
One workshop with your architects and the data owner. Results you choose, €1,000 each. The standard review includes the layer decision record and the cost and break-even sheet.
AI architecture layers review
Intake, a 3 to 4-hour workshop, written results within 5 business days.
€2,000 each, 3 to 4 hours, remote or on-site
Delivered within 5 business days after the last session.
Layers review
Layers review + diagram
Architecture advisory
Hybrid and cost, documented in the open.
Each link opens a write-up on ai-solutions.wiki, the open engineering reference I maintain. Architecture, services and trade-offs are explained there in full.
On-premise vs cloud AI
Where self-hosting wins, where managed services win, and the costs people forget.
Read on ai-solutions.wiki →MediaHybrid cloud AI video pipeline with FSx for NetApp ONTAP
Footage stays on file shares; AWS AI services read it through S3 access points.
Read on ai-solutions.wiki →GeospatialGIS and AI architecture on AWS
Geospatial processing with AI services and natural language queries over spatial data.
Read on ai-solutions.wiki →GuideAI total cost of ownership
Model, infrastructure, people and operations cost in one view.
Read on ai-solutions.wiki →GuideCapacity planning for AI
Sizing GPUs and throughput before you buy them.
Read on ai-solutions.wiki →ComparisonSageMaker vs Vertex AI
The two managed ML platforms side by side.
Read on ai-solutions.wiki →The AWS services behind this work.
Plain-language notes on the services I use for this kind of project. Each name links to the official AWS documentation.
Amazon Textract
Reads text, forms and tables from scanned documents and returns them as structured data.
Amazon Bedrock
Managed access to foundation models from several providers through one API. You choose the model and write the prompt; AWS runs the model.
Amazon Bedrock AgentCore
Runtime, memory, identity and tool gateway for AI agents built with any framework, such as Strands or CrewAI.
Amazon SageMaker AI
Managed platform for the whole machine learning lifecycle: notebooks, training jobs, model registry and endpoints.
Red Hat OpenShift Service on AWS (ROSA)
Managed OpenShift clusters in your AWS account. The base for running OpenShift AI next to other AWS services.
Amazon EC2 G6e instances
GPU instances with NVIDIA L40S cards for self-hosted inference, for example with vLLM.
AWS Outposts
AWS-managed racks in your own data centre for workloads that must stay on site.
Amazon FSx for NetApp ONTAP
Managed NetApp file storage on AWS. With S3 access points it bridges on-premises file shares and cloud AI services.
What people ask before booking.
What is the difference between Amazon Bedrock and SageMaker AI?
Bedrock is a managed model service: you call foundation models through one API and configure prompts, retrieval, guardrails and agents. SageMaker AI is a managed ML platform: you bring data and code, run training jobs and host your own model endpoints.
When should we run Red Hat OpenShift AI instead?
When a constraint requires you to hold the platform: data that may not leave the building, the same platform across cloud and on-premises, or full control of the model runtime. It needs a team that can run OpenShift and GPUs.
Is self-hosting an open model cheaper than Bedrock?
Only at high, steady utilisation. For Llama 3.1 8B on one g6e.xlarge the list-price break-even is about 6.18 billion tokens a month, before engineering and platform costs. Most workloads are far below that.
Can Bedrock run on premises?
No. Bedrock is a Regional service and does not install on AWS Outposts. For on-premises inference you run a model yourself, for example with vLLM on OpenShift AI, and route requests by data class.
What does the architecture review cost?
€2,000 for the workshop plus €1,000 per written result. The standard review with layer decision record and break-even sheet is €4,000 net.
Linda Mohamed
I teach this map in the master course “Von Managed zu Hybrid: KI-Architekturen” at Hochschule Burgenland and use it in client reviews on AWS and Red Hat platforms. AWS Community Hero, organiser of the AWS User Group Vienna, maintainer of ai-solutions.wiki. Based in Vienna, remote across Europe, English or German.
Where the facts come from.
Prices, service names and limits change. The linked official pages are the reference; figures on this page were checked in September 2026.
- AWS: AI services overview
- AWS shared responsibility model
- AWS: Amazon Bedrock pricing (Llama 3.1 8B and Llama 3.3 70B, us-east-1, checked August 2026)
- AWS: Amazon EC2 On-Demand pricing (g6e.xlarge, g6e.12xlarge)
- AWS: Amazon EC2 Reserved Instances pricing
- NVIDIA NIM for LLMs: benchmarking and performance
- AWS: Amazon SageMaker AI pricing
- AWS: SageMaker AI model deployment options and limits
- AWS: Amazon Textract document limits
- AWS: What is Amazon Bedrock AgentCore?
- Red Hat: OpenShift AI supported configurations 3.x
- Red Hat OpenShift AI Self-Managed 3.5 documentation
- Red Hat: ROSA architecture models
- AWS: FSx for NetApp ONTAP, S3 access points
- AWS Machine Learning Blog: RAG with data residency using AWS hybrid and edge services
- AWS Outposts rack
- vLLM documentation
- AWS: Amazon Bedrock cross-Region inference
- The C4 model for visualising software architecture
- ai-solutions.wiki: On-premise vs cloud AI
- ai-solutions.wiki: AI total cost of ownership
- ai-solutions.wiki: Capacity planning for AI
- AWS: Custom Model Import in Amazon Bedrock
Start with one conversation.
30 minutes to look at your idea, your data and the right starting point. Or book 15 minutes if you only have a question.
- Which layer your use case needs
- Where your data may go
- Whether hybrid routing is worth it
- What the review would cover
Vienna, Austria · remote across Europe · [email protected]
