Requirements
Everything to prepare before installing a self-hosted AI Runtime gateway: where it runs, how much it needs, the data stores behind it, what to expose, and which connections it opens.
Starting points, not limits
The figures on this page are starting points. Final sizing depends on your traffic volume and detection options, and is agreed with ByteHide for your deployment. Contact sales to scope it.
Platform
| Item | Requirement |
|---|---|
| Cloud | AWS, Microsoft Azure or Google Cloud. Virtual machines also run in your own data center |
| Orchestration | Managed Kubernetes (EKS, AKS, GKE) with Helm, recommended for production. Docker Compose on virtual machines. Managed container services (ECS/Fargate, Container Apps, Cloud Run) from a reference design |
| Infrastructure as code | Terraform or Pulumi |
| Container registry | Access to ByteHide's registry with read-only credentials, or your own registry with the image mirrored to it |
Sizing
Gateway
The gateway scales by adding replicas, never by making one replica bigger. Each replica is a single process, and memory, not CPU, is what limits it: a production replica typically uses well under 1 vCPU.
| Detection profile | CPU (minimum / recommended) | Memory (minimum / recommended) |
|---|---|---|
| Standard PII detection (recommended for production) | 1 / 2 vCPU | 2 / 4 GB |
| Extended PII detection, two languages | 2 / 4 vCPU | 16 GB |
| Extended PII detection, one language | 2 / 4 vCPU | 3 / 4 GB |
| Patterns only | 0.5 / 1 vCPU | 0.5 / 1 GB |
The PII detection profile is the setting that moves memory the most. Plan 16 GB per replica for extended detection in two languages until your own load tests show otherwise.
A replica takes 60 to 180 seconds to start with full PII detection, and under 5 seconds with patterns only. Configure the startup probe with a margin of 180 to 300 seconds.
Data Stores
| Component | Starting size | Note |
|---|---|---|
| PostgreSQL | 2 vCPU, 4 GB memory | Storage grows with traffic: one incidence per inspected request |
| Redis | 1 vCPU, 1 to 2 GB memory | No persistence needed |
High Availability
Starting points for a production deployment that survives the loss of one replica or one availability zone:
| Component | Starting point |
|---|---|
| Gateway | At least 2 replicas in different availability zones, each sized for your detection profile (table above). 3 replicas across 3 zones keep 2 serving while one is replaced or a zone fails. Roll out without dropping below your running replica count, and roll back automatically when a new version fails its health checks |
| PostgreSQL | 2 vCPU and 4 to 8 GB of memory on a non-burstable class, with a synchronous standby in a second zone and automatic failover (Multi-AZ, zone-redundant or regional high availability, depending on your cloud). Storage autoscaling, and at least 7 days of point-in-time recovery |
| Redis | A primary and at least one replica in another zone, with automatic failover and cluster mode disabled. 1 to 2 vCPU and 1 to 2 GB per node, with TLS and authentication |
| Load balancer and egress | The load balancer enabled in every zone where replicas run, and one NAT gateway per zone so egress survives a zone failure. Each NAT adds one fixed egress address to allow-list |
PostgreSQL
| Item | Requirement |
|---|---|
| Version | 12 or later. 14 or later recommended |
| Managed services | RDS for PostgreSQL or Aurora PostgreSQL, Azure Database for PostgreSQL Flexible Server, Cloud SQL for PostgreSQL or AlloyDB. Any standard connection string works |
| Connection pooling | Put PgBouncer or your provider's proxy in front. Without a pooler, the gateway opens a connection per request |
| Two connection strings | A pooled one for the gateway, and a direct one, with schema-change privileges, used only by the migration job |
| Availability | Required to start and to serve traffic: if it goes down, routes stop resolving |
| Backups | Automated backups and point-in-time recovery, with a tested restore |
| Retention | Define a retention and purge policy before production. The gateway does not purge incidences on its own |
Redis
| Item | Requirement |
|---|---|
| Version | 6.0 or later. 7 recommended. Needs sorted sets with expiry and Lua scripting |
| Managed services | ElastiCache for Redis or MemoryDB, Azure Cache for Redis, Memorystore for Redis |
| Cluster mode | Disabled |
| When required | With more than one gateway replica. It shares rate limits, token limits, automatic bans and the gateway key map across replicas |
| Availability | Make it highly available. By default the gateway rejects requests when Redis is down |
Network
Ports
| From | To | Port |
|---|---|---|
| Your load balancer | Gateway | 8080 (HTTP, configurable). The only port the gateway listens on: there is no separate metrics or management port |
| Gateway | PostgreSQL | 5432 |
| Gateway | Redis | 6379 |
| Gateway | ByteHide classification service, ByteHide platform, IP reputation | 443 (HTTPS) |
| Gateway | Model providers, MCP servers, identity providers | 443, or the port your private MCP servers use |
TLS
- The gateway does not terminate TLS. Terminate it at your load balancer or ingress, with TLS 1.2 as the minimum and TLS 1.3 preferred.
- Use certificates from your cloud provider or your corporate PKI.
Paths to Expose
Your load balancer routes by path. Expose only what your callers need:
| Path | Expose to | Used by |
|---|---|---|
/{org}/{route} and the provider-native paths under it | Your applications | Gateway mode |
/mcp/{org}/{route} | Your agents and MCP clients | MCP protection |
/mcp-oauth/callback | The browsers of the people who sign in | OAuth sign-in for MCP servers |
/validate, /monitor/validate, /monitor/config | Your applications | In-app mode. Blocking them leaves in-app applications without a verdict |
/health | Your load balancer | Liveness probe |
| Control API | Your operators' network only, on a separate internal hostname | The console, from your team's browsers |
Block every other path at the load balancer, including administration and development paths. The hardening guide delivered with the deployment lists them.
The control API URL must use HTTPS on the default port, and ByteHide registers it on your organization so the console can reach it.
Load Balancer
- Configure the gateway to trust the
X-Forwarded-Forheader only when a trusted load balancer sits in front of it. Otherwise a forged header would bypass per-IP limits. - If browsers call the gateway directly, restrict CORS to your origins at the load balancer.
- Session affinity on the conversation ID improves the gateway's cache hit rate with several replicas.
Outbound Connections
| Destination | Required | If blocked |
|---|---|---|
| ByteHide classification service | Yes, in every deployment option | The gateway fails closed with 503 |
| Model providers you configured | In gateway mode only | No completion works |
| MCP servers you connected | If you use MCP routes | Those routes return transport errors |
| Identity providers of your MCP routes and MCP servers | If you use them | Callers or servers authenticated with those providers stop working |
| ByteHide platform | No | Console sign-in, alert notifications and SIEM forwarding stop. Inspection keeps working |
| IP reputation service | No | Incidences lose country, network and reputation |
MCP servers inside your network are reached over your internal network, not the internet; see Integrations for how internal addresses are handled. See Network Requirements for the traffic paths.
Secrets
Keep every secret in your cloud's secret manager (AWS Secrets Manager, Azure Key Vault or Google Secret Manager), never in plain environment variables or repositories:
| Secret | Purpose |
|---|---|
| Gateway keys | Authenticate callers on the data plane |
| Admin keys | Service credential for the control API, for automation only. Expose it on your private network only, and rotate it when staff with access changes |
| Classification service key | Issued by ByteHide for your gateway |
| Encryption key | Encrypts provider secrets at rest. Back it up separately. It rotates through a key ring |
| Hash pepper | Keeps hashed values from being reversed. It must stay stable: changing it breaks correlation with past hashes |
| Database and Redis connection strings | Data store access |
| Scheduler secret | Authenticates the scheduler on the internal paths |
Container Security Context
The gateway image runs as a non-root user (UID and GID 10001), with no compilers in the final image. It supports a read-only root filesystem when /tmp and its cache directory are writable, no-new-privileges, dropped capabilities and the default seccomp profile.
Identity
| Who | How they authenticate |
|---|---|
| Your team, in the console | ByteHide sign-in, with TOTP multi-factor authentication. Members can sign in with Microsoft Entra ID single sign-on over OpenID Connect, configured in your organization's security settings |
| Applications | Gateway key or consumer credentials |
| People calling MCP routes | Your own identity provider (Okta, Entra ID, Auth0...) over OAuth. See MCP Authentication |
Next Steps
On-Premises Deployment
Install options, steps and upgrades
Network Requirements
Traffic paths and outbound destinations
Architecture
Components, request paths and failure modes