Platform Engineer's Guide to AI Agent Infrastructure

Platform Engineer AI Agent Infrastructure: What You're Now Responsible For

Platform engineering teams used to own compute, networking, CI/CD pipelines, and observability. Now there's a new category on the list: platform engineer AI agent infrastructure. Agents are no longer research projects — they're hitting production, calling external APIs, reading private data, sending emails, and making financial decisions. The platform team is expected to make that work safely, repeatably, and at scale.

This guide is written for platform and infrastructure engineers who are being asked to build the foundation that lets AI agents operate in production. It covers the architectural components you need, the governance primitives that matter, where self-built solutions break down, and how to evaluate managed options without getting locked into enterprise sales cycles.

Why AI Agents Need Dedicated Platform Engineer Infrastructure

Traditional application infrastructure assumes a human is somewhere in the loop — clicking buttons, approving deployments, reviewing outputs. AI agents break that assumption. A well-designed agent can autonomously chain dozens of tool calls, authenticate against external services, exfiltrate data, and run up significant API costs before anyone notices something went wrong.

According to Gartner, by 2028, 33% of enterprise software applications will include agentic AI — up from less than 1% in 2024. That's a massive shift in what platform teams will be responsible for. The OWASP Top 10 for LLM Applications (2025 edition) specifically calls out excessive agency and insecure tool use as top risks for agentic systems.

The infrastructure problem breaks into three distinct concerns:

Enablement: Giving agents the tools they need to do real work — web access, email, B2B data, financial market feeds, third-party APIs
Governance: Enforcing what agents are allowed to do, with what credentials, at what rate, and with whose approval
Observability: Logging every action agents take in enough detail to audit, debug, and comply with regulations

Most platform teams are currently cobbling these three concerns together from separate tools. That works until it doesn't — usually at the worst possible moment.

The Core Components of Platform Engineer AI Agent Infrastructure

1. The Agent Control Plane

An agent control plane sits between your agents and the external world. Every tool call, API request, and credential lookup routes through it. The control plane is where you enforce policy — not in the agent's system prompt (which can be manipulated or ignored), not at the network layer (which is too coarse-grained), but at the operation level.

A robust control plane for AI agent infrastructure needs to handle:

Per-agent credential isolation (agents should never share API keys)
Operation-level rules (e.g., "Agent A can read email but not send", "Agent B can query financial data but not place trades")
Rate limiting and cost budgets per agent or per workflow
Approval workflows for high-risk actions
Immutable audit logs for every operation

Without a control plane, your governance lives in prompts. Prompt-based governance is not governance — it's a suggestion the model can ignore under the right conditions.

2. Credential and Identity Management for Non-Human Identities

Agents are non-human identities (NHIs). They need credentials to act, and those credentials represent a significant attack surface. A compromised agent credential can have the same blast radius as a compromised employee account — potentially worse, because agents operate at machine speed.

Platform engineers need to solve:

Credential issuance: How does each agent get its credentials? Hardcoded secrets in code are unacceptable. Per-agent secret injection at runtime is the minimum viable approach.
Credential rotation: API keys and OAuth tokens expire or get revoked. Agents need automatic rotation without downtime.
Credential scoping: Each agent should have the minimum permissions required for its specific task. An agent that summarizes emails doesn't need the ability to delete them.
OAuth connection management: Many services require OAuth flows that were designed for humans. Platform teams need to handle token refresh, scope management, and consent without blocking agent execution.

The non-human identity management problem is one of the most underestimated parts of agent infrastructure. Tools like Okta AI Agent Identity address this from an enterprise IAM angle, but they're designed for security teams navigating compliance requirements, not for platform engineers who need to ship agent infrastructure quickly. Handler takes a different approach: built-in credential management that's accessible via API key, MCP server, or CLI — no enterprise sales call required.

3. Tool Superpowers: What Agents Actually Need to Do Work

Governance without capability is just a firewall. Your agents need real tools to deliver real value. Platform teams are frequently asked to wire up agents to:

Web search and scraping
B2B data providers (company firmographics, contact data)
Email sending and reading
Financial market data
CRM and ticketing systems
Internal APIs and databases

Building and maintaining these integrations yourself is expensive. According to a 2024 survey by Postman, the average engineering team manages over 900 internal and external APIs. Keeping agent integrations current across that surface area — handling auth changes, rate limit policy changes, schema updates — is a significant ongoing maintenance burden.

The platform engineering challenge is to provide these integrations in a way that's governed from day one. An agent that can send email is useful. An agent that can send email only to approved domains, at most 50 per hour, with every message logged is production-ready.

4. Approval Workflows and Human-in-the-Loop Controls

Not every agent action should be fully autonomous. Platform teams need to define which operations require human approval before execution. This is especially critical for:

Financial transactions above a threshold
Sending external communications on behalf of the company
Modifying production databases or configurations
Accessing sensitive personal data

Approval workflows need to be asynchronous — an agent should be able to pause, wait for approval, and resume without losing context. Most existing workflow tools weren't designed for this pattern. Building it from scratch means implementing a queue, a notification system, an approval UI, and a state machine for the agent — a significant investment for something that's table stakes for safe agent deployment.

For a deeper look at implementation patterns, the guide on how to set up AI agent approval workflows covers the specifics of queue design and state management.

5. Observability and Audit Trails

When an agent does something unexpected in production, you need to know exactly what happened — which tool was called, with what parameters, what the result was, and what the agent did next. Standard application logging is insufficient because agents make decisions that span multiple tool calls over extended time periods.

Agent observability requires:

Per-operation logs with full request/response payloads
Trace IDs that span entire agent sessions
Tamper-evident storage (especially for compliance)
Queryable history for post-incident review

This overlaps significantly with compliance requirements. The EU AI Act's Article 12 requires logging for high-risk AI systems. SOC 2 Type II audits increasingly ask about AI system access logs. Building audit infrastructure that satisfies both operational and compliance needs from the start saves significant rework later.

Platform Engineer AI Agent Infrastructure: Build vs. Buy Comparison

The build-vs-buy decision for agent infrastructure is more nuanced than it is for most platform tooling. Here's an honest breakdown:

Component	Build It Yourself	Managed Platform (e.g., Handler)
Credential management	6-12 weeks; requires secrets management infra, rotation logic, per-agent scoping	Built-in; per-agent API keys with scoped permissions from day one
Tool integrations (web, email, B2B data, finance)	Ongoing maintenance across 200+ potential APIs; auth changes break things	Managed integrations; handler maintains auth and schema compatibility
Operation-level governance rules	Custom middleware; must be applied consistently across all agent frameworks	Centralized rule engine; works with any framework via MCP or API
Approval workflows	Queue + notification + state machine; 4-8 weeks to build correctly	Pre-built; configurable thresholds and approval routing
Audit logging	Custom log pipeline; tamper-evidence requires additional infrastructure	Immutable logs per operation; queryable and exportable
Rate limiting and cost control	Per-agent budgets require custom accounting; easy to miss edge cases	Built-in per-agent rate limits and spend caps
Framework compatibility	Must re-implement for each framework (LangChain, OpenAI Agents, Claude Code, etc.)	Works with any framework via MCP server or direct API
Time to first governed agent	3-6 months for a production-ready system	Minutes with an API key or MCP server configuration

The honest case for building: if your agent use case is sufficiently narrow and internal, a custom lightweight solution may fit better than a general-purpose platform. But most teams underestimate the ongoing maintenance cost — especially credential rotation and integration upkeep — and end up building more than they planned.

How Handler Fits Into Platform Engineer AI Agent Infrastructure

Handler is built for platform engineers who need to ship governed agents, not security analysts who need compliance dashboards. The design philosophy is enablement and governance as a unified layer — not two separate tools you have to integrate.

In practice, this means a platform engineer can:

Issue a scoped API key for a specific agent in seconds
Configure which tools that agent can use (web search, email, financial data, B2B data, 200+ connectable services)
Set operation-level rules (read-only email access, max 100 web requests per day, no external sends without approval)
Expose that governed toolset to any agent framework via the Handler MCP server or REST API
Get an immutable audit trail of every operation, automatically

This works with Claude Code, Cursor, OpenAI Agents SDK, LangChain, custom agent loops — anything that can make HTTP calls or connect to an MCP server. There's no lock-in to a specific framework or a specific LLM provider.

The DevOps guide to AI agent governance covers how this maps to existing infrastructure patterns like GitOps and policy-as-code — worth reading if your team is already managing infrastructure declaratively.

Pricing is transparent: the Basic plan is $30/month with a $30 included allowance — useful for teams evaluating whether a managed platform fits before committing to a larger rollout. Try Handler free and have a governed agent running in under an hour.

For teams comparing options, it's worth knowing where Handler differs from adjacent tools:

vs. Astrix Security / Oasis Security: These are security-first tools focused on NHI risk detection. Handler adds the enablement side — agents can't just be audited, they need working tools. See the Astrix Security alternative breakdown for a detailed comparison.
vs. DashClaw / AgentControl.dev: Both are open-source self-hosted control planes. Handler is managed SaaS — no infrastructure to run, zero-setup superpowers, and production-grade reliability without ops overhead.
vs. Speakeasy / Peta.io: MCP-focused governance tools. Handler covers MCP but also API keys, OAuth connections, and the full tool integration layer beyond MCP.

Operational Patterns for Production Agent Infrastructure

Treat Every Agent as a Service Account

The most reliable mental model for agent identity is: every agent is a service account with a defined role. That role has specific permissions, specific rate limits, and specific logging requirements. When you onboard a new agent, you should be filling out a form that looks a lot like your existing service account provisioning process.

Apply Least Privilege at the Operation Level

Network-level least privilege (firewall rules) is necessary but not sufficient. An agent that has network access to your email server can, absent operation-level controls, send any email to anyone. Operation-level governance means defining the specific actions an agent is allowed to perform — read vs. send, query vs. modify, specific endpoints vs. full API access.

Design for Failure and Escalation

Agents will hit situations they weren't designed for. Your infrastructure needs to handle this gracefully: pause execution, log the state, notify the right human, and resume when approved — without dropping data or corrupting state. This requires explicit design, not just retry logic.

Version Your Governance Rules Like Code

Agent governance rules should live in version control, go through code review, and be deployed with the same rigor as application code. A rule change that widens an agent's permissions is a security-relevant change. Treat it accordingly.

Build Cost Visibility From Day One

Runaway agents are a real cost risk. Per-agent spend tracking — not just aggregate LLM costs, but total cost including API calls, data purchases, and compute — needs to be built into your infrastructure from the start. Retrofitting cost attribution is painful. The rate limiting and cost control guide covers the technical implementation patterns in detail.

Frequently Asked Questions

What's the difference between an agent control plane and a standard API gateway?

An API gateway routes and rate-limits HTTP traffic. An agent control plane understands the semantic meaning of agent operations — it knows the difference between an agent reading an email and an agent sending one, and can apply different policies to each. API gateways operate at the transport layer; agent control planes operate at the application and intent layer. You need both, but they serve different purposes.

How should platform engineers handle credential rotation for long-running agents?

Credential rotation for agents requires two things: a mechanism to issue new credentials without restarting the agent, and a short enough credential lifetime that rotation happens frequently. For OAuth tokens, implement automatic refresh before expiry. For API keys, consider credential injection via a sidecar or secrets manager that agents poll on a defined interval. The goal is zero-downtime rotation — agents should never fail because a credential expired mid-task.

Which agent frameworks does managed infrastructure typically support?

Any framework that can make HTTP calls or connect to an MCP server can use a managed agent infrastructure layer. This includes LangChain, OpenAI Agents SDK, Claude Code, Cursor, AutoGen, CrewAI, and custom agent loops built on raw model APIs. Framework-agnostic infrastructure is a key requirement — avoid any governance layer that only works with one framework, as it becomes a migration risk.

How does agent governance relate to SOC 2 or EU AI Act compliance?

Both frameworks require demonstrating control over automated systems that handle sensitive data or make consequential decisions. SOC 2 Type II auditors increasingly ask for evidence of access controls, audit logs, and change management for AI systems. The EU AI Act requires logging, human oversight mechanisms, and documented risk assessments for high-risk AI. An agent governance layer that produces immutable per-operation audit logs and enforces human approval workflows gives you the artifacts both frameworks require.

When does it make sense to build agent infrastructure vs. using a managed platform?

Build when: your use case is narrow and unlikely to expand, you have strong existing infrastructure (secrets management, observability, access control) you can extend, and your team has bandwidth for ongoing maintenance. Buy when: you're building across multiple agent use cases, you need integrations to many external services, your team's time is better spent on the agent logic itself, or you need to be in production quickly. Most teams underestimate the ongoing maintenance cost of self-built solutions — especially credential rotation and integration upkeep as external APIs evolve.