AI GovOps · AI FinOps · AI Performance Engineering · AI Development

I make enterprise AI governed, cost-controlled and fast — in production.

I'm John D'Esposito, founder of Amalfi AI. For 20+ years I've built production infrastructure that has to be fast, observable and auditable — API gateways, cloud platforms, performance engineering. Now I apply that discipline to AI: agents, LLM traffic and the data platforms they run on.

Databricks Snowflake Amazon Bedrock LangGraph Kong AI Gateway Nginx / OpenResty Lua Claude Code Gemma & local LLMs MLflow
United Airlines

Ran the Kong API platform for mobile

Kong Gateway and Kong Mesh at airline scale, automated across every AWS region, with custom Lua plugins for authorization, request validation and routing — plus early Kong AI Gateway adoption.

Goldman Sachs

Apple Card cloud infrastructure

Multi-account, multi-region infrastructure for customer-facing banking transactions, with CIS-tested AMI pipelines.

Production AI

Multi-agent triage in production

A LangGraph system that classifies, validates and diagnoses operational tickets against live warehouse data. Case study ↓

Patents

3 patents in performance & load balancing

Real-time network and HTTP monitoring, server clustering for load balancing, and web-performance testing.

Featured Case Study

TriageBot: the agent proposes, deterministic code decides.

Client: a national healthcare payments company · Multi-agent AI · Snowflake · Amazon Bedrock

Operational tickets arrived through several service desks and issue trackers. Many were underspecified, routing was manual, and diagnosis meant senior engineers reconstructing system state by hand across the data warehouse.

I designed and built TriageBot, a multi-stage agent pipeline that became the single front door. It kicks back incomplete tickets before they consume engineering time, classifies the rest, and diagnoses root cause from live warehouse state — reconstructing what the data looked like when the problem occurred — then proposes a remediation for an engineer to approve.

The model never has the last word. It adjudicates over a closed registry of known failure modes, recommendations come from a deterministic rule table, and when the evidence isn't there the system abstains instead of guessing. Diagnosis that once meant an engineer reconstructing system state by hand now arrives as an evidence-backed brief, ready for review.

88–93%

Classification accuracy on live weekly holdouts — above the 84% rate at which human agents agreed with each other.

95%

Accuracy on high-confidence predictions — the ones safe to automate.

27→90%

Recommendation agreement, from early development to production holdouts.

0

Fine-tuning runs. It learns from human-verified precedent in retrieval, not from retraining.

The six-stage pipeline
  1. Hydrate

    Pull the ticket and its context from the source system.

    Code
  2. Classify

    Hybrid semantic + lexical retrieval against resolved precedent.

    Model proposes
  3. Validate

    Required fields checked; incomplete tickets kicked back.

    Code decides
  4. Diagnose

    Point-in-time warehouse queries, adjudicated against known failure modes.

    Model proposes
  5. Recommend

    Deterministic rule table; invariants stop contradictions.

    Code decides
  6. Act

    Write back to the tracker for a human to approve.

    Human approves

Evaluation before shipping

No prompt or code change ships without passing regression gates over frozen evaluation sets and blind weekly holdouts, traced end to end in MLflow.

Cost discipline by design

Frontier models only where judgement is needed; small open-weight models run off-peak for extraction and evaluation. Bounded queries per ticket — no runaway loops.

Auditable end to end

Every incoming event is kept in an immutable audit store for replay, and every decision is traceable to the evidence that produced it.

LangGraphAmazon BedrockOpenSearch ServerlessSnowflakeMLflow TracingKubernetes (EKS)TerraformOpen-weight SLMs
What I Help With

Four disciplines. One production standard.

Every engagement starts with a measurement you can act on, and ends with controls running in your own infrastructure. Consulting is delivered through Amalfi AI.

01 — Governance

AI GovOps

Governance as a continuous flow, not a quarterly checklist.
The problem
AI usage is growing faster than anyone can review it, and audit evidence is reconstructed after the fact.
The outcome
Risk classification, guardrails and audit evidence produced at the point of every AI interaction, mapped to ISO 42001, NIST AI RMF, the EU AI Act and HIPAA.
Typical first engagement
Governance baseline — where your AI traffic flows, what is exposed, and the controls that close the gap.
02 — Cost

AI FinOps

Know what every agent, team and model costs — then stop the waste.
The problem
AI spend arrives as one monthly bill with no owner, and agents can burn budget faster than anyone notices.
The outcome
Every AI call — LLM, MCP, agent-to-agent — metered, attributed and reconciled, with costs reduced in-line and budgets enforced where the spend happens.
Typical first engagement
AI spend baseline — your real spend per team, agent and model from live traffic, with no application changes.
03 — Performance

AI Performance Engineering

Fast, reliable AI in production — engineered, not hoped for.
The problem
Latency, rate limits and provider outages surface as user complaints instead of being designed for.
The outcome
Latency, throughput and resilience engineered into the AI path — down to the Nginx, OpenResty and Lua layer of the gateway — and held to SLOs with real telemetry, across every model and provider.
Typical first engagement
Performance baseline — SLOs, tail latency and failure modes measured, with the fixes ranked by impact.
04 — Development

AI Development

Agents that are accurate because they are constrained.
The problem
Prototypes impress in demos, then stall: no evaluation, no guardrails, no way to prove they are right.
The outcome
Production agents and LLM applications grounded in your data — on frontier, open-weight or local models — with deterministic guardrails, evaluation gates and tracing from day one.
Typical first engagement
Production pilot — one real workflow, an evaluation set built from your history, and a go/no-go backed by measured accuracy.
Where I Build

On the platforms you already run.

AI governance, cost and performance live in the data platform that holds the evidence, the gateway that carries the traffic, the observability stack that watches it — and the models themselves, hosted or local.

Databricks

Medallion architecture with Unity Catalog lineage from raw data to model prediction; MLflow for experiments, tracing and evaluation; Mosaic AI Gateway and Lakehouse Monitoring for governed model serving.

Unity CatalogMLflowMosaic AI GatewayDelta LakePySpark

Snowflake

Agents grounded in warehouse truth: guarded, read-only, point-in-time SQL that reconstructs system state, and query-history mining that turns undocumented expert practice into reusable playbooks.

Snowflake SQLData-grounded agentsForensics

Gateway & observability

Kong AI Gateway, Kong Mesh and the Nginx / OpenResty engine beneath them as the enforcement point for AI traffic — custom Lua plugins for authorization, prompt validation and routing, and the data plane tuned for latency. OpenTelemetry and Dynatrace put spend per call, latency per model and governance SLAs on one pane.

Kong AI GatewayKong MeshNginxOpenRestyLuaOpenTelemetryDynatraceAWSTerraform

Models & AI tooling

Model-agnostic by design. Frontier models — Anthropic Claude, OpenAI GPT, Google Gemini — through Amazon Bedrock and private endpoints; open-weight models such as Gemma and Qwen served locally with vLLM where cost, privacy or latency call for it. An AI gateway like Kong puts every model, hosted or local, behind the same governance, metering and SLOs. Day to day I build with Claude Code, custom MCP servers and multi-model review panels.

ClaudeClaude CodeOpenAIGeminiGemmaQwenvLLMAmazon BedrockMCP
How I Work

Measure. Engineer. Operate.

I work with a small number of enterprises at a time, close to the engineering team. No slide-deck strategies — controls, metering and agents that run in your infrastructure.

01

Measure

A short baseline from your live AI traffic and data: what it costs, how it performs, how accurate it is, and where the governance exposure sits.

Outcome — a baseline you can act on

02

Engineer

Guardrails, metering, performance policies and agents built as code, tested like software, and embedded in the platforms you already run.

Outcome — controls in the request path

03

Operate

SLOs, budgets, accuracy and audit evidence reviewed continuously as new models, agents and regulations arrive.

Outcome — an operating model, not a one-off project

Background

Twenty years of production engineering, now applied to AI.

  1. 2024 — present

    Amalfi AI

    Founder

    Consultancy operationalizing AI GovOps, AI FinOps and AI Performance Engineering; creator of PromptClassify.

  2. 2021 — 2024

    United Airlines

    API platform, MLOps & AI traffic management — mobile cloud transformation

    Moved mobile services to ECS Fargate, introduced Kong Gateway and Kong Mesh with full Terraform automation and custom Lua plugins, built OpenTelemetry tracing to Dynatrace, and introduced Kong AI Proxy for AI traffic.

  3. 2020 — 2021

    Nomi Health

    DevOps & automation architect

    AWS infrastructure automation across business units; GitHub Actions and serverless delivery.

  4. 2018 — 2020

    Goldman Sachs

    DevOps & automation architect — Apple Card

    Multi-account, multi-region AWS infrastructure for Apple Card transactions; AMI pipelines with CIS security testing; HashiCorp Vault and Consul.

  5. 2016 — 2018

    Liberty Mutual · GE Digital

    DevOps & automation architect

    Compliance as code, secrets management and configuration automation for mission-critical cloud systems.

Patents

  • Remote and real-time network and HTTP monitoring with a real-time predictive end-user satisfaction indicatorUS 8,972,569 B1 · issued 2015 · sole inventor
  • System and method for clustering servers for performance and load balancingUS 6,965,938 B1 · issued 2005 · co-inventor, IBM
  • Method and computer program for web site performance monitoring and testing by variable simultaneous angulation

Publications

  • Creating an Effective DevOps Strategy
  • Creating ChatOps & Incident Response Solutions

Education & certifications

  • Lafayette College — B.S., Electrical Engineering
  • Advanced Dynatrace Certification
  • Service Mesh Certification
Start the Conversation

Book a 30-minute AI operations review.

Bring one question — what your AI really costs, whether it would pass an audit, why it's slow, or whether an agent is ready for production. You'll leave with a clear next step, whether or not we work together.

or write directly: johndesp@amalfi.ai
Latest Posts

Latest Posts.

Short write-ups from production: one result, how it was measured, what it took.

Infographic — three ticket systems feed a five-stage triage pipeline (Classification, Validation, Preparation, Recommendation, Action); measured outcomes fan out on the right.

Every support ticket triaged in minutes, not hours or days

I built and run an AI triage agent in production for a national healthcare payments company. Tickets arrive from Zoho, Zendesk and Jira. Every one is read in the first minutes of its life.

Before: a ticket waits in the queue. Hours or days later an engineer picks it up, finds what is missing, and sends it back. A full round trip, lost.

After: within minutes of filing, a triage record is written beside the ticket. The requester is told exactly which fields are missing, in the request type's own words. The engineer who picks it up finds the briefing already written: linked records, live evidence, precedents.

Five stages, in order, every time: Classification, Validation, Preparation, Recommendation, Action. The LLM proposes at each stage. Deterministic code decides what happens next. No LLM stage holds a write tool.

What it measured in production:

  • 7,600 tickets a year, about 20 a day, each one triaged within minutes
  • Zero false send-backs

What it took:

  1. Gates in code between every proposal and every action
  2. A year of real tickets replayed through the production image before any change shipped
  3. A trace on every decision, so a wrong call is found the same day

The stack: LangGraph agents with an LLM supervisor; Claude on Amazon Bedrock for the proposals and an open-weight SLM for ground truth; a vector database for precedents; MLflow tracing; EKS.

To learn more, book 30 minutes: calendly.com/johndesp-amalfi

Infographic — a stream of AI requests enters a black box labelled "your AI spend"; measured at the request, four tagged outputs leave it: who (the team or the agent), what (the model and the tokens it billed), where (the application or environment), why (the task or the outcome). Caption — measured per request, your AI spend becomes a budget; read once a month, it stays a surprise.

You can't manage your AI spend without measuring it

AI FinOps starts with one measurement per request, taken while it runs.

Today most AI spend is a black box. Measure it at the request and four things come out of the box:

  • who: the team or the agent
  • what: the model and the tokens it billed
  • where: the application or environment it ran in
  • why: the task or the outcome

The invoice arrives once a month and says only a total.

What the provider billed is the only exact number; an estimate is not a measurement.

A dollar with no team, agent, place or purpose behind it cannot be budgeted, capped or forecast.

Measured per request, your AI spend becomes a budget; read once a month, it stays a surprise.

AI Financial Governance is the discipline. It is where I am spending 2027.

Infographic — "Chat vs agents: why the token bill grows with the square of the turns": what changed (a chat prompt vs an agent turn that re-sends prior context) and what 2027 asks of you (meter per turn, attribute every turn to an owner, reconcile against the invoice); a chart of tokens billed rising with turns.

Chat vs agents: why the token bill grows with the square of the turns.

In 2025 your AI bill scaled with prompts. In 2026 it scales with turns.

A chat prompt is one question and one answer: one line on the invoice.

An agent turn resends the whole conversation plus every tool output so far, then adds another. Turn 30 carries the weight of turns 1 to 29.

Simple arithmetic, with the same amount of new context on every turn:

  • turn 30 costs 30× the first turn
  • the 30-turn session costs 465× the first turn
  • thirty chat prompts would have cost 30×

That is why "cost per prompt" stopped being a useful number.

The enterprises that get 2027 right will govern AI spend the way the traffic actually behaves:

  1. Meter per turn, not per prompt. The invoice total hides the unit that matters.
  2. Attribute every turn to an owner. Which agent, which team, which decision spent it.
  3. Reconcile against what the provider actually billed, not against an estimate.

AI Financial Governance is the discipline. It is where I am spending 2027.

(Arithmetic, not a measurement: each turn adds the same new context and resends all of it. Real agents vary; the shape does not.)

Infographic — "Fragmented AI telemetry is exposing your organization": the four blind spots of AI governance beside a machine that unifies telemetry, and the medallion data lake solution with cost-effective compliance.

Fragmented AI Telemetry Is Exposing Your Organization

If you can't classify your AI risk, you can't manage it.

If you can't normalize your AI telemetry, you can't govern it.

If you can't query it under one schema, you can't prove compliance.

Here's what a Unified AI Governance Model actually looks like:

⚡ Enterprise AI data is fragmented across 4-6 disconnected channels — different formats, different destinations, zero unified visibility

🔓 Your BAA does not satisfy YOUR audit obligation — HIPAA §164.312(b) requires controls in YOUR systems, not your vendor's

🧩 Dumping everything into one store is not governance — without normalization, it's a bigger mess in one place

🛡️ Every prompt needs real-time classification — PHI detection, risk scoring, policy enforcement before the model sees it

🚦 Every AI request needs a gateway — routing, guarding, and telemetry capture across all providers including A2A traffic

🏗️ The AI Governance Golden Canonical Model normalizes fragmented telemetry into one governed schema — this is where multiple languages become one

📊 A Medallion Data Lake (Bronze → Silver → Gold) on Snowflake or Databricks gives you immutable ingestion, canonical normalization, and Role-Based Dynamic Data Masking

🔍 One query across all channels, all providers, 6-year retention, audit-ready — the infrastructure to answer any auditor question

The pattern:

Classify → Enforce → Capture → Normalize → Govern

An auditor asking "show me every AI interaction involving patient data in the last 90 days" expects an answer from YOUR infrastructure. Not a vendor dashboard.

Build the infrastructure to answer that question.

promptclassify.ai | amalfi.ai

Infographic — "If you can't classify AI risk, you can't manage it": the blind spot of unclassified prompts, a real-time classification pipeline (ingest, classify, score, route) and the governed output with risk score, audit record and policy decision.

If you can't classify your AI risk, you can't manage it.

Every management principle we've ever learned starts here. You can't manage what you can't measure. You can't govern what you can't see. You can't prove what you don't record.

And right now, most enterprises can't classify the risk in a single AI prompt.

—

🔍 Think about what that means.

Thousands of prompts per day flowing to AI providers. Some contain PHI. Some contain PII. Some contain trade secrets. Some are clean.

You don't know which is which — because nobody is classifying them.

—

📋 Your AI governance policy says "don't put sensitive data in prompts."

Great. How do you enforce that? How do you detect when someone does? How do you prove to an auditor that you had controls in place?

You can't — unless you classify.

—

⚠️ Classification is the foundation. Without it:

❌ You can't risk-score an interaction

❌ You can't enforce a policy decision — pass, warn, or block

❌ You can't generate a compliance-grade audit record

❌ You can't prove to a regulator that you govern AI interactions

❌ You can't detect PHI exposure before it reaches the model

✅ With it, everything else becomes possible.

—

🚀 That's why we built PromptClassify.

📦 A single container deployed in your environment

⚡ Sub-200ms classification on every prompt

🏥 Real-time PHI and PII detection using your organization's identifiers — not generic patterns

📊 A 0-100 risk score with a policy decision and human-readable reasoning

📝 An immutable audit record tagged to HIPAA, ISO 42001, NIST AI RMF, and OWASP LLM Top 10

🔐 Your data never leaves your perimeter.

🔄 The Regulatory Intelligence feed updates daily — so your rules evolve with the regulatory landscape, not six months behind it.

—

💡 Classification isn't a feature. It's the prerequisite for everything else in AI governance.

If you can't classify it, you can't manage it.

🌐 promptclassify.ai

Built by Amalfi AI.

Infographic — "Your BAA does not do what you think it does": what the BAA covers beside what it does not — vendor compliance vs your responsibility, HIPAA mandates local controls, the auditor's test, compliance APIs and webhooks.

Your BAA Is Not Enough — Why You Need Local Controls

Having a BAA with your AI vendor doesn't mean you're HIPAA-compliant.

It means your vendor is accountable for their environment. It says nothing about what you can prove from yours.

🏛️ HIPAA §164.312(b) doesn't care about your vendor's dashboard.

The law requires YOUR organization to implement mechanisms that record and examine activity in systems containing ePHI. That obligation doesn't transfer when you sign a BAA — it stays with you.

🔍 The Auditor Test: "Show Me"

If an auditor asks for 90 days of AI interactions involving PHI, pointing to a vendor portal you don't control is not a valid answer. You need those logs in YOUR infrastructure, under YOUR retention policies, queryable on YOUR timeline.

⚙️ The Fix: Activate Compliance APIs and Webhooks

Most enterprise AI platforms already expose the pipes — compliance APIs, webhook endpoints, audit log exports. The gap isn't technology. It's activation. Pull prompts, responses, and timestamps into your own infrastructure to close the audit gap.

📐 The bottom line:

A BAA protects you from vendor mishandling. It does NOT satisfy your independent regulatory requirement to record and examine AI activity.

These are two different obligations. Treat them that way.

If you're deploying AI in a healthcare environment and your compliance strategy starts and ends with "we have a BAA," it's time to revisit your local controls.

Infographic — "PromptClassify: secure enterprise AI governance architecture": enterprise integration points flowing into a customer-controlled VPC with illuminated data pathways and real-time risk intelligence.

Introducing "Prompt Classification as a Service"

Every unclassified prompt is a blind spot — unknown risk, unknown regulatory exposure, unknown data leakage. Multiply that by thousands of prompts per day across your organization and you have an audit trail full of question marks.

Most enterprises aren’t classifying prompts. Not because they don’t care — because there hasn’t been a practical way to do it at scale without building and maintaining the classification logic yourself.

That’s the problem PromptClassify solves.

📦 A Docker container that runs in your environment — your VPC, your cloud, your perimeter. Your data never leaves.

🔄 A daily Regulatory Intelligence feed that keeps classification rules current across ISO 42001, HIPAA, EU AI Act, NIST AI RMF, and OWASP LLM Top 10 — so you don’t have to track every regulatory change yourself.

⚠️ A 0-100 risk score for every prompt — Pass, Warn, or Block. Your systems decide what to do with it.

🔌 Integrates with Kong AI Gateway, Solo.io Agent Gateway, Azure API Management, Google Cloud API Gateway, Kagent, or any system that makes an HTTP call. Custom agents can call it directly.

PromptClassify doesn’t replace your governance program. It gives your governance program a way to touch every AI interaction — before the call, after the call, against your historical logs, or as data flows into your observability lake.

Classify the prompt. Score the risk. Route the logs wherever you want. Focus on what matters.

Visit 🌐 promptclassify.ai

Built by Amalfi AI.

Infographic — "AI Gov Ops: closing the enterprise operationalization gap": the problem of static policy governance on the left and the solution of operational controls with real-time policy propagation on the right.

As we scale Enterprise AI, a new discipline is emerging: AI Gov Ops.

Just as DevOps revolutionized software delivery and MLOps streamlined the model lifecycle, AI Gov Ops is here to bridge the gap between adopting frameworks (ISO 42001/NIST) and actual production enforcement.

Here is a quick breakdown of how we are closing the gap.

Infographic — "The missing discipline: AI Gov Ops": a continuous control loop replacing traditional compliance, with a compliance shift matrix from paper binders and annual audits to declarative code and real-time observability.

The Missing Discipline: AI Gov Ops

There's a discipline missing from enterprise AI.

⚙️ DevOps to ship software

🛡️ SecOps to defend it

🧠 MLOps to manage models

But nobody's operationalized governance.

Most organizations are still governing AI with paper binders, annual audits, siloed teams, and reactionary checklists. Meanwhile their infrastructure is processing hundreds of LLM calls per minute.

Static policy can't govern dynamic systems.

AI GovOps is the practice of embedding governance controls directly into the operational lifecycle:

📄 → 💻 Paper Binders → Declarative Code

📅 → 📡 Annual Audits → Real-Time Observability

🏢 → 🔗 Siloed IT & Legal → Integrated Workflows

✅ → 🎛️ Reactionary Checklists → Continuous Control Plane

It's not a product. It's a discipline — and every enterprise deploying AI in production needs it.

⚠️ If your AI governance strategy is a PDF in SharePoint, you don't have a strategy. You have a liability.

Infographic — "AI Gov Ops: automating AI governance from policy to production": before and after the operationalization gap (DevOps to software delivery, MLOps to model lifecycle, AI Gov Ops to AI governance) and the four pillars — centralized control planes, gateway-layer guardrails, audit-ready logging, real-time policy propagation.

AI Gov Ops: Automating AI governance from policy to production.

Your AI governance framework is probably a PDF.

You're governing the most powerful technology your enterprise has ever deployed... with a document that nobody reads, nobody enforces, and nobody can audit at runtime.

I've spent the last two years deep in enterprise AI governance across healthcare, financial services, and airlines. And I keep seeing the same pattern:

✅ Framework: adopted

✅ Committee: formed

✅ Policy: published

❌ Actual enforcement in production: nonexistent

This is the operationalization gap. And it's massive.

We have ISO 42001. We have NIST AI RMF. We have EU AI Act compliance checklists longer than most codebases. What we don't have is the connective tissue between a governance framework and a running system.

AI Gov Ops is that connective tissue.

It's the discipline of embedding governance controls directly into AI infrastructure — not as afterthought policies, but as enforceable, observable, auditable runtime behaviors.

Think of it this way:

→ DevOps operationalized software delivery

→ MLOps operationalized model lifecycle

→ AI Gov Ops operationalizes AI governance

In practice, this means:

  • Governance logic lives in centralized control planes — not scattered across application code
  • Prompt injection detection, PII filtering, and toxicity guardrails execute at the gateway layer
  • Every LLM interaction is logged, traceable, and audit-ready by default
  • Policy changes propagate across every model and endpoint in minutes, not quarters

The organizations that get this right won't just be compliant. They'll move faster — because governance becomes an accelerator, not a bottleneck.

If your governance can't be observed, enforced, and audited at runtime, it's not governance.

It's theater.

The era of PDF governance is over. AI Gov Ops is what comes next.

Infographic — "Kong AI Gateway: centralized OWASP LLM Top 10 security enforcement": consumers on the left, the gateway as a zero-trust chokepoint in the centre mapping each OWASP LLM risk to a control, foundational LLM providers on the right.

Meeting OWASP Top 10 for LLM Applications? Your AI Gateway Is Your First Line of Defense.

Security teams are getting handed a new mandate: secure every LLM in the enterprise. The OWASP Top 10 for Large Language Model Applications gives you the threat framework. But where do you actually enforce it?

Not in application code. Not with prompt engineering best practices documents that developers may or may not follow. You enforce it at the infrastructure layer — the same way you enforce network security at the firewall, not inside each application.

Here's how Kong AI Gateway maps its capabilities directly to the OWASP LLM threat categories:

🛡️ Prompt Guarding → OWASP LLM01: Prompt Injection Every prompt passes through the gateway before reaching any model. Kong's AI Prompt Guard plugin detects and blocks injection attacks, jailbreak attempts, and adversarial payloads at the perimeter. This is your WAF equivalent for LLMs — uniform protection across every model endpoint, regardless of which team or vendor deployed it.

🏷️ Prompt Classification → OWASP LLM06: Sensitive Information Disclosure Prompts carrying PII, PHI, credentials, or proprietary data get classified and flagged before they ever leave your environment. For SOC teams, this means data exfiltration via LLM becomes a controllable, monitorable risk — not an unknown one.

📊 Logging & Observability → OWASP LLM05: Supply Chain Vulnerabilities & LLM09: Overreliance Full request/response capture at the gateway creates an immutable audit trail for every model interaction across the enterprise. Which user, which prompt, which model, what response, when. This is the forensic evidence your incident response team needs, and the continuous monitoring your compliance program requires. No more piecing together logs from 15 different application teams after a breach.

⚙️ Prompt Transformation → OWASP LLM07: Insecure Plugin Design & LLM08: Excessive Agency Inject system-level security constraints, enforce role boundaries, restrict output scope, and strip dangerous tool-calling instructions — all at the gateway before the model processes anything. One policy update propagates across every AI application instantly. Your red team tests it once, and it's enforced everywhere.

🔐 The cybersecurity principle here isn't new. Defense in depth. Centralized policy enforcement. Least privilege. Zero trust. We've applied these patterns to networks, APIs, and cloud infrastructure for years. The AI layer shouldn't be any different.

⚠️ If your current LLM security strategy depends on every development team implementing OWASP controls correctly inside their own applications — that's not a security architecture. That's a hope-based strategy.

Infographic — "Risk-aware LLM routing with Kong AI Gateway": LLM consumers on the left, a PHI/PII classifier plugin and a risk router plugin in the gateway, a public route for sanitized data and a default route for PHI to the model providers on the right.

Solving the enterprise LLM privacy gap with automated AI routing

📈 Adopting Generative AI in an enterprise environment brings fantastic efficiencies, but it also introduces a massive security bottleneck:

Accidental data leakage of PHI (Protected Health Information) and PII (Personally Identifiable Information).

🛑 You can’t just rely on end-user training to sanitize every single prompt.

🛡️ The only scalable solution is an architectural control at the AI gateway layer.

🗺️ This diagram shows a robust approach: Risk-Aware LLM Routing built on Kong AI Gateway.

✅ The flow ensures compliance happens before the request reaches the model endpoint.

⚙️ The core mechanics depend on two key capabilities:

🔍 1️⃣ PHI/PII Risk Classification: The Kong PHI-PII-Classifier Plugin inspects inbound traffic from consumers like Claude Code, N8N, and internal Enterprise Agents, automatically flagging sensitive data.

✨ 2️⃣ Risk Routing: This is the magic. The Kong AI Risk Router Plugin takes the classification results and makes a real-time decision.

Depending on the risk profile of the data, the request is directed along different paths:

🔄 Public Route (Sanitized PII Data): Clean, safe data is sent to general model endpoints.

🚫 Default Route (PHI Data): Sensitive data can be dynamically blocked, obfuscated, heavily logged, or routed only to highly secure, private model instances (like compliant AWS Bedrock deployments) to ensure regulatory compliance.

💬 How is your organization enforcing data compliance policies across your LLM integrations? Drop a comment below. 👇

Infographic — "Competitive analysis: Snowflake vs Datadog for AI observability": a technical comparison matrix of a permanent audit record in a data lake against ephemeral performance monitoring, across cost, data retention, healthcare compliance and AI context.

Why You Should Consider a Private Data Lake vs. Traditional Logging & OTEL for AI

Most teams treat AI observability like traditional microservices: they pump every trace and log into an APM tool (like Datadog).

🛑 But that "traditional" approach is quickly becoming a massive liability.

As AI scales, especially in regulated industries like Healthcare or Finance, the shift toward a Private AI Data Lake (using Snowflake or similar) isn't just an architecture choice; it’s a business necessity.

Here’s why:

1️⃣ The Death of the "Indexing Tax" 💸

Traditional observability tools charge you to ingest and index data. When you’re dealing with high-volume LLM prompts and responses, you’re looking at TBs of data per day.

💰 The Problem:

In an APM, long-term retention is prohibitively expensive (often exceeding $1M/year for heavy workloads).

⚖️ The Data Lake Advantage:

Storage is decoupled from compute. You pay commodity rates ($23–$40/TB/mo) for a permanent record. You only pay for compute when you actually run an audit or a query.

2️⃣ Native Governance vs. Manual Redaction 🔐

In the world of HIPAA, PHI, and ISO-42001, "best effort" privacy isn't enough.

🕵️‍♀️ Traditional OTEL:

Requires complex, manual middleware to redact PII before it leaves your environment. If one prompt leaks, it's indexed and stuck in a third-party proprietary cloud.

🛡️ Data Lake:

Offers native, cell-level Role-Based Access Control (RBAC) and automatic masking. Your data stays in your governed environment, meeting strict audit cycles without third-party "black box" risks.

3️⃣ "Auditability" is the New "Availability" 🕰️

Traditional logging is designed for latency and error tracking. That’s great for knowing if a service is down, but AI requires knowing why a specific model reached a specific conclusion six months ago.

🔗 Prompt-to-Response Lineage:

A data lake allows you to use native intelligence (like Snowflake Cortex) to perform natural language auditing on your entire history. You get end-to-end lineage that legacy DQL (DataDog Query Language) simply wasn't built to handle.

👉 The Verdict

If you need to know if your API is up right now, use your APM.

⚖️ But if you need a permanent, cost-effective, and compliant record of every decision your AI makes, it’s time to move toward a private Data Lake.

Are you seeing your observability costs skyrocket as you scale AI?

How are you handling compliance data? 👇

Infographic — "The secret to sustainable AI scaling: governing A2A traffic": the agent-to-agent wild west on the left against a centralized AI gateway solution with route enforcement, logging, security and traffic guardrails on the right.

The Hidden AI Scaling and Observability Bottleneck: A2A Traffic

We talk a lot about User-to-Application (U2A) security, but there’s a quiet explosion happening behind the scenes: A2A (Application-to-Application) traffic.

As we integrate more LLMs, third-party APIs, and microservices into our tech stacks, the complexity of managing these connections is reaching a breaking point. 📈

The Problem: "The Wild West" of Connections 🌵

Without a centralized strategy, A2A traffic often becomes a blind spot. Organizations are facing:

⚠️ Security Gaps: Hardcoded credentials and unencrypted internal "chatter."

🙈 Zero Visibility: No clear audit trail of which service is calling which AI model. 💸 Cost Overruns: Unmonitored API calls to LLMs leading to "sticker shock."

The Solution: The AI Gateway 🛡️

We need more than just a proxy; we need a centralized AI Gateway to act as the brain of our infrastructure. Think of it as the "Air Traffic Control" for your data.

By routing A2A traffic through an AI Gateway, you gain:

1️⃣ Centralized Management: One place to govern all internal and external AI interactions.

2️⃣ Robust Security: Automated credential rotation and prompt injection filtering at the source.

3️⃣ Real-Time Auditing: Full traceability for compliance and performance monitoring.

4️⃣ Traffic Guardrails: Rate limiting and cost-capping to ensure AI usage scales sustainably.

The future of enterprise AI isn't just about the models we use—it's about how we manage the flow between them.

Is your organization centralizing A2A management, or are you still in the "Wild West" phase? Let’s discuss in the comments. 👇

Infographic — "Centralized AI governance: Kong AI Gateway as the control plane": consumers (IDE, CLI, apps, agents) on the left, the gateway with governance, security, rate limiting and PII scrubbing in the centre, LLM endpoints and data sources on the right.

Stop the AI Shadow IT: Why Your Enterprise Needs an AI Gateway

The biggest risk to your AI strategy isn't a "hallucination"—it's a lack of governance.

As organizations scale from simple chatbots to complex Agentic Workflows and Model Context Protocol (MCP) integrations, they hit a wall. Developers are pulling API keys for Anthropic, Gemini, and Perplexity. Data Science teams are spinning up endpoints on AWS Bedrock and Databricks Mosaic.

Without a central "Command Center," you have no audit trail, no cost control, and zero visibility into what’s actually being sent to these models.

🏛️ The Architecture of Governance: Kong AI Gateway

To move safely, you need a unified control plane. I envision the Kong AI Gateway as the single point of entry and exit for every AI interaction in the enterprise.

How it works:

⬅️ The Consumers (Left): Every internal application, AI Agent, and MCP client must pass through the gateway.

🚦 The Core (Center): This is where AI Governance lives.

Centralized Auditability: Log every prompt and response (with PII scrubbing) for compliance.

Policy Enforcement: Apply global rate limiting and cost quotas across all models.

Standardized Security: One place to manage secrets for Bedrock, Databricks, and public LLMs.

➡️ The Sources (Right): Your specialized MCP servers, autonomous agents, and model providers (Gemini, Anthropic, etc.).

🛡️ Why "Auditability" is the New "Agility"

If you can’t see it, you can’t secure it. By centralizing traffic, you’re not just managing APIs; you’re protecting your company’s intellectual property and ensuring that your AI agents aren't "going rogue" in a vacuum.

Governance isn't a bottleneck—it’s the fuel that allows you to deploy AI into production with confidence.

Infographic — "Risk-aware LLM routing (via Kong)", drawn as a notebook sketch: LLM consumers, a classifier plugin and a risk router in the Kong AI Gateway, public and default routes to the model providers.

Risk Aware LLM Routing: Turning "No" into "Safe"

Enterprise AI adoption often hits a wall because of one word: Risk. How do you let your teams use Claude Code, N8N, or Custom Agents without worrying that PHI, PII, or HIPAA data is leaking into public model training sets?

At Amalfi AI, we’ve moved past manual policies and "hopes and prayers." We’ve architected a real-time, risk-aware control plane using the Kong AI Gateway and a dual-plugin strategy.

The Two-Step Guardrail:

1️⃣ The Classifier Plugin: Every prompt is intercepted at the gateway. It identifies sensitive data types—from Social Security numbers to protected health information—before the request ever leaves your network.

2️⃣ The Risk Router Plugin: Based on that classification, Kong makes an executive decision.

Clean Data?

It takes the "Public Route" to models like Anthropic, Gemini or Perplexity.

Sensitive/HIPAA Data?

It is automatically forced through the "Default Route" to BAA-compliant environments like AWS Bedrock.

The Result?

You don't have to ban AI to stay compliant. By turning your API Gateway into a Security Control Plane, you provide the guardrails that allow your developers to move fast without the data exfiltration nightmares.

If you’re struggling to balance AI innovation with ISO 42001 or HIPAA requirements, let’s talk about operationalizing your AI Governance.

Infographic — "Unified AI governance: the north-south and east-west control plane": the AI gateway for ingress and egress above a service mesh for internal agent-to-agent traffic, with the key metrics each governs.

The Architectural Synergy: North-South meets East-West

In the rush to deploy GenAI, many organizations treat "The Gateway" as a simple proxy. But in a mature enterprise architecture, AI traffic management is a coordinate system. If you aren't managing both axes, you don't have control—you just have a connection.

🌐 North-South: The Centralized Egress Control Plane (Kong AI Gateway)

Kong is the Corporate Boundary. It is the hardened "Front Door" and "Exit Gate" where the business of AI meets infrastructure.

It provides the heavy-duty operational guardrails required for production:

⚙️Centralized Egress & Auditing: Every request leaving the corporate network is intercepted, inspected, and logged. No "Shadow AI" or rogue API keys.

Risk-Based Routing: Intelligently directing traffic to different LLM providers based on cost, latency, or compliance requirements.

⚙️Performance Engineering: Utilizing semantic caching and advanced load balancing to ensure the lowest possible latency for high-demand apps.

Cost Control & Quotas: Implementing strict token-based rate limiting and cost-tracking to prevent runaway cloud spend.

⚙️Edge Security: Standardizing OIDC, mTLS, and WAF protection across all AI endpoints in a single location.

🏠 East-West: The Internal Lifecycle Layer (Databricks Mosaic AI)

While Kong protects the boundary, Mosaic AI governs the Internal Nervous System.

This is where the "Science" happens—closely coupled with your data and model performance:

⚙️MLflow Integration: Deep lineage tracking of model versions, experiments, and deployments from within the Lakehouse.

⚙️Inference Tables: Automatically capturing every transaction into Delta Tables for audit, evaluation, and future fine-tuning.

⚙️Data Integrity: Ensuring RAG applications pull strictly from Unity Catalog-governed data sources.

⚙️Safety Guardrails: Real-time monitoring of model outputs for toxicity, bias, or PII leakage before it leaves the internal environment.

💡 The Architecture Takeaway

⚙️Use Kong AI Gateway to govern Egress, Consumption, and Cost. * Use Mosaic AI to govern Model Lifecycle and Data Lineage.

⚙️When you combine them, you aren't just "running AI"—you're architecting a governed, scalable, and audit-ready AI Data Factory.

Infographic — "Solving the ZDR compliance paradox: from the Splunk trap to the security data lake": the legacy SIEM funnel on the left against an immutable security data lake on the right, with a verdict to build your own audit trail.

Claude Enterprise - Zero Data Retention (ZDR) is a security win, but a compliance nightmare.

I’ve been spending a lot of time recently on the architecture for Claude Enterprise deployments.

The most common question from CISOs?

"How do we enable Zero Data Retention without losing our audit trail?"

In 2026, the answer isn't a traditional SIEM like Splunk. It’s a Security Data Lake.

Here is why:

When you flip the ZDR switch in Anthropic, your data is processed in RAM and deleted immediately. To stay compliant, you must use the Compliance API to "catch" the data in real-time.

The Splunk Trap:

If you pump every raw JSON interaction from 5,000 employees into Splunk, your licensing costs will skyrocket faster than your ROI.

The Data Lake Advantage:

Immutable Vault:

Store raw logs in Databricks/Snowflake with Object Lock. It’s cheaper and technically "uneditable"—the gold standard for auditors.

Context Re-Hydration:

With Claude’s 1M token window, you can feed a year of "Governance logs" back into the model to find policy drift. Good luck doing that with a 30-day Splunk index.

Schema-on-Read:

AI logs are messy. Don't waste time structuring them at ingest. Dump them into the lake and query them with SQL when the regulator knocks.

The Real Take:

Security is about keeping data out of the wrong hands. Governance is about keeping data in the right ones. If you're building an AI stack today, build your lake before you turn on your agents.

Infographic — "The technical specification for factory-ready data": three interlocking gears — identity-aware, contextually sanitized, audit-traceable — across source state, transport state and governance state.

Why "Raw Data" is a Liability for GenAI.

Most companies are rushing to feed their data into LLMs, but they’re missing a critical step: The Technical Specification for ‘Factory Ready’ Data.

If your data isn't governed at the source, in transit, and at the point of consumption, you aren't building an AI solution—you're building a compliance nightmare.

At Amalfi AI, we look at data through three distinct lenses to ensure it is secure, compliant, and performant:

  1. The Source State: Beyond Flat Files

Data can no longer be "flat." It must be identity-aware and tied to Role-Based Access Control (RBAC). Your system needs to know, in real-time, the difference between a Finance query and a Healthcare query to ensure the right people see the right things.

  1. The Transport State: Contextual Sanitization

As data moves through the Model Context Protocol (MCP), it must be stripped of "Noise" and "Lethal PII". By removing non-essential fields before tokenization, we don't just improve security—we reduce latency and costs.

  1. The Governance State: Audit-Traceability

This is where the rubber meets the road for ISO 42001. Every data point used in a prompt must have a lineage mapping back to established controls. If you can't prove who saw what version of a data point, you aren't audit-ready.

The Goal: Moving from "Experimental AI" to "Industrial-Grade AI" requires a factory mindset.

Are you currently mapping your data lineage back to ISO controls, or are you still in the "experimental" phase? Let’s talk in the comments. 👇

Infographic — "Why wrapping an API with MCP doesn't work: from API wrappers to intentional tools": the legacy side with a rigid API contract against the agentic side where tools are designed around intent.

Why "Wrapping" an API with an MCP Server Doesn’t Work

The hype around Model Context Protocol (MCP) is real, but a fundamental misunderstanding is brewing. Many teams think they can just "wrap" their existing REST APIs with an MCP server and call it a day.

That’s a mistake. Here’s why:

  1. A Tool is NOT a Data Pipe

An API contract is for a developer. It’s rigid: "Provide exactly these parameters in this format, or I will fail."

An MCP Tool is a service contract for the Agent’s intention. It is the inverse of an API contract. Instead of the developer defining a static path, the Tool defines a capability boundary, letting the LLM navigate it based on real-time context.

  1. Intention vs. Execution

When you "just wrap," you pass through too much noise.

APIs care about CRUD and state management.

MCP Tools care about outcomes.

If your MCP server is a 1:1 mirror of your endpoints, your Agent will spend more tokens managing "over-fetching" and schema validation than actually solving the user’s problem.

  1. The "Semantic" Translation

An MCP server isn't a bridge; it’s a translator. It turns the messy, probabilistic reasoning of an LLM into the deterministic execution of a server. To work, you have to re-architect the interface to be "LLM-native."

Infographic — "Optimizing LLM performance and cost: the semantic caching layer": the request flow from client through a semantic cache and the AI gateway to the LLM provider, with the cache hit path short-circuiting the call.

How to optimize LLM performance and costs through a semantic caching layer.

If you’re moving GenAI from a "cool demo" to a production service, you’ve likely hit the Production Wall:

The Latency Tax: Users won't wait 5+ seconds for a response.

The Redundancy Tax: Paying for the same tokens repeatedly because your cache only understands "exact matches".

Semantic Caching is the "Adult in the Room" strategy.

By moving beyond rigid string matching and utilizing vector similarity, the AI Gateway understands the intent of a query. If a similar question has been answered before, we serve it in <50ms from a local vector DB instead of waiting on a remote LLM API.

Why this is critical for 2026:

⚡ Performance Delta: Slashing response times from ~5,000ms cold starts to sub-50ms retrieval.

📉 Granular Cost Attribution: Tracking spend not just by model, but by specific Agent ID and Model Type to eliminate Shadow AI.

🛡️ Governance-First: The Gateway acts as a Nexus, ensuring every interaction is logged and audited for compliance—a core requirement for ISO 42001 standards.

The goal of great AI engineering isn't just about using the biggest model; it’s about building the smartest infrastructure to govern it.

Are you still letting redundant tokens eat your margins, or are you implementing an intercept layer? 👇

Infographic — "Eliminating shadow AI in engineering": a developer CLI routed through an AI gateway intercept that fans out to audit and compliance, financial cost attribution and secure LLM provider access.

Stop guessing. Start governing. Eliminating 'Shadow AI' starts at the Gateway.

Engineering speed shouldn't come at the cost of enterprise visibility. "Shadow AI" creates a massive blind spot—untracked costs, unaudited data leaks, and zero accountability for generated code.

At Amalfi AI, we solve this by moving governance to the infrastructure layer.

By intercepting traffic at the AI Gateway, we provide engineering leaders with a "Clinical Control Plane" that offers:

Total Transparency: Every interaction is logged and audited in real-time.

Precision Attribution: Granular cost tracking back to specific developers or teams.

Risk-Aware Routing: Ensuring LLM traffic is governed before it ever reaches the provider.

Stop letting AI run in the dark. Build a backbone of trust into your engineering workflow.

Infographic — "The OpenClaw threat map: architectural breach anatomy": a compromised autonomous bot pulling credentials through a one-click breach path, with the high-velocity threats and the verdict on the lethal trifecta.

The rise and risk of OpenClaw and Autonomous Bots

We’ve officially moved past "Chatbots" to "Autonomous Agents." But in 2026, giving an AI shell access changes the game from productivity to "Host Acquisition." 🔓

Unlike standard LLMs, agents like OpenClaw have "hands and eyes" on your OS. My latest architectural threat map (see below) breaks down why this is the new "fiduciary imperative" for every board of directors:

1-Click "God Mode": CVE-2026-25253 proves a single malicious link can steal authentication tokens in milliseconds.

The Poisoned Context: Indirect Prompt Injection allows a "hidden" command in an email or PDF to force your agent to exfiltrate SSH keys without you ever clicking a button.

Token Burn & Financial Risk: A simple logic loop can drain thousands in API credits while you sleep.