Field guide / Internal AI architecture
Internal AI systems connect company context to controlled action.
An internal AI system gives a model current company context, a bounded job, permitted tools, clear limits, and a feedback loop tied to real outcomes.
Working model
The internal AI system stack
Clear layers make the system easier to secure, evaluate, and change.
- 01 / Knowledge
What is true
Product facts, policy, definitions, playbooks, and ownership.
- 02 / Context
What is happening
CRM, usage, conversations, tickets, timing, and history.
- 03 / Agent
What to decide
A bounded role, objective, inputs, tools, and output contract.
- 04 / Control
What is allowed
Identity, permissions, policy, approvals, limits, and escalation.
- 05 / Evaluation
What worked
Quality, failures, human edits, adoption, cost, and outcome.
- 06 / Feedback
What changes
Update knowledge, rules, prompts, tools, and operating design.
01
Start by drawing the system boundary.
Name the users, business decision, data sources, actions, and accountable owner. Then mark what the system cannot access or do. Drawing the boundary exposes real interfaces and risks before anyone invents an agent persona.
For an expansion system, the boundary might include product usage, customer conversations, account hierarchy, entitlements, success plans, and approved offer rules. It might prepare an opportunity brief but never change pricing or contact the customer without an account owner.
02
Separate durable knowledge from live context.
Product knowledge changes, but it has a different lifecycle from customer activity. Policies, capability definitions, positioning, and ownership need curated sources and named maintainers. Usage events, calls, tickets, and CRM records need recency, identity resolution, and access controls.
Retrieval should be task-specific. Giving an agent access to every document and record creates noise and expands the security surface. Assemble a small context package for the decision, keep source references where they help verification, and expose missing evidence instead of asking the model to fill gaps.
03
Give every agent a contract, not a personality.
A useful agent definition states its job, trigger, required inputs, permitted tools, expected output, confidence rule, escalation path, and owner. Personality can shape tone, but it cannot replace an operational contract.
Before an agent reaches production, answer:
- Which facts must be present before it runs?
- Which systems can it read and which can it change?
- Which claims or actions require deterministic checks?
- When does it stop and ask a person?
- How can an operator reproduce and inspect its decision?
04
Controls and evaluation are part of the product.
Apply permissions at retrieval, output, and tool execution, based on the user and task. High-impact actions need deterministic policy checks and a durable approval record.
Evaluation begins before launch with representative cases and continues in production with sampled review, explicit feedback, failure monitoring, and outcome measures. Keep a small reference set of important cases, including exceptions. When knowledge, prompts, models, or tools change, rerun that set before increasing exposure.
05
Feed outcomes back to the right layer.
Not every failure is a prompt problem. A wrong answer may come from stale product knowledge, a broken identity match, a missing CRM field, an unclear policy, a tool error, or a poor workflow boundary. Route feedback to the layer that can actually fix it.
Assign owners for knowledge, workflow logic, platform reliability, and business policy. Review exception patterns together, then fix the layer that failed instead of tuning the model around bad inputs.