Prompt quality still matters. But in production, reliable AI behavior depends on the systems that select context, constrain authority, manage state, route inference and evaluate outcomes.
Review item: the prompt-only prototype
Use current, relevant context.
Follow the applicable policy.
Use available tools when needed.
Return a valid result.This instruction is readable. It is also carrying several responsibilities that prose cannot reliably enforce.
“Current” depends on which sources the system can access, how they are selected and when they were last updated. “Applicable policy” depends on identity, permissions and policy version. “Available tools” says nothing about whether a specific action is authorized. “Valid result” needs a definition that can be checked.
The prompt is doing useful work: it expresses intent. The engineering review begins when we ask which parts of that intent need to be implemented as system behavior.
Review comment 1: make context an input with properties
A model cannot use information it was never given. Asking it to “use relevant context” does not decide what relevance means, whether the material is fresh, or whether the current user has permission to see it.
Those decisions belong to context preparation: retrieving and selecting information, carrying its source and timing, and applying access rules before it reaches inference. A larger context window can hold more material; it does not, by itself, establish that the material is useful, current or authorized.
Context selection also changes what can be evaluated. A system can be tested for whether it retrieved the right source, used the right version or preserved the source’s meaning. Those are engineering questions, not prompt edits.
Review comment 2: separate tool availability from authority
“Use tools when needed” combines two different questions:
- Can the model identify a useful action?
- Is this system allowed to perform that action now?
The first is a reasoning problem. The second is an authority problem.
Agent research such as ReAct makes the interaction between reasoning, actions and external observations explicit. Once a system can act and observe, the surrounding architecture must also account for what happens when an action fails, returns partial information or is repeated. [2]
A tool description can help a model choose a call. It should not be the only thing that controls access to the tool. Identity, permission checks and action limits need enforcement at the execution boundary. If the action can change an important record or trigger a consequential workflow, the system also needs a defined approval or escalation path.
Review comment 3: turn “valid” into something testable
A model can be asked for a valid result. The system still needs to define what valid means.
That may mean a schema check, required fields, acceptable values, policy checks or a review when evidence is incomplete. If validation fails, the next step should be designed too: reject the result, ask the model to repair it, defer the task or send it to a person.
The distinction matters because polished output can still be wrong in ways that matter operationally. A well-formed response is not the same as an authorized transition or a verified fact.
Review comment 4: represent state outside the conversation
Prompts are often treated as if each inference were a self-contained exchange. Agentic systems may instead perform repeated cycles of preparation, inference, tool execution and observation.
The system needs a durable account of where the work stands. Has an action been proposed, authorized, executed or confirmed? Did the operation complete, fail or leave an uncertain result? Can a retry safely run again?
If those distinctions exist only in conversational text, the system has no dependable basis for deciding what to do next. Explicit state makes recovery, human review and audit possible.
Review comment 5: route inference as a system decision
A production system may choose among models or runtimes according to the task, expected quality, latency, cost, data boundary or output requirements. It may also need a fallback when a runtime is unavailable or returns an unusable result.
That routing should have defined conditions. A fallback that silently changes the system’s data boundary, authority or validation standards is not merely a change in model selection; it changes the operating contract.
Research on DSPy illustrates a related shift in the unit of optimization. Instead of relying only on hand-tuned prompt templates, it represents language-model work as a pipeline whose components can be optimized against a metric. The point is not that every application needs a compiler. It is that the behavior being improved may belong to the pipeline, not to one string of instructions. [1]
Review comment 6: evaluate the system boundary
A prompt test can reveal whether an instruction improves a response on selected examples. It cannot establish that the deployed system consistently retrieves the right context, respects permissions, handles retries or reports failures.
Evaluation should follow the actual path of work: context selection, inference, tool calls, state changes, validation, fallback and escalation. Observability should preserve enough information to explain what happened—for example, which context and policy versions were used, which route was chosen, and whether an action completed.
This is also why governance cannot be reduced to a final approval button. NIST’s AI Risk Management Framework treats governance as cross-cutting across the system lifecycle, alongside mapping, measuring and managing risk. [4]
What prompt engineering still does well
Prompts remain valuable. They help define a task, establish useful constraints, provide examples, guide decomposition and describe the form of an answer. Tool descriptions matter too: the model needs to understand the interface it is being asked to use. Anthropic’s engineering guide discusses routing and workflow patterns and notes that tool design can require substantial optimization in agent systems. [3]
The change is one of scope. Prompt engineering improves an instruction to a model. System engineering defines the conditions under which that instruction is interpreted, the evidence available to it, the actions it may propose, and the checks required before a result changes anything.
A prompt can say what the system should try to do. The system must determine what it can do, how it will know whether it succeeded, and what happens when it cannot.
CARM’INK’s architectural reading
For CARM’INK, this shift connects Semantic Intelligence with Inference Engineering. Structured meaning and selected context shape what an inference can work with; runtime choices, policies, state, identity and controls shape how its output proceeds.
The model remains important, but it is one component inside the system. The engineering task is to make the full arrangement coherent: knowledge and context prepared for a purpose, inference applied where it helps, and execution bounded by explicit authority and oversight.
That is the move from prompt engineering to system engineering. Optimize the instruction, then engineer the conditions that make its use reliable.
References
- Omar Khattab et al., “DSPy: Compiling Declarative Language Model Calls into State-of-the-Art Pipelines”, ICLR 2024.
- Shunyu Yao et al., “ReAct: Synergizing Reasoning and Acting in Language Models”, ICLR 2023.
- Erik S. and Barry Zhang, “Building Effective AI Agents”, Anthropic, 2024.
- National Institute of Standards and Technology, “Artificial Intelligence Risk Management Framework (AI RMF 1.0)”, 2023.