Semantic Intelligence: Why Structure Should Precede Generation

Ask an AI system: Which clause governs this supplier’s 2026 renewal?

A search can return the original contract, an amendment, a pricing note, and a similar agreement. A language model can summarize them fluently. But to answer reliably, the system must identify the supplier, connect the documents to the right contract, determine which clauses apply to the renewal date, and preserve where each claim came from.

That work is semantic. It establishes what the information refers to and how its parts relate before the system uses them to answer.

In CARM’INK’s framing, Semantic Intelligence is the engineering of information into structured, traceable knowledge that can shape context, support inference, and constrain action. This is a CARM’INK thesis, not a claim that every researcher uses the phrase identically.

Generation cannot resolve every relationship by itself

Language models can produce useful answers from prompts and retrieved text. Retrieval-augmented generation (RAG) demonstrated one way to combine a language model with an external document index for knowledge-intensive tasks. The original study also identified provenance and knowledge updating as open problems. Its results support retrieval as a useful component in the settings tested; they do not show that retrieval alone resolves identity, authority, applicability, or conflict. (Lewis et al., NeurIPS 2020)

Consider the contract question. A passage can mention the correct supplier and the word “renewal” without governing this renewal. A system must distinguish the supplier’s current agreement from a prior contract, link the amendment to the agreement it changes, and evaluate the relevant dates and scope.

Text similarity can help find candidates. It does not, by itself, represent all the relationships needed to decide which candidate applies.

Make meaning explicit enough to use

A semantic layer records the elements that matter to a task:

• Entities and identity: which supplier, contract, clause, product, person, or event is being discussed
• Types and properties: what kind of entity something is and which attributes are relevant
• Relationships: how entities connect, such as an amendment changing a contract or a clause governing a renewal
• Constraints and time: which conditions apply, when they apply, and when they expire
• Provenance: which source supports a fact, who or what produced it, and when
• Uncertainty: which links are inferred, unresolved, incomplete, or in conflict

These elements need not live in one knowledge graph. They may be represented through an ontology, typed records, relational data, rules, document links, provenance records, or a combination selected for the task. RDF, for example, provides a framework for expressing and linking information about resources; W3C’s PROV-O provides a vocabulary for representing and exchanging provenance. (W3C RDF 1.1 Primer; W3C PROV-O)

The purpose is not to add structure for its own sake. It is to make relevant identities, relationships, constraints, and sources inspectable by the components that retrieve, reason, generate, and act.

Structure shapes context

Once the system has a representation of the task and its knowledge, it can select a context for a particular request. That context might include a small set of linked records, relevant passages, applicable rules, and the evidence that connects them.

This is different from passing a large collection of possibly related material to a model and expecting generation to sort out every distinction. In “Lost in the Middle,” Liu and colleagues found that, on their multi-document question-answering and key-value retrieval tasks, model performance could vary depending on where relevant information appeared in a long input. The result is specific to the tested tasks and models. It reinforces a practical design point: context quality involves relevance and organization, not simply length. (Liu et al., TACL 2024)

A 2024 GraphRAG preprint offers another example of structure changing how a system handles context. Edge and colleagues described deriving an entity knowledge graph and community summaries from documents. For a specific class of global sensemaking questions over datasets in the million-token range, they reported improvements in answer comprehensiveness and diversity compared with a conventional RAG baseline. These findings are bounded to the questions and datasets they tested; they do not establish that graph-based retrieval is best for every application. (Edge et al., Microsoft Research, 2024 preprint)

The engineering implication is to select context from meaning as well as textual relevance. A system should be able to explain why a record or passage belongs in the context: it concerns the right entity, is valid for the relevant time, supports a required relationship, or records a constraint.

Let inference operate on structured context

Structure does not replace inference. It gives inference a better-defined problem.

A system can use deterministic code for exact calculations, policy rules for explicit constraints, retrieval for missing evidence, and model reasoning for ambiguity or multi-step interpretation. The right operation depends on the task. Each operation should return results that can be checked against the represented evidence and constraints.

Structure can also be incomplete or wrong. Entity resolution can fail. An extracted relationship can be mistaken. A knowledge base can preserve an outdated rule. A model may help propose structure from unstructured sources, but a proposed fact or link should remain distinguishable from a verified one. Provenance helps trace a statement; it does not prove the statement is true.

A practical pipeline therefore needs feedback. If validation finds an unresolved supplier identity, a missing amendment, or a conflict between sources, the system should return to evidence resolution rather than generate a more confident answer from the same incomplete context.

A working architecture

The principle can be expressed as a sequence:

1. Observe: collect documents, records, events, and tool outputs relevant to the task.
2. Structure: resolve identities and represent relationships, constraints, time, provenance, and uncertainty.
3. Select context: retrieve and assemble the evidence that applies to the current request.
4. Infer: use models, deterministic tools, and policies suited to the remaining work.
5. Validate: check claims, calculations, constraints, and action proposals against their sources.
6. Respond or act: provide a traceable answer, request missing information, abstain, or proceed within authorized limits.

This is not necessarily a one-way pipeline. Inference may reveal that more evidence is needed. New evidence may require updating the semantic representation. The governing principle is that a consequential answer or action should be based on an explicit, inspectable account of meaning—not on fluent generation alone.

What CARM’INK means by Semantic Intelligence

CARM’INK uses Semantic Intelligence to describe an engineering approach that makes meaning operational: complex information is structured into knowledge, relevant context is assembled for a task, and inference is connected to evidence and constraints.

This makes Semantic Intelligence broader than a knowledge graph, ontology, search method, or language model. Those may be components. The discipline is how they work together to preserve identity, relationships, provenance, and control as a system moves from information to reasoning and action.

Structure should precede generation because a system needs to establish what its information means before it can responsibly use that information to answer or act.

References

1. Lewis, P. et al. “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks.” Advances in Neural Information Processing Systems 33 (NeurIPS 2020). Paper.
2. Liu, N. F. et al. “Lost in the Middle: How Language Models Use Long Contexts.” Transactions of the Association for Computational Linguistics, 12, 157–173, 2024. Paper.
3. Edge, D. et al. “From Local to Global: A Graph RAG Approach to Query-Focused Summarization.” Microsoft Research, 2024 preprint. Publication page.
4. W3C. “RDF 1.1 Primer.” Specification.
5. W3C. “PROV-O: The PROV Ontology.” Recommendation.