When AI reads what isn’t there

The risk of uncontrolled requirements extraction

When a system specification runs to hundreds of pages, extracting requirements by hand stops being realistic. Teams reach for generic chat assistants to turn source documents into structured requirements, and the speed is impressive: within minutes you have a list of hundreds of apparent requirements. But speed without control creates a hidden liability. What isn’t in the document gets generated just as effortlessly as what is.

The invisible risk: hallucinations and implicit assumptions

An empirical study by Huang et al. (ACM, 2025) shows that large language models hallucinate in repository-level software engineering contexts. These aren’t surface-level typos; the models generate deep, plausible-sounding deviations inside complex development scenarios. In requirements work, that translates into something specific and dangerous: a model can quietly promote an implicit assumption into an explicit obligation.

A source document expresses a preference, and the assistant reformulates it as a hard requirement. Or worse, it introduces constraints that appear in no paragraph at all. The output looks structured, but the structure is often a reconstruction of what the model thinks the document should have said.

What this looks like in practice

Consider a familiar situation on an infrastructure tender. The source specification states that a maintenance access hatch is “preferably” positioned on the north face of a structure — a preference, not a requirement. The extraction returns a clean line: “The structure shall provide a maintenance access hatch on the north face.” Grammatically correct, correctly formatted, and entirely wrong in status. A preference has become a binding obligation.

Now scale that. Somewhere in the same run, the model adds a requirement referencing a safety standard the document never cited — a standard that sounds right for the domain, so nobody questions it. Neither of these survives scrutiny if someone checks the source. The problem is that at volume, nobody does.

The scaling trap: from twenty to five hundred requirements

The real issue is not that errors occur, but that they become invisible as you scale. At twenty requirements you still check line by line. At five hundred, the brain inevitably switches to pattern recognition. The fluency of the output mimics reliability: an invented function or a non-existent interface specification slips into the requirements database unnoticed, simply because the sentences are grammatically correct and technically plausible.

This is not an occasional glitch but a structural property of how these models work, and it scales with your output. As the list grows, so does the risk hidden inside it — and the two grow together.

Why generic tools make it worse

Generic chat assistants work as a black box. They offer no paragraph numbers, no source references, and no classification of what they found. You get a list, but no traceability.

The parallel with code generation is direct. CACM (August 2025) documented how LLMs invent non-existent software packages and APIs and present them as if they were real components. For requirements extraction, this translates into non-existent regulatory obligations, invented technical constraints, or improvised interface definitions. Without a mechanism that ties every extraction to a concrete piece of source text, telling fact from fiction is simply not possible.

What’s at stake: downstream and compliance

A hallucinated requirement does not stay neatly in the extraction spreadsheet. It flows downstream into system architecture, test protocols, and validation gates. If a supposed regulatory obligation was never in the source document but your AI extracted it anyway, you end up designing and building against a fiction — and correcting it means unwinding every downstream artifact that referenced it. That rework cascade is what makes such an error far costlier than the time the generic tool ever saved.

For compliance and certification, this is especially damaging. Auditors demand traceability from requirement to source. A black-box extraction with no paragraph reference or context gives you an answer, but not evidence.

The solution: give the agent rules, and keep a human in the loop

The problem is not AI itself, but the absence of rules and verification loops. Basewise DRE (Document Requirements Extractor) is built specifically for systems engineering workflows and treats extraction as a governed process rather than an open-ended question. In practice, that governance shows up in two ways.

In fully automatic mode, a two-pass structure runs a structural scan of the document first, then extracts in parallel against a defined schema — so the model follows a fixed structure instead of interpreting freely. In human-in-the-loop mode, you select text segments inside a live PDF viewer and choose, per segment, between verbatim extraction and an INCOSE-structured rewrite. In both cases the human stays in control of what counts as a requirement.

Every extraction carries a unique ID, a source reference (document and paragraph), its surrounding context, and a classification label: requirement, assumption, obligation, or explicitly “not a requirement.” The output is a structured table that imports directly into requirements management tools. Basewise DRE is deliberately positioned as the step before classic requirements management tools, not as a replacement. The goal is traceable, validated input, so that downstream tools work from facts demonstrably tied to a source.

Closing: a practical rule of thumb

Use generic AI as a thinking partner, not as a requirements extractor. If you can’t point to a specific paragraph and a classification label, it isn’t a requirement yet. The rule is simple: check the source before the requirement enters the chain. At twenty requirements you do that out of habit. At five hundred, you need a tool that enforces it.