When Consensus Is Not Knowledge

I have been working on a framework called REIM: the Reticular Epistemic Inference Model.

It begins with an unglamorous but consequential observation. Whenever many people, systems, or agents report on the same thing, we usually take the average.

Product ratings, sensor readings, expert evaluations, survey responses, model judgments: collect enough of them, put them in a column, and produce a number.

The number looks like consensus. It is often only arithmetic.

The Average Has No Theory of the Observer

An average assumes that every observation deserves the same weight. In the real world, they do not.

Some observers are careful and well calibrated. Some are inconsistent. Some have only partial access to what they are evaluating. Some are reporting on a version of the system that no longer exists. Some are actively trying to move the result.

Treating those observations as equivalent is not neutrality. It is a decision to ignore the conditions under which the observation was produced.

This matters more now because the number of observers is growing faster than our ability to inspect them. An organization may ask several AI agents to assess a claim, a policy, a generated answer, or a business decision. It may add human review, a benchmark, a compliance check, and a customer signal. The resulting dashboard can show apparent agreement while concealing a more important question:

Is this a robust judgment, or several versions of the same unreliable one?

REIM is an attempt to make that question computable.

Truth Is Not a Vote

REIM estimates two things together: the latent property of the system being observed, and the reliability of each observer.

An observation from a source that has been consistently close to the inferred truth receives more weight. A source that is noisy, badly calibrated, or adversarial receives less. The model does not need a prior declaration of who is trustworthy. It infers reliability from behavior across observations.

The principle is simple:

A judgment is not only a value. It is also an event produced by an observer with a history, a position, and limits.

That last part is important. REIM is not an attempt to discover an omniscient judge. It starts from the opposite premise: every observer is internal to the system it describes. Its view is local, partial, and noisy. What we call truth has to be reconstructed from the interaction of limited viewpoints.

The framework is derived from my Reticular Theory of Reality, which treats knowledge as something produced inside a network of observers and systems, not delivered from outside it. The computational model makes that premise operational.

Consensus Needs an Audit Trail

The first implementation of REIM covers standard batch inference, real-time updates, hierarchical systems, and multi-dimensional analysis. It can correct for the age of observations, carry information through a system hierarchy, and provide uncertainty estimates where the data is sparse.

On synthetic data, this changes the outcome materially. With 20% adversarial observers, REIM reduced error by up to 93% compared with a simple average. As the adversarial share increased, the average deteriorated predictably; REIM remained comparatively stable because the unreliable observers lost influence.

But the result I find most interesting is not the score. It is the reversal of the usual question.

Instead of asking only what does the group think?, we can ask:

• which observers are carrying the conclusion;
• how certain the conclusion really is;
• where credible observers disagree;
• whether a result survives the removal of one model, team, prompt, or source;
• and whether the system has enough independent, competent viewpoints to justify confidence at all.

Those are governance questions as much as statistical ones.

AI Judgment Is a Special Case

This is where the work becomes especially relevant to AI systems.

Organizations are beginning to use models to evaluate models: one agent checks factuality, another checks policy compliance, another rates quality, and a human reviewer is brought in only when something looks uncertain. This can be useful. It can also create a very polished form of circularity.

Several agents can agree because they share a model family, a prompt pattern, a missing source, or the same blind spot. A confident consensus may be a monoculture in disguise.

I am developing AuditREIM as a governance extension for this problem. It treats model, prompt, policy version, evaluator, and human reviewer as observers with domain-specific reliability, bias, calibration, and epistemic horizons. A factuality checker should not dominate a legal judgment. A model without retrieval should not be treated as equally competent on claims that require it. A result that depends on one observer cluster should be marked as fragile, not celebrated as consensus.

The aim is not to automate responsibility away. It is to make the structure of a judgment visible enough for responsibility to be exercised.

From More Signals to Better Judgment

We are building systems that can produce almost unlimited evaluation. The scarce resource is not another score. It is a defensible basis for trusting one.

REIM is still early work, and it is not a claim that every form of judgment can be reduced to an inference loop. It is a framework for situations in which many partial observations exist and the reliability of the observers matters to the answer.

That includes product reviews and distributed sensing. Increasingly, it includes the judgments that pass between people and AI systems inside organizations.

The question is no longer whether we can collect more opinions. We can.

The question is whether our systems can distinguish a conclusion supported by many independent, reliable observations from one merely repeated often enough to look true.

Consensus is a signal. It is not yet knowledge.

The REIM source code, technical report, and the proposed AuditREIM extension are available at the project repository.