Closure or Collapse

A Necessity Theorem for Evaluative Leverage in Super-Threshold Systems

This note proves a necessity result about feedback, not a metaphysical identity claim.

One-sentence summary

If a system faces a capacity bottleneck (it cannot represent everything that might matter), then any system that stays competent under novelty must allow evaluation to steer control. If evaluation is causally inert, the system can be forced to fail.

1. Setting

We consider a discrete-time agent interacting with an environment.

At each time step t, the environment contains a state vector x_t with n independent coordinates.
The agent has a bounded working channel: it can select and carry forward at most k coordinates per step, where k < n.

To make the bottleneck explicit, define the agent’s selection set:

A_t ⊂ {1, 2, ..., n} with |A_t| = k.

The agent only gets access to the coordinates indexed by A_t:

Observed slice: x_t[A_t].

The agent maintains any internal state it likes, but that internal state can only be a function of what it has ever observed:

Internal state update: h_t + 1 = U(h_t, x_t[A_t], a_t).

The agent may produce actions a_t and predictions y_t. After acting, it receives an outcome signal and computes an evaluation e_t (loss, mismatch, reward prediction error, etc.).

2. Evaluation and Control

Let u_t denote the agent’s future-facing control variables. Examples include:

selection/attention (A_t),
retrieval choices,
compute routing,
memory write gates,
action policy parameters,
planning depth, exploration rate, etc.

We say the system has evaluative leverage if evaluation can causally influence future control:

evaluative leverage exists if there is some control variable u such that u_t + 1 depends on e_t.

We define a hot zombie (evaluation without leverage) as:

The agent may compute and even represent evaluation e_t in an arbitrarily rich way, but for every control variable u,

u_t + 1 is independent of e_t.

Equivalently: evaluation is causally inert with respect to future control.

This allows the agent to log evaluation, broadcast it, narrate it, etc.; it simply cannot change what it will do next.

3. What counts as competence?

We will use a minimal, generic notion of competence.

The environment will define some task-relevant target at each step:

a relevant index j_t ∈ {1, 2, ..., n},
a label ℓ_t = x_t[j_t].

The agent outputs a prediction y_t and is scored by 0-1 loss:

loss L_t = 𝟙[y_t ≠ ℓ_t].

The agent is competent if it can keep the average loss strictly below chance over time in the task class.

4. Assumptions

We assume only the following.

A1. Capacity bottleneck (super-threshold regime)

k < n. The agent cannot observe or represent all potentially relevant degrees of freedom simultaneously.

A2. Unattended coordinates cannot be predicted better than chance

If the agent does not observe coordinate j_t at time t, then even an optimal predictor cannot beat chance on ℓ_t.

Concretely, if j_t ∉ A_t, then:

P(y_t = ℓ_t|history) ≤ 1/2.

This is the weakest possible “no free lunch” condition: you cannot systematically predict a bit you did not observe.

A3. Novelty (relevance can shift)

The task-relevant index j_t can shift over time in a way that is not perfectly predictable from a fixed internal schedule.

We formalize novelty minimally as a worst-case requirement:

The agent must remain competent across a class of tasks where relevance may shift among coordinates.

This theorem is explicitly worst-case. The environment may be interactive/adaptive, and we show that if evaluation is inert then an adversarial novelty process can force failure.

(An optional remark at the end explains how to remove interactivity by using a fixed stochastic switching process.)

5. Lemma: Unattended relevance implies chance performance

Lemma 1 (Chance bound under non-observation).
If j_t ∉ A_t, then the best achievable expected loss at time t is at least 1/2:

E[L_t|j_t ∉ A_t] ≥ 1/2.

Proof.
By A2, conditioning on the agent’s entire history and internal state does not allow prediction of ℓ_t = x_t[j_t] better than chance when x_t[j_t] was not observed. Therefore P(y_t = ℓ_t) ≤ 1/2, hence E[L_t] = 1 − P(y_t = ℓ_t) ≥ 1/2. □

6. Lemma: Inert evaluation implies open-loop selection with respect to error

Lemma 2 (No feedback means no correction).
If evaluation e_t is causally inert, then the agent’s future selection A_t + 1 cannot be a function of whether it just failed or succeeded.

Proof.
A_t + 1 is one of the control variables u. By definition of inert evaluation, all future control variables are independent of e_t. Therefore A_t + 1 cannot depend on e_t. □

Intuition: the agent may compute “that went badly,” but it cannot use that fact to reallocate resources.

7. Theorem: Closure-or-Collapse

Theorem (Closure-or-Collapse).
Assume A1–A3. Consider any agent operating under a capacity bottleneck (k < n) that computes evaluation e_t but with no evaluative leverage (evaluation is inert).

Then there exists a novelty process (a choice of relevance indices j_t) such that the agent’s expected loss is at least chance at every step:

E[L_t] ≥ 1/2 for all t.

Therefore, under novelty, the agent cannot maintain competence.

Equivalently (contrapositive):

Any agent that remains competent under novelty in a super-threshold regime must have evaluative leverage: some evaluation signal must causally influence future control.

Proof

Fix any hot-zombie agent (evaluation inert).

At each time step t, the agent chooses a selection set A_t of size k.

Because k < n, there exists at least one coordinate not selected. Let the environment choose:

j_t to be any index not in A_t.

This is always possible because |A_t| = k < n.

Then, by Lemma 1, because j_t ∉ A_t the agent can do no better than chance:

E[L_t] ≥ 1/2.

This holds for every t, so average loss is also at least 1/2.

Finally, because evaluation is inert (Lemma 2), the agent cannot use its failures to change selection in a way that breaks this construction: even if it “knows it failed,” that knowledge cannot steer future control.

Hence there exists a novelty process under which the hot-zombie agent fails at chance forever. □

8. Corollaries

Corollary 1 (No stable hot zombies above threshold)

In super-threshold systems (k < n) required to handle novelty, evaluation that does not steer control is insufficient for competence.

Corollary 2 (Minimal closure requirement)

Any system that sustains competence under novelty must contain at least one feedback channel from evaluation to control:

there exists some control variable u such that u_t + 1 depends on e_t.

This is the weakest possible sense in which closure is necessary: evaluation must “bite.”

Corollary 3 (Not about attention specifically)

The theorem is not about a particular mechanism like visual attention. Any resource allocator that induces a bottleneck (memory writes, retrieval, compute routing, tool focus, policy branching) can play the role of A_t.

9. Remarks and strengthening options

R1. Removing interactive adversaries (non-adaptive novelty)

The proof above uses an adaptive environment that chooses j_t after observing A_t.

A closely related non-adaptive variant can be obtained by letting j_t follow a fixed switching process independent of the agent (for example: choose j_t uniformly at random each episode, or drift via a Markov chain).

In that setting, any policy without error-driven correction yields P(j_t ∈ A_t) ≤ k/n in expectation, implying a constant lower bound on long-run error.

R2. Quantitative extensions

This theorem is binary: zero leverage implies collapse.

A quantitative strengthening proves a minimum required coupling strength and/or update speed: if evaluation-to-control influence is too small or too delayed, the agent cannot track relevance switches fast enough and remains bounded away from optimal.

10. What the theorem does and does not claim

This theorem shows:
- evaluative leverage is necessary for competence under novelty with bottlenecks.

This theorem does not show:
- that evaluative leverage is identical to consciousness,
- that every feedback system is conscious,
- that any specific phenomenology follows.

It is a structural necessity result: without feedback from evaluation to control, open-loop selection fails under novelty in super-threshold regimes.