A striking feature of 2026-era digital markets is that the ``contract’’ is often not a static document but an adaptive algorithm: platforms continuously tune commissions and minimum guarantees for gig workers, marketplaces adjust revenue shares for creators, and firms deploy automated procurement and pricing schemes that interact with increasingly autonomous agents (from bidding bots to LLM-based assistants). In these environments, participation decisions are frequent and decentralized: an agent can accept the current terms and act, or opt out and incur no further cost. Yet the same adaptivity that makes these systems economically attractive also introduces a failure mode that classical contract theory typically abstracts away from: participants are not only heterogeneous, they are learning. They may misestimate their continuation values, misunderstand state-dependent terms, or explore actions that generate low realized output. When such learning-driven participation meets per-period participation costs, an agent can accept in good faith and nonetheless suffer realized losses along the way.
This paper is motivated by a simple but practically consequential observation: the standard individual rationality (IR) lens—usually formulated ex ante or in expectation under correct beliefs—does not directly address the design question faced by an algorithmic principal who must interact with boundedly rational or learning agents. In a platform or automated contracting pipeline, we often want a stronger safety guarantee: Without such a guarantee, the system can exhibit the very outcomes we observe in practice: early negative earnings leading to churn, strategic disengagement, regulatory scrutiny, and, in multi-agent settings, feedback loops where participation collapses precisely in the states where data are most needed for learning.
We formalize this concern as a ``practical IR’’ (pIR) requirement tailored to repeated interactions with opt-out and learning. The key conceptual move is to shift attention from the agent’s equilibrium expected utility under correct optimization to the realized payoff process along trajectories that may arise under arbitrary (possibly nonstationary) learning dynamics. In other words, pIR is designed to be robust to the fact that an agent may accept today based on an inaccurate estimate of future value, may take exploratory actions that temporarily depress output, and may confront stochastic realizations that differ from its training distribution. In such settings, ensuring nonnegative realized utility is not merely a normative desideratum; it can be essential for stable participation and for the principal’s ability to collect information and sustain productive interaction over time.
Our model captures a common design pattern in modern pay and pricing systems: homogeneous, state-dependent linear contracts with limited liability. Each period, the principal posts a minimum guarantee (a ``floor’’) and a share rule that pays agents a fraction of their verifiable output. Agents can reject and earn zero, or accept and incur a fixed per-period participation cost. This captures, for example, time and effort costs of being active on a platform, compute costs incurred by an automated agent that chooses to engage, or compliance burdens required to transact. The combination of (i) a reject option, (ii) nonnegative verifiable outcomes, and (iii) a fixed cost creates a sharp worst case: an agent may accept yet realize low or even zero output, precisely when it is exploring or when the environment is temporarily unfavorable.
Within this framework, we establish a set of results that are deliberately elementary but operationally useful. First, we show that there is a sharp and transparent condition for pathwise pIR: setting the minimum guarantee at least as large as the participation cost. Intuitively, because output is nonnegative and the share parameter is nonnegative, the worst realized accepted-step payoff occurs when output is zero. A floor that covers the cost therefore immunizes the agent against losses on every accepted step, regardless of how its policy is formed or how inaccurate its value estimates might be. Conversely, if the floor falls below the cost, then a single accepted step with sufficiently low output produces a realized loss, violating the practical guarantee. This ``safe floor’’ principle is attractive precisely because it does not require modeling the agent’s learning rule, its risk preferences, or its belief updates; it is a pointwise inequality that can be audited and enforced.
Second, we complement the pathwise guarantee with an ex ante variant that speaks directly to acceptance decisions made under bounded value-estimation error. In many algorithmic settings, agents decide to participate when an estimated advantage is nonnegative; such estimates may be biased or noisy during learning. We show that a modest buffer in the floor—inflating it by the estimation error bound—ensures that whenever the agent accepts according to its (possibly wrong) estimate, the expected return from acceptance is still nonnegative. The role of the reject option is crucial: it pins down a nonnegative baseline continuation value, so the principal can focus the safety margin on the immediate participation margin.
Third, we clarify what is and is not at stake from the principal’s
perspective by establishing a transfer-invariance property: holding
fixed the realized action and transition process (and thus outputs and
acceptance indicators), total welfare is invariant to the contract
parameters. Floors and shares reallocate surplus between principal and
agents but do not mechanically destroy value; the only real resource
term is the participation cost. This matters for practice because
discussions of
minimum guarantees'' often conflate redistribution with efficiency. In our environment, imposing a pIR floor is not inherently welfare-reducing; any welfare effect must operate through behavior (e.g., reducing rejection, changing effort, or altering learning dynamics). This distinction helps interpret platform policy debates: the central efficiency question is how safety affects participation and adaptation, not whether transferswaste’’
surplus.
Finally, to connect the safety constraint to standard incentive
considerations, we provide a closed-form benchmark in which the floor is
set at the pIR-minimal level and the share parameter controls effort. In
that benchmark, the optimal interior share takes the familiar value of
one-half, illustrating how the safety floor can be separated from the
incentive margin: the floor ensures participation safety, while the
share governs the marginal return to productive action. While we do not
claim this benchmark captures all strategic subtleties of multi-agent
learning, it provides a useful reference point for thinking about how
insurance'' andincentives’’ interact in linear contract
classes.
Taken together, these results highlight a tradeoff that the model is designed to illuminate. On the one hand, practical IR constraints can be made extremely simple—a floor that covers a fixed cost, plus an optional robustness buffer for estimation error. On the other hand, this simplicity is purchased by strong primitives (notably nonnegative verifiable outcomes and the availability of reject each period) and by focusing on linear contracts. We view this as a feature rather than a bug: modern algorithmic contracting often uses precisely such simple, monitorable payment rules, and regulators and practitioners often demand constraints that are easy to certify. At the same time, we acknowledge limitations. If outputs can be negative (e.g., liability, returns, or externalities), if participation costs vary endogenously with actions, or if the principal can restrict future rejection, then the sharp floor characterization will require modification. We return to these considerations when discussing extensions and the boundaries of practical IR.
The remainder of the paper proceeds as follows. We next situate our contribution relative to classical IR/IC/LL constraints, principal–agent reinforcement learning, and fairness-aware objectives in multi-agent systems. We then formalize the Markov game and pIR definitions, present the main propositions and their proofs, and discuss implications for contract design under learning, including how safety floors can interact with fairness regularization and participation dynamics.
Our analysis sits at the intersection of three literatures: classical contract-theoretic constraints (IR/IC/LL), principal–agent models studied through the lens of reinforcement learning and online adaptation, and fairness-aware objectives in multi-agent systems. The common thread is a shift from static, correctly-optimized participation to repeated interaction in which terms and behavior evolve, and where safety guarantees are valued precisely because learning and exploration can produce adverse realized outcomes.
In standard principal–agent theory, individual rationality (IR) is typically imposed in expectation under correct beliefs, often at the outset of the relationship (ex ante IR) or conditional on a signal or type (interim IR). Incentive compatibility (IC) then selects actions or reporting behavior, and limited liability (LL) restricts transfers when agents cannot make payments to the principal. This triad is foundational in mechanism design and moral hazard models; see, e.g., . In dynamic settings, participation constraints may be imposed at multiple dates, and can be written as promise-keeping or continuation-value constraints as in dynamic contracting and repeated moral hazard . These models emphasize intertemporal incentives and the role of continuation utilities, but they typically assume agents evaluate contracts using the correct continuation value (or at least behave as if they do), so that IR is meaningful as an equilibrium object.
Our notion of practical individual rationality (pIR) is
complementary: rather than requiring that a correctly optimizing agent
weakly prefers participation in expectation, we require that , even when
the agent’s policy is misspecified, exploratory, or nonstationary. This
places pIR closer in spirit to robust or
safe'' constraints that are certifiable without modeling beliefs, learning dynamics, or risk preferences. There are clear analogies to worst-case or distributionally robust contract design \cite{hansen_sargent2008}, and to robust mechanism design that hedges against misspecification \cite{bergemann_morris2005}, but our emphasis is different: the uncertainty is not only about primitives or types, but about the agent's own decision process as it learns. The pIR constraint therefore functions as a \emph{trajectory-level} participation safeguard, reflecting operational concerns in platforms and automated marketplaces whereearly
losses’’ can trigger churn, reputational harm, or regulatory scrutiny
even if long-run expected utility would be positive.
The specific contract class we study—state-dependent linear sharing rules with a minimum guarantee—connects to a long tradition of linear contracts in principal–agent theory. Linear contracts are central both as tractable benchmarks and as empirically common payment forms (commissions, revenue shares, piece rates), with classic results on their optimality under particular informational and technological conditions . Minimum guarantees are likewise common in practice (e.g., guaranteed earnings floors, sign-on guarantees, creator funds), and can be interpreted as insurance against low output realizations. In our environment, the minimum guarantee plays an additional role: it is the lever that enforces pIR under minimal assumptions (nonnegative verifiable outcomes and per-period opt-out). In this sense, our results provide a ``compliance-style’’ characterization—a pointwise inequality on posted terms—rather than a full equilibrium characterization of optimal contracting under hidden information.
A second related thread is principal–agent reinforcement learning (RL) and, more broadly, the interaction between incentives and learning in dynamic environments. A growing literature studies how a principal can use contracts or mechanism parameters to shape the behavior of learning agents, and how an agent’s exploration and value estimation affect outcomes . In algorithmic marketplaces, the principal may also be a learning system, adjusting prices, commissions, or guarantees online; this connects to online learning and bandit approaches to contract design . Much of this work evaluates performance in expectation (regret, long-run average reward) and typically presumes that the agent’s reward is the contract payment, so that an ``IR’’ concern is not separately enforced. Our contribution is to isolate a simple condition under which learning-driven exploration cannot produce negative realized payoffs for participants, even if their acceptance decisions are made using imperfect value estimates. This safety layer can be viewed as orthogonal to (and potentially supportive of) learning efficiency: by preventing downside realizations from participation, a principal may sustain engagement long enough for both sides to learn.
Our pIR perspective is also related to the broader safe RL and risk-sensitive control literature, which asks how to constrain policies to avoid catastrophic outcomes . The key difference is the object being protected: rather than constraining the system’s state trajectories or enforcing chance constraints on events, we constrain . In platform applications, this distinction matters because the salient ``catastrophe’’ may be negative earnings (net of participation costs) that are immediately experienced by the participant and that feed back into participation dynamics. The fact that our pIR floor is independent of the agent’s learning rule is especially relevant when the principal interacts with heterogeneous, proprietary, or opaque agents (including automated agents whose internal algorithms are not observable).
A third literature concerns fairness in multi-agent learning and
economic allocation. In multi-agent RL (MARL), fairness is often
introduced via regularizers or constraints that penalize inequality
across agents, enforce proportionality, or guarantee minimum performance
levels . In economics, related ideas appear in social welfare
functionals, inequality indices, and Rawlsian criteria. Our framework
accommodates these concerns through fairness-aware objectives over
realized (or expected) wealth vectors. The main conceptual link is that
minimum guarantees and revenue shares are transfers that can be tuned to
manage the distribution of surplus, potentially reducing dispersion in
cumulative payoffs across heterogeneous agents. At the same time,
because our welfare decomposition makes clear that (holding behavior
fixed) contracts are transfer instruments, any efficiency impact of
fairness-motivated redistribution must operate through
behavior—participation, effort, or learning dynamics—rather than through
mechanical loss of surplus. This clarifies a common policy debate:
floors and guarantees are frequently criticized as
distorting'' orinefficient,’’ but in environments where the
principal and agents jointly produce nonnegative output and the only
real resource cost is participation, redistribution is not itself
wasteful; the crucial question is how redistribution changes engagement
and incentives.
Finally, our work is related to minimum earnings regulations and platform policy discussions (e.g., guaranteed pay standards for gig work), where a core practical demand is that participants not be exposed to systematic losses after accounting for time, effort, or operating costs. While our model is deliberately stylized, the pIR condition provides a theoretical justification for a design principle that aligns with these debates: if participation carries a fixed, unavoidable cost and output is nonnegative and verifiable, then a minimum guarantee equal to that cost is the sharp contract-level safeguard against realized losses. Where agents are learning, the additional buffer we highlight can be interpreted as a robustness margin that accounts for predictable miscalibration early in deployment.
We study a finite-horizon principal–agent Markov game in which a
single principal repeatedly posts contractual terms to a population of
n agents who can opt out in
any period. The central modeling choice is to make participation and
through contract parameters that are observable and enforceable at the
time they are posted, while allowing agents to behave in a potentially
nonstationary, misspecified, or learning-driven manner. Our results
later will leverage two simple primitives: (i) the availability of a
per-period outside option (reject) and (ii) nonnegativity
of verifiable output.
Time is indexed by t ∈ {0, …, T − 1}. At each period the interaction is summarized by a publicly observed state st ∈ S. The state collects payoff-relevant information available to all parties at contracting time (e.g., demand conditions, historical performance signals, or platform-side features). State transitions may be stochastic and depend on the current state and realized actions; we write this generically as st + 1 ∼ P(⋅ ∣ st, a1, t, …, an, t). Agents may be heterogeneous in ways not directly observed by the principal (e.g., skill, cost shifters, or learning algorithms), and this heterogeneity can influence realized outcomes and thus the state evolution. We intentionally do not parameterize this heterogeneity, because our participation-safety guarantees will not rely on identifying it.
Each agent i ∈ {1, …, n} has a base
action space Ai describing
the feasible work'' oreffort’’ choices if it participates.
In addition, in every period the agent may choose reject, interpreted as declining
to participate for that period and receiving zero payoff. We denote the
augmented action space by Aia := Ai ∪ {reject}
and the realized action by ai, t ∈ Aia.
The opt-out option is available repeatedly; thus participation is not an
all-or-nothing relationship but a sequence of reversible decisions.
A key cost primitive is a fixed per-period participation cost c > 0 that is incurred an agent chooses any action in Ai (i.e., whenever it does not reject). This cost can represent time, fuel, attention, or any other unavoidable expense of being active in the system during a period. Importantly, c is independent of which particular ai, t ∈ Ai is chosen, so agents may still explore within Ai without changing the fixed cost exposure.
If agent i participates at
time t (i.e., ai, t ≠ reject),
an outcome yi, t ≥ 0
is realized. We interpret yi, t
broadly as a observed by the principal, such as revenue generated, tasks
completed, or a scaled reward signal. The nonnegativity assumption is
natural in many platform contexts (gross earnings, clicks, sales) and
will be the knife-edge condition behind sharp participation floors: the
worst-case realized outcome is yi, t = 0.
We allow yi, t
to depend on the state, the agent’s action, other agents’ actions, and
idiosyncratic randomness.
At each time t, after observing st, the principal posts a contract bt consisting of two state-dependent parameters: a minimum guarantee mt(st) ≥ 0 and a share αt(st) ∈ [0, 1]. The contract is in the sense that the same (mt(st), αt(st)) is offered to all agents in that period/state, rather than being individually tailored. This restriction captures posted terms (e.g., a platform-wide guarantee and commission rate) and is also a natural design constraint when individual heterogeneity is hidden.
Given participation, agent i’s payment is linear in the
verifiable outcome:
The lower bound mt(st) ≥ 0
is a limited-liability (LL) condition on transfers from principal to
agent: the principal cannot impose a negative fixed transfer. The share
restriction αt(st) ∈ [0, 1]
ensures the agent is not paid more than one-for-one in output and also
rules out ``negative commissions’’ that would effectively charge the
agent for producing output; both are standard in applications.
This sequence makes explicit that agents commit to participation before outcomes are realized, so any participation safeguard must protect them against low realizations of yi, t as well as against their own potentially exploratory action choices.
For an agent, the contractual payoff (utility flow) equals payment
minus the fixed participation cost, conditional on participating:
Thus reject yields ri, t = 0
by construction.
The principal’s flow payoff is the realized output net of payments to
participating agents:
The principal therefore trades off stronger agent insurance (higher
mt) and
incentives (higher αt) against its
own retained surplus (1 − αt)y.
We evaluate performance over the horizon via expected cumulative
wealth. For each agent i and
the principal,
Let W denote the vector of
terminal (cumulative) wealths, including the principal if desired. A
central accounting identity is that total welfare equals output minus
real participation costs:
This expression makes clear that (mt, αt)
are transfer instruments: holding behavior fixed, changing contract
terms only reallocates surplus between principal and agents.
To accommodate distributional objectives, we allow the principal to
value both its own wealth and a fairness functional F(W) that summarizes
inequality or minimum-guarantee concerns (e.g., negative variance, 1−Gini, or a Rawlsian minimum). A generic
objective is
where πp
is the principal’s policy mapping states to contract parameters and
λ ≥ 0 indexes the weight
placed on fairness. This formulation highlights the practical role of
minimum guarantees and shares as levers for both participation safety
and distributional control. In the next section, we formalize a
participation notion tailored to learning and exploration: rather than
asking whether participation is optimal under correct beliefs, we ask
when posted terms ensure that accepting cannot generate realized
losses.
Classical individual rationality (IR) is typically stated as an
condition: under correct beliefs and equilibrium play, each agent
prefers participating to its outside option in expectation. In our
setting, however, agents may be learning, exploring, or otherwise
misspecified, and participation decisions are made the realization of
yi, t.
In such environments, standard IR can be too weak for the kinds of
guarantees that matter operationally (e.g.,
a worker should not lose money by showing up'' ora user
should not be harmed by opting in’’). We therefore formalize a
participation requirement that is (i) defined on realized paths, (ii)
robust to arbitrary agent behavior, and (iii) checkable through posted,
auditable contract parameters.
To see the issue, suppose a contract is IR for a fully informed best-responding agent because high expected output makes participation profitable on average. A learning agent, however, may (a) accept while experimenting with low-performing actions, (b) accept in states where it mistakenly overestimates continuation value, or (c) accept and then experience adverse realizations of yi, t early on. With a fixed participation cost c, any accepted step with sufficiently low realized yi, t can generate a realized loss even if the expected value under an optimal policy is nonnegative. In short, equilibrium IR protects rational ; it does not protect against , exploration, or bad luck. In platform and policy contexts, this distinction is first-order: a guarantee is often intended to be rather than .
We define a strong, trajectory-level notion of individual rationality
that conditions only on whether the agent participates. Let a realized
trajectory be denoted by τ = (s0, a0, y0, s1, a1, y1, …, sT − 1, aT − 1, yT − 1),
suppressing agent indices where unambiguous. Using the flow payoff
ri, t(τ) = (mt(st(τ)) + αt(st(τ)) yi, t(τ) − c) 𝟙 [ai, t(τ) ≠ reject],
we say that a contract policy πp satisfies if,
for every agent i, for every
(possibly history-dependent, nonstationary) agent policy profile (π1, …, πn),
and for almost every realized trajectory τ induced by (πp, π1, …, πn),
The interpretation is deliberately
accounting style'': whenever an agent participates, it accumulates realized gains and losses, and pIR requires that the total realized net payoff over the episode is never negative. Because \(\texttt{reject}\) yields \(0\), condition \eqref{eq:pathwise-pir} is the naturalno-harm
from participation’’ requirement relative to the outside option, without
appealing to beliefs or equilibrium reasoning.
Two remarks are useful. First, is : it does not assume the agent plays a best response, uses correct models, or even evaluates the contract correctly. Second, pIR is stronger than any purely expectation-based participation constraint: it implies that even an unlucky sequence of realized outcomes cannot bankrupt the agent . This strength is precisely what makes pIR suitable as a safety guarantee but also highlights a limitation: pIR is not an efficiency statement and does not ensure that participation is attractive in expectation; it only rules out realized losses from participating.
In some applications we may want a weaker notion that aligns more
directly with decision-making: whenever an agent accepts at (t, s), the of that
acceptance should be nonnegative. Let Vi, t(s)
denote agent i’s true
continuation value from state s at time t under the induced policies
(including the option to reject in future periods). Define the true
acceptance value
where the conditional expectation integrates over outcomes and
next-state transitions induced by the (possibly learning) policies. We
say the contract satisfies if
Condition is weaker than pathwise pIR because it allows for negative
realized payoffs along some paths, provided the conditional expectation
at the acceptance moment is nonnegative. Nevertheless, it is the
relevant object if we want to ensure that an acceptance decision is not
a mistake .
To connect to bounded rationality, we incorporate a simple error model: agent i uses an estimate V̂i, t(s) of its continuation value and satisfies a uniform bound |V̂i, t(s) − Vi, t(s)| ≤ ε. Under a myopic acceptance rule—accept whenever the estimated advantage is nonnegative—the agent may accept in states where the true advantage is slightly negative. This is exactly where standard IR arguments become brittle: they rely on correct best responses, while here acceptance is driven by V̂ and may be ``near-indifferent’’ due to estimation noise.
Accordingly, we define as the requirement that holds for any agent
whose estimation error is bounded by ε and whose acceptance is based on
V̂. The key design idea is then
to use the contract’s floor mt(s)
as an against decision errors: by making the worst-case immediate
accepted payoff sufficiently positive, we can ensure that even an agent
who is wrong by up to ε does
not incur a negative true expected return conditional on acceptance. In
the next section we provide sharp conditions under which such buffers
are sufficient, leveraging two primitives of the model: nonnegativity of
yi, t
and the perpetual availability of reject, which together make the
``worst case’’ at acceptance transparent.
Our central observation is that, once we insist on a no-loss guarantee under arbitrary (possibly learning) behavior, the contract must insure the agent against the worst realized accepted-step outcome. Because yi, t ≥ 0 and αt(s) ∈ [0, 1], the share term cannot reduce the agent’s payoff, so the only way an accepted step can be harmful is through the fixed participation cost c not being covered when output happens to be low. This immediately suggests a ``safe floor’’ principle: set the minimum guarantee mt(s) high enough that even when yi, t = 0, the agent does not lose money by participating.
Assume yi, t ≥ 0
almost surely and that the cost c > 0 is incurred whenever ai, t ≠ reject.
Then the homogeneous contract class pi, t = mt(st) + αt(st)yi, t
satisfies pathwise pIR (as in ) for all agents and all agent policies if
and only if
Under acceptance, the worst realized outcome is the lowest feasible yi, t, which is 0. Since αt(s) ≥ 0, the share cannot make the accepted-step payoff worse, so the minimal accepted-step payoff is mt(st) − c. If this is nonnegative uniformly, then every accepted step is individually safe, and summing over accepted steps yields nonnegative cumulative payoff regardless of learning, exploration, or miscoordination. Conversely, if mt(s) < c somewhere, a single accepted step with yi, t = 0 produces a realized loss, immediately violating the pathwise guarantee.
(Sufficiency) For any realized trajectory τ and any time t,
ri, t(τ) = (mt(st(τ)) + αt(st(τ))yi, t(τ) − c)𝟙[ai, t(τ) ≠ reject] ≥ (mt(st(τ)) − c)𝟙[⋅] ≥ 0,
where the first inequality uses yi, t(τ) ≥ 0
and αt(⋅) ≥ 0.
Summing over t yields .
(Necessity) If there exists (t, s) with mt(s) < c,
consider any trajectory on which st = s,
the agent accepts, and yi, t = 0;
then ri, t = mt(s) − c < 0,
hence the cumulative sum is negative. ▫
Condition is ``auditable’’ and policy-relevant: it requires no modeling of behavior, no equilibrium computation, and no estimation of heterogeneous productivities. Moreover, it cleanly separates from : once m ≥ c ensures that participation cannot cause losses, the share schedule α can be chosen to manage effort, exploration incentives, or profit-sharing objectives without threatening the no-harm guarantee.
Suppose agents accept at (t, s) whenever their
estimated acceptance advantage is nonnegative (equivalently, they may
accept when Q̂i, t(s, accept) ≥ 0).
Assume the reject option remains available in future periods, so Vi, t + 1(s′) ≥ 0
for all s′. If
agents satisfy a uniform value-estimation error bound |V̂i, t(s) − Vi, t(s)| ≤ ε
and the principal chooses
then conditional on acceptance, the true expected acceptance value is
nonnegative at every acceptance decision.
Estimation error expands the set of states in which an agent might (mistakenly) accept a contract it is nearly indifferent about. A buffer of ε in the accepted-step payoff immunizes the decision against such near-indifference: even if the agent’s continuation value is overestimated by ε, the realized immediate term m + αy − c is large enough that the true acceptance value cannot dip below zero. The option to reject later ensures the continuation component cannot be negative in the true model, so the buffer only needs to protect the current step.
From ,
Qi, t(s, accept) = 𝔼 [mt(s) + αt(s)yi, t − c + Vi, t + 1(st + 1) ∣ st = s, ai, t ≠ reject].
Under and yi, t ≥ 0,
we have mt(s) + αt(s)yi, t − c ≥ ε
almost surely, and by the reject option Vi, t + 1(st + 1) ≥ 0.
Hence Qi, t(s, accept) ≥ ε ≥ 0.
The bounded-error acceptance model motivates why such a buffer is
operationally relevant: agents may accept with Q̂ ≈ 0, but the floor ensures the
Q remains nonnegative. ▫
Both sharpness results rely on two primitives that are natural in many platform settings but not universal: (i) nonnegativity of verifiable contributions (yi, t ≥ 0), and (ii) a fixed per-period participation cost that is incurred whenever the agent participates. If outcomes can be negative (e.g., penalties, liabilities) or if costs depend on the chosen action, then the ``worst case’’ may no longer be y = 0, and a simple uniform floor may be insufficient without additional restrictions (such as caps on losses or action-contingent guarantees). In that sense, m ≥ c should be viewed as the sharp condition the limited-liability, nonnegative-output class.
Holding fixed the joint action/transition realizations (in
particular, holding {yi, t}
and {𝟙[ai, t ≠ reject]}
fixed), total welfare is invariant to the transfer parameters (m, α):
$$
w_p+\sum_{i=1}^n w_i
=\mathbb{E}\!\left[\sum_{t=0}^{T-1}\sum_{i=1}^n
(y_{i,t}-c)\,\mathbb{1}[a_{i,t}\neq \texttt{reject}]\right].
$$
Moreover, increasing the floor by Δmt(s)
decreases the principal’s expected wealth by
$$
\mathbb{E}\!\left[\sum_{t=0}^{T-1}\Delta m_t(s_t)\sum_{i=1}^n
\mathbb{1}[a_{i,t}\neq \texttt{reject}]\right],
$$
and increases the agents’ aggregate expected wealth by exactly the same
amount.
Floors and shares are accounting transfers: they redistribute realized output between principal and agents but do not, by themselves, create or destroy surplus. Therefore, imposing the pIR floor does not reduce welfare. Any welfare effect must operate through behavioral responses—most importantly, through participation decisions under learning. In practice, this means a safety floor can be ``paid for’’ by reducing costly rejection or churn, even if it reallocates profit away from the principal in a static accounting sense. This transfer-invariance lens also clarifies what one can and cannot conclude from pIR constraints: pIR is a safety requirement, not an efficiency statement, and its efficiency consequences hinge on how the floor changes agent behavior in the underlying Markov game.
To make the preceding ``safe floor’’ logic operational, it helps to separate two channels that are intertwined in richer Markov games: (i) (can an agent ever be made worse off by accepting?) and (ii) (how does the contract shape productive effort). We therefore study a one-agent, one-state benchmark in which we can solve both sides of the contract in closed form and read off exactly when the floor matters.
A single agent chooses effort e ∈ [0, 1] each period after
accepting. Output is
y = θe,
where θ > 0 is
productivity. Effort entails a standard quadratic disutility $k(e)=\tfrac{\kappa}{2}e^2$ with κ > 0, and participation incurs
the fixed cost c > 0
whenever the agent accepts. The principal offers the same linear
contract as in the general model,
p = m + αy, α ∈ [0, 1], m ≥ 0,
so that the agent’s one-period utility from accepting and choosing
effort e is
while the principal’s one-period profit is
We interpret – as a stylized model of many platform contracts: m is a base payment (or ``show-up
fee’’), while α is a
commission rate.
Fix (α, m). Since
m − c enters
additively, the agent’s optimal effort depends on α but not on m (nor on c). The first-order condition for an
interior optimum is
$$
\frac{\partial u}{\partial e} = \alpha\theta - \kappa e =0
\quad\Longrightarrow\quad
e = \frac{\alpha\theta}{\kappa}.
$$
Imposing the feasibility constraint e ∈ [0, 1] yields the best-response
function
This is the key ``non-distortion’’ message: within the risk-neutral,
linear-output, action-independent-cost class, increasing the floor m is a pure transfer that does
change the agent’s marginal tradeoff between output share and effort
cost.
To connect to the safety results, we first impose the pIR-minimal choice m = c (the tightest floor that prevents an accepted step from being a loss when y can be low). Substituting into gives two regimes.
Here e*(α) = αθ/κ,
so
Maximizing over α ∈ [0, 1]
yields the familiar interior optimum
$$
\alpha^*=\frac{1}{2},\qquad
e^*=\frac{\theta}{2\kappa},\qquad
\Pi(\alpha^*)=\frac{\theta^2}{4\kappa}-c.
$$
Thus, once we have fixed a safe floor, the principal’s remaining problem
is the classic incentive tradeoff: raising α increases effort linearly but
reduces the principal’s retained share one-for-one, with the balance
achieved at 1/2 under quadratic
costs.
If αθ ≥ κ,
then e*(α) = 1 and
profit becomes
Π(α) = (1 − α)θ − c,
which is decreasing in α. The
principal therefore chooses the share that still induces maximal
effort:
$$
\alpha^*=\min\Big\{1,\frac{\kappa}{\theta}\Big\},\qquad e^*=1,\qquad
\Pi(\alpha^*)=\theta-\kappa-c\quad\text{(when }\kappa/\theta\le
1\text{)}.
$$
This corner highlights a practical lesson: if productivity θ is high relative to effort
curvature κ, only a modest
commission rate is needed to elicit ``full effort,’’ and additional
sharing is pure rent transfer to the agent.
If the agent can reject, acceptance requires u(e*(α); α, m) ≥ 0.
With the safe floor m = c, the agent’s
acceptance utility simplifies to
$$
u(e^*(\alpha);\alpha,c)= \alpha\theta
e^*(\alpha)-\frac{\kappa}{2}(e^*(\alpha))^2 \;\ge\; 0,
$$
with strict inequality whenever α > 0. Hence in this benchmark,
the pIR-minimal floor not only prevents losses, it also makes acceptance
automatically feasible for any α > 0.
If, instead, the floor were set below cost (say m < c), then acceptance could still be optimal for sufficiently large α because incentives can compensate for the negative base term. However, this is precisely where pathwise pIR and standard expected IR diverge: when realized output can be low (or when the agent explores suboptimal actions), m < c exposes the agent to realized losses on accepted steps. The benchmark therefore clarifies why a ``safe floor’’ is not merely a participation constraint in the usual sense; it is an (trajectory-level) harm constraint.
Within –, raising the floor from m = c to m = c + Δ shifts
payoffs but leaves behavior unchanged:
e*(α) is
unchanged, Π(α) decreases by Δ for every
α.
This is exactly the sense in which safe floors are ``behaviorally
cheap’’ in this class: the principal pays for safety in profits, not in
weaker effort incentives.
That said, the non-distortion conclusion is not universal; it is a feature of additive transfers under risk neutrality and deterministic linear production. Two extensions are particularly relevant in practice. First, if output is noisy and agents are risk-averse, then α provides incentives but also transmits risk, and increasing m can permit a lower α while keeping participation attractive—in which case safety policy attenuate incentives through an insurance channel. Second, if participation costs depend on the chosen action (or if there are hidden penalties so that y can be negative), then protecting agents may require action-contingent guarantees or stronger restrictions than a uniform m, and incentive interactions become unavoidable. We view the present benchmark as a deliberately clean baseline: it illustrates, in the simplest possible setting, why enforcing a pIR-style floor can be compatible with strong incentive provision through α, while also flagging the modeling features under which this compatibility may fail.
We now describe a practical learning procedure for the Markov game in which the principal learns a contract policy πp while agents learn behavioral policies {πi}i = 1n. The core design goal is to ensure that the principal searches only over contracts satisfying the pathwise pIR floor, while still permitting rich state-dependent sharing αt(s) and compatibility with standard deep RL tooling (e.g., PPO). Our baseline is a two-timescale scheme: agents update quickly in response to the current contract, while the principal updates slowly using rollouts generated by (approximately) adapted agents.
Because the principal affects the agents’ incentives and thus the
transition distribution over states and outcomes, joint learning can be
unstable if both sides update aggressively. We therefore adopt a
separation of learning rates (or update frequencies) that treats the
agents as
near best responses'' to the current contract during a principal update. Concretely, letting \(\eta_i\) denote agent learning rates and \(\eta_p\) the principal learning rate, we impose \(\eta_p \ll \eta_i\), or equivalently update the principal once every \(K\) agent updates with \(K\) large. This is the standard heuristic behind actor--critic training in games, but here it has an additional normative interpretation: safety constraints are enforced \emph{structurally} at the contract level, so the principal’s slow adaptation changes the environmentgently’’
from the perspective of boundedly rational agents.
At each decision (t, st),
the principal outputs parameters (mt(st), αt(st)).
To enforce limited liability and the pIR floor, we do not optimize over
m directly; instead we
parameterize it via an unconstrained network output zm and a
nonnegative transform:
Choosing csafe = c
implements the sharp pathwise pIR floor, while csafe = c + ε
implements the error-robust buffer used in the bounded-value-error
acceptance model. Similarly, we map an unconstrained output zα(s)
to a share in [0, 1] via
where σ(⋅) is the logistic
function. This ``hard’’ parameterization has two practical advantages.
First, it eliminates the need for Lagrange multipliers or penalty tuning
to satisfy the floor constraint. Second, it guarantees that safety holds
, including under exploration noise or distribution shift, because the
constraint is never violated at the action-selection level.
When we use a stochastic contract policy (useful for exploration or
for representing mixed strategies), we sample from an unconstrained
distribution and then transform. For instance, with Gaussian
logits,
zm(s) ∼ 𝒩(μm(s), σm2(s)), zα(s) ∼ 𝒩(μα(s), σα2(s)),
and then apply –. This yields a reparameterizable stochastic policy
suitable for policy gradients while preserving the almost sure
constraint mt(s) ≥ csafe.
Each agent i observes (st, mt(st), αt(st))
and chooses ai, t ∈ Ai ∪ {reject}.
We train πi with PPO
using the realized flow payoff
ri, t = (mt(st) + αt(st)yi, t − c) 𝟙[ai, t ≠ reject],
and we include reject as a
genuine action with reward 0 and no
cost. This explicit opt-out matters empirically because it gives the
policy a low-variance ``safe default’’ during early learning. In
implementation, we maintain an agent value function Vϕi(st, mt, αt)
to reduce variance and compute GAE advantages Ai, t.
The PPO update takes the standard clipped form
maxθi 𝔼[min (rt(θi)Ai, t, clip(rt(θi), 1 − δ, 1 + δ)Ai, t)],
with rt(θi) = πθi(ai, t ∣ oi, t)/πθiold(ai, t ∣ oi, t)
and oi, t = (st, mt, αt).
Nothing in the PPO machinery needs to be altered for safety: the floor
guarantee is enforced upstream by the principal’s parameterization.
The principal receives flow payoff
$$
r_{p,t}=\sum_{i=1}^n\Big((1-\alpha_t(s_t))y_{i,t}-m_t(s_t)\Big)\,\mathbb{1}[a_{i,t}\neq\texttt{reject}],
$$
and may also incorporate a terminal fairness functional F(WT)
with weight λ. We implement
the principal objective as an episodic return
and train πp via PPO (or
any actor–critic variant) using trajectories generated by the current
population of learning agents. Because F(WT)
depends on realized wealths, it is convenient to treat it as a terminal
reward term; equivalently, one can distribute it across time with
potential-based shaping, but we keep the implementation simple and
transparent.
A subtlety is that the principal’s action affects not only immediate profit but also future agent behavior through learning. Our two-timescale approach approximates the relevant gradient by holding agent parameters fixed within a principal update window, which is consistent with the idea that the principal is optimizing against the agents’ behavioral response. Operationally, each principal iteration proceeds as follows: (i) freeze πp, (ii) train agents for K episodes to adapt to the fixed contract policy, (iii) collect rollouts under the adapted agents, and (iv) update πp once (or a few epochs) using PPO on . This reduces non-stationarity relative to fully simultaneous updates.
Even with the hard parameterization , numerical issues can arise (e.g., floating-point underflow, or hand-coded baselines that inadvertently subtract c twice). We therefore implement redundant guards:These checks are cheap and turn the theoretical floor into an engineering invariant. Importantly, they preserve the interpretation of the floor as a : violations are treated as bugs, not as rare events.
Because m is a pure transfer, the principal may face weak learning signals if exploration occurs primarily through m rather than α. We therefore recommend (i) initializing zm near zero so that m ≈ csafe + log 2, and (ii) using larger initial exploration variance for zα than for zm. When fairness is included, F(WT) can be high variance; we mitigate this by normalizing wealths within a batch and by using a principal value baseline that conditions on state summaries predictive of future wealth dispersion. These choices do not change the constraint logic; they simply stabilize optimization.
Overall, the algorithmic message mirrors the economic one: we can enforce pIR safety with a simple structural restriction m ≥ csafe, and then let learning allocate incentives through α and (optionally) distributional objectives through λF(WT).
Our experiments are organized around a single empirical question suggested by the theory: when agents can always opt out and face a fixed participation cost c, does enforcing the hard floor mt(s) ≥ csafe (with csafe ∈ {c, c + ε}) eliminate practical-IR failures mechanically degrading welfare, and does it improve learning behavior by reducing early-stage rejections? Proposition~1 predicts that, as long as realized outcomes satisfy yi, t ≥ 0, pathwise pIR violations should be identically zero under m ≥ c, regardless of how misspecified the agents are. Proposition~2 predicts an additional robustness margin: when acceptance is driven by noisy value estimates, raising the floor to c + ε should protect against mistaken acceptances that would otherwise be losses in expectation. Finally, Proposition~3 cautions us that any welfare change we observe must come from behavioral responses (acceptance, effort-like actions, coordination), not from the transfer itself.
We adapt sequential social-dilemma benchmarks in which agents
interact repeatedly and generate verifiable, nonnegative contribution
signals. Each period produces a state st summarizing
local observations and the recent interaction history, and each agent
chooses either an environment action (e.g., move/collect/coordinate) or
reject, which removes the
agent from play for that period and yields payoff 0. We define yi, t ≥ 0
as a scaled, verifiable contribution statistic (e.g., collected “public”
items, team-aligned events, or other nonnegative counters) so that the
model’s nonnegativity requirement is exactly satisfied. Heterogeneous
types enter by varying how effective an agent’s action is at generating
yi, t
(productivity differences) and/or by varying the mapping from local
observations to latent opportunity (informational advantages). The
principal observes st and posts a
homogeneous contract (mt(st), αt(st))
that applies to all agents, so any compensation differences arise only
through realized yi, t,
not through personalized parameters.
In this setting, the most salient failure mode under unconstrained contracting is straightforward: if exploration or early learning drives mt(st) < c in some states, then an agent who accepts and happens to realize low contribution (often yi, t ≈ 0 early on) suffers a realized loss, which can trigger persistent rejection thereafter. The safe-floor contract is designed precisely to remove this “scar” from exploration. We therefore treat the sequential dilemma as a stress test for safety under high non-stationarity: simultaneously learning agents, changing interaction patterns, and state-dependent contracts.
Our second domain is an explicitly economic task-market with
congestion. Each period, a random number of tasks arrive with
nonnegative verifiable values, and agents decide whether to accept a
task (choosing among feasible assignments) or reject. The state st includes
queue lengths, recent arrival rates, and coarse summaries of market
tightness. Realized outcomes yi, t ≥ 0
are task completions or realized task values net of verifiable
nonnegative adjustments (e.g., service-level credits), ensuring
compatibility with limited liability. Hidden heterogeneity reflects
agent-specific service rates or match quality with task classes, which
affects yi, t
but is not directly observed by the principal. This environment is
useful because it creates a genuine dynamic tradeoff: aggressive
incentives can reduce congestion (by increasing task take-up), but they
can also induce misallocation if agents over-accept tasks they are
poorly suited for.
To operationalize the bounded-error acceptance model behind Proposition~2, we introduce controlled learning errors by limiting critic capacity, injecting observation noise, and/or adding small stochastic perturbations to estimated advantages that determine acceptance tendencies. This gives a regime in which agents sometimes accept when their true continuation value is low, precisely where the ε-buffer is meant to matter.
We report four primary families of metrics. First, the captures
participation:
$$
\mathrm{RejectRate} \;=\;
\frac{1}{nT}\,\mathbb{E}\Big[\sum_{t=0}^{T-1}\sum_{i=1}^n
\mathbb{1}[a_{i,t}=\texttt{reject}]\Big].
$$
Second, we measure pathwise, matching the definition used in the theory.
Let realized accepted-step wealth for agent i be
w̃i = ∑t: ai, t ≠ reject(mt(st) + αt(st)yi, t − c).
A trajectory exhibits a pIR violation if miniw̃i < 0.
We report both the violation probability ℙ[miniw̃i < 0]
and the expected shortfall 𝔼[min {0, miniw̃i}],
since small numerical bugs or environment edge cases can manifest as
rare but severe violations. Third, we report and its decomposition into
principal and agents’ wealths, emphasizing that welfare shifts are
interpreted as behavioral. Fourth, we report via a terminal fairness
summary, using 1 − Gini(WT) and
a Rawlsian criterion miniwi,
which together distinguish “equalizing transfers” from genuinely
improving the worst-off agent.
We conduct three systematic ablations. (i) Participation costs c: increasing c raises the binding floor and thus tightens the principal’s feasible set; the theory predicts that safety remains trivial to maintain (m ≥ c), but incentives through α may need to work harder to sustain acceptance. Empirically, we test whether higher c amplifies the participation value of the safe floor by preventing early negative shocks. (ii) Estimation error ε: we compare csafe = c versus csafe = c + ε in the noisy-acceptance regimes of the queueing domain, where mistaken acceptances are common. The prediction is not that c + ε dominates in profit, but that it reduces conditional-on-acceptance losses and stabilizes participation under misspecification. (iii) Stochastic vs. deterministic contract policies: we compare deterministic outputs (mt, αt) to stochastic policies that sample logits before transforming to (m, α). This isolates whether exploration in contract space improves downstream welfare/profit by discovering better state-dependent sharing rules, and whether the hard floor remains effective under stochasticity (it should, because it is enforced almost surely by construction).
To interpret results, we include two conceptually simple baselines: a fixed contract baseline (constant m, α tuned coarsely) and an unconstrained-learning baseline in which the principal learns m directly (with only soft penalties for violating m ≥ c). The central comparison is that the unconstrained baseline can exhibit transient m < c during exploration, while the safe-floor method cannot. We aggregate results across random seeds and report means with uncertainty intervals over evaluation rollouts. Across both domains, the key empirical objects are (i) whether pIR violations are eliminated in practice (not just in expectation), and (ii) whether the induced reduction in early rejection improves long-run outcomes by keeping agents engaged long enough to learn high-y behaviors.
Our analysis isolates a simple but practically consequential design principle: when agents can opt out each period and participation triggers a fixed cost, a hard minimum guarantee operates as a that is independent of behavioral assumptions. This perspective is useful because many empirical failures in learning-to-contract settings look less like subtle incentive incompatibilities and more like ``accidental debt’’ episodes created during exploration. The floor condition mt(s) ≥ c eliminates those episodes pathwise under y ≥ 0, and the ε-buffer mt(s) ≥ c + ε extends the logic to acceptance decisions driven by misspecified continuation values. In this section we discuss how these statements change when we relax modeling choices, and how the floor can be interpreted as an auditable compliance constraint rather than a purely theoretical sufficient condition.
We imposed a homogeneous floor and a constant participation cost c to emphasize the sharpness of the pIR characterization. In practice, participation costs are often state-dependent (e.g., higher effort intensity in congested states, higher cognitive load under uncertainty, or switching costs when re-entering after rejection). If the cost is observable and contractible as ct(st), the natural extension is immediate: the sharp pIR floor becomes mt(s) ≥ ct(s) pointwise. The more interesting case is or agent-specific costs ci, t, which can be interpreted as private disutility, heterogeneous opportunity costs, or latent resource constraints. Then a homogeneous floor cannot guarantee pIR for agents unless it covers the worst-case type, which may be infeasible. A useful intermediate notion is ``high-probability pIR’’ under a known bound or distribution of ci, t: one chooses m to cover c for all but a small tail of types, and uses rejection/attrition of those types as the remaining margin. This clarifies a practical tradeoff: stronger safety for disadvantaged or high-cost agents requires higher transfers to all accepted agents, shifting rents away from the principal and potentially altering participation and selection.
Our pIR definition takes reject to yield zero flow payoff
and no cost. This captures many opt-out implementations (do nothing,
incur no penalty), but in labor-market and platform settings the outside
option is often positive and time-varying (e.g., alternative gigs,
unemployment insurance, or endogenous reputational benefits from not
participating). If rejecting yields utout(st),
then the relevant safety requirement is not nonnegativity per se but
dominance over the outside option along the realized path: each accepted
step should not reduce realized cumulative utility relative to always
rejecting. In the simplest extension with a constant outside option,
this amounts to mt(s) ≥ c + utout(s)
under y ≥ 0. When the outside
option depends on history (e.g., rejecting today improves tomorrow’s
opportunities), one must compare to a dynamic benchmark, and a stepwise
floor may no longer be sufficient for pathwise dominance. Nonetheless,
the logic behind Proposition~2 still provides a usable design heuristic:
if future outside options preserve nonnegativity of continuation values,
then protecting the immediate step with a buffer remains the main lever
for avoiding mistaken acceptances that are regretted ex post.
The nonnegativity assumption yi, t ≥ 0
is natural for many ``contribution’’ proxies but excludes settings with
fines, losses, or negative externalities. If outcomes can be negative
but are bounded below by ymin(s), then
the worst-case accepted-step payoff becomes mt(s) + αt(s)ymin(s) − c,
so a sufficient and (under mild regularity) necessary condition for
pathwise pIR is
mt(s) ≥ c − αt(s) ymin(s),
capped by limited liability at mt(s) ≥ 0.
This reveals a previously hidden coupling: with downside risk, incentive
intensity α tightens the
required floor, because the contract exposes the agent to negative y realizations. In environments
where y is unbounded below or
heavy-tailed, no finite floor can provide pathwise safety, and one must
move to alternative instruments (e.g., truncating the performance
measure, using a concave payment schedule, or explicitly insuring
losses). More broadly, if the goal is protection against rather than
worst-case outcomes, then pathwise pIR may be overly strong and one may
adopt probabilistic safety constraints; our framework can be adapted by
replacing almost-sure bounds with concentration bounds on realized
cumulative payoffs.
We treated c as an exogenous per-period cost independent of the chosen non-reject action. In many applications, however, the relevant ``cost’’ is endogenous: effort disutility, fatigue, or risk exposure rises with the action profile and may depend on state. If the cost function is known and contractible, one can fold it into the per-step safety constraint by requiring m + αy ≥ k(a, s) for all feasible accepted actions a, which is generally too strong unless one restricts the action set or allows action-dependent contracts. If costs are private information, the safe floor can still play a role as a robust guarantee, but it will not prevent agents from taking privately costly actions that reduce their own utility. This points to an important limitation: pIR in our sense protects agents from losses induced by the contract and realized outcomes, not from self-inflicted losses due to unmodeled preferences. Addressing this requires either richer behavioral models (effort choice, risk aversion) or mechanism constraints that limit harmful actions (e.g., safety policies at the environment level).
A salient advantage of the floor constraint is that it is . Unlike incentive constraints that depend on unobserved counterfactuals, the condition mt(s) ≥ csafe can be checked directly on the posted contract parameters, and it has a clear interpretation as a minimum-wage-like protection in opt-out task markets. This suggests a ``safe harbor’’ compliance rule: platforms or principals may be allowed to optimize incentive shares αt(s) freely, provided they certify a minimum guarantee that covers the participation cost (and, under bounded-estimation models, an additional buffer). In automated contracting systems, this can be implemented as a hard projection step or a parameterization that enforces the inequality almost surely, making violations attributable to engineering failures rather than strategic behavior. The transfer-invariance result further clarifies how such regulation should be evaluated: floors primarily reallocate surplus unless they change acceptance and behavior; hence empirical evaluation should focus on participation, learning stability, and distributional outcomes.
Several modeling choices delimit what our results can and cannot claim. We assumed verifiable outcomes y and ruled out misreporting; in practice, measurement is noisy and sometimes manipulable, which reintroduces classic auditing and mechanism-design issues. We assumed homogeneous contracts; allowing agent-specific parameters could improve efficiency but raises fairness, discrimination, and information concerns, especially under hidden heterogeneity. We also abstracted from principal budget constraints and limited commitment; if the principal cannot credibly commit to future floors or faces liquidity constraints, agents may rationally reject despite a currently safe mt(s). Finally, our welfare invariance is conditional on fixed action/transition realizations; understanding how floor constraints interact with equilibrium selection in multi-agent dynamics, with exploration by both sides, and with fairness objectives F(W) remains an open area. A promising direction is to treat safety floors as in a joint learning problem: the principal learns state-dependent shares αt(s) subject to auditable pIR, while agents learn actions in an environment where early negative shocks are ruled out by construction.