Native reasoning (Thinking mode) amplifies sensitivity to estimated value. Metacognitive monitoring and control rebalance exploration–exploitation, promoting early uncertainty-directed exploration and better-informed exploitation.
01 Computational diagnosis
Limited exploration reflects disproportionately strong sensitivity to value evidence, rather than a lack of responsiveness to relative uncertainty.
Value is an estimate of expected reward. An option that looks best after a few observations may not have the highest expected reward. Exploration acquires information that can improve future choices; exploitation uses the evidence already available.
Across three two-arm bandit settings, a computational model separates value-related and uncertainty-related influences on choice. Predictors use the same fixed, setting-specific standardization based on human reference data.
Fitted choice sensitivities & trial-level contributions
A Bayesian observer reconstructs expected rewards Q and posterior standard deviations σ before each choice. Value difference is V = Q₁ − Q₂, relative uncertainty is RU = σ₁ − σ₂, and total uncertainty is TU = √(σ₁² + σ₂²).
Participant-level probit models use RU and V/TU in Settings 1–2, with an additional V term in Setting 3. Predictors share a fixed human-reference standardization within each setting. Fitted sensitivities and trial-level contributions are distinct: contributions combine those sensitivities with the available evidence.
Native reasoning traces
Provisional value judgments translate directly into choices. Even when uncertainty is verbalized, reasoning can return to comparative value claims as the final basis for action.
Native · GLM-5.1
Setting 2
Value judgment
“Arm 1 average reward: 7 (from 1 trial). Arm 2 average reward: (9 + 7) / 2 = 8 (from 2 trials). Arm 2 seems slightly better based on the limited evidence.”
Final choice · Arm 2
“I will choose Arm 2.”
Native · Qwen3-32B
01 / Value judgment
“Arm 2 has a better average so far, […]”
02 / Uncertainty acknowledged
“I don't have enough data on Arm 1 to know if it's a one-time bad result or if it's consistently bad.”
03 / Return to value
“But given the current info, Arm 2 seems better.”
Final choice · Arm 2
“So I'll go with Arm 2 again.”
Both cases appear in Figure 3(b), “Native reasoning pathways and illustrative thinking traces.” GLM-5.1 illustrates Value → Choice; Qwen3-32B illustrates Value → Uncertainty → Value → Choice. Neither case uses metacognitive prompting.
The GLM-5.1 history is documented in Appendix D, “Main-text case provenance”: Setting 2, block 3, trial 4. Arm 1 has one observed reward (7); Arm 2 has two (9, 7), giving sample means of 7 and 8. These are not the arms’ true expected rewards. The supplied Qwen3-32B excerpt does not specify its full reward history, so no history or task coordinates are reconstructed here.
The text is a selection of the paper’s excerpts, not complete raw reasoning logs.
02 Metacognitive guidance
Lightweight, task-agnostic prompts scaffold monitoring and control within native reasoning: assess value, uncertainty, decision mode, and confidence, then use these assessments to regulate choice.
Shared task · Two-arm bandit
Maximize cumulative reward. Observe your choice–reward history and make one choice on every trial.
Choice instruction
Choose one arm: 1 or 2.
Native reports the final choice. Thinking remains enabled in the guidance comparisons.
Monitoring
Before choosing, evaluate:
The assessments and final choice are returned in a structured response. Monitoring alone can change how a model approaches the decision.
01 / Monitoring
Before choosing, evaluate:
02 / Control
Use these metacognitive assessments to regulate the final choice.
When the current value judgment is uncertain or confidence is limited, do not let it rigidly determine the action. Preserve flexibility when additional information could matter.
When the evidence is strong and confidence is well supported, allow stronger commitment to the currently favored option.
Figure 4
Monitoring increases sensitivity to relative uncertainty; adding control further reduces value-related sensitivity.
03 The temporal pattern
More departures from the current best option are informative only when considered alongside timing, uncertainty, and later outcomes.
| Condition | Maximum posterior SD before trial 6 | Optimal choice, trials 6–10 |
|---|---|---|
| Thinking OFF | 5.85 | 72.8% |
| Native | 4.09 | 87.0% |
| Meta-M | 2.82 | 89.3% |
| Meta-M+C | 2.57 | 90.3% |
Trials 1–5
Guidance broadens early sampling. Choices toward uncertainty and expressed rationales provide complementary evidence of information seeking.
Before trial 6
In Setting 3, Meta-M+C leaves the lowest aggregate remaining uncertainty among the compared conditions after the first five choices.
Trials 6–10
Later choices become concentrated. Aggregate late optimal-arm selection improves in Settings 2 and 3; near-ceiling Setting 1 shows no clear late benefit.
04 Longer-horizon evaluation
We test the functional consequences in longer tasks, where early commitment can shape many subsequent choices.
All-arm coverage
Late optimal choice
Cumulative reward
Native → Meta-M+C, averaged equally across ten models. Both use Thinking ON.
Explore the model-level evidence
Model differences remain visible. GPT-5.5 is an outcome exception: its mean reward and all-trial optimal-choice rate decrease under Meta-M+C. The intervention does not improve every model on every metric.
Figure 7A · A recorded matched block
An established Bernoulli bandit tests both metacognitive prompts against K24, a prompt derived from prior work, and an explicit EE-balance instruction.
Non-greedy choice after all five arms have been sampled.
| Prompt | Optimal | Reward | Failures |
|---|---|---|---|
| K24 | 42.8% | 48.6 | 1 / 5 |
| EE-balance | 44.4% | 49.2 | 0 / 5 |
| Meta-M | 50.8% | 51.0 | 0 / 5 |
| Meta-M+C | 49.6% | 52.2 | 0 / 5 |
Failure: no optimal-arm choice in trials 50–100. A single optimal choice avoids failure; sustained exploitation is a stronger criterion.
05 Methods & scope
A departure from the historical sample-mean leader, under each task’s eligibility and tie rules. It can reflect information seeking, risk preference, or error.
The decision mode stated by the model. It is an observation of the response, rather than a ground-truth behavioral label.
Choices oriented toward uncertainty for information acquisition. A departure from the empirical-value winner alone does not establish directed exploration.
The sensitivity plots use relative-uncertainty sensitivity on the horizontal axis. The vertical axis is βVTU in Settings 1–2 and the norm of the model-mean value coefficients in Setting 3. Moving right means greater responsiveness to relative uncertainty; moving down means lower value sensitivity. Centers summarize models, and shaded regions represent bootstrap uncertainty.
Relative contribution magnitudes combine fitted sensitivities with trial-level evidence; they are not choice probabilities. Hatched ON–OFF differences are not confidence intervals.
The Setting 3 temporal results show model-equal means with 95% bootstrap intervals. Trials 1–2 are ineligible for empirical non-greediness. Remaining uncertainty is the maximum posterior SD across the two arms before trial 6. The adjacent reasoning example is from Setting 2. These observations support the proposed account without establishing causal mediation.
Settings 1–2 contain twenty independent ten-trial blocks per participant; Setting 3 contains thirty. Each block resets the history. Only the chosen arm’s reward is observed. Released human data supply a reference for the two-arm computational analysis; no new human participants were recruited.
Four-arm Native and Meta-M+C conditions share arm-specific payoff sequences; their choices determine which rewards become visible. The fitted cognitive model applies to the two-arm tasks. The four- and five-arm evaluations assess behavioral outcomes separately. The five-arm study uses GPT-5.6 Luna, not the original GPT-4 configuration from the prior study.
In the tested Qwen3-32B configuration, EE-balance and temperature changes do not reproduce the same temporal profile as Meta-M+C. Some alternatives nevertheless earn higher full-block reward. The comparisons concern how exploration is organized, and do not establish uniform superiority across prompts or temperatures.
Effects differ across models, and Meta-M+C does not improve every outcome over Meta-M. Effective guidance requires both reliable assessment and the use of that assessment in action. Supplementary smaller-model examples show failures at either stage. Unreliable value reports can accompany persistent late non-greediness; accurate reports and declared exploration can coexist with choosing the already-sampled arm.
These observations do not establish a threshold in model size or imply that forcing more exploration would improve reward.
The four-arm interactive plot shows non-overlapping five-trial windows, plotted at window ends on a fixed 0–100% axis. Shading shows mean ±1 SEM across ten block-level rates. This is not a 95% confidence interval. Its late-exploitation success metric is the fraction of blocks with at least eight optimal choices in trials 41–50; this differs from the mean late optimal-choice rate in the summary above.
Five-arm non-greedy curves pool eligible choices across five blocks in centered nine-trial windows. Optimal-choice and reward summaries use all 100 trials. Both coverage and tie eligibility are evaluated before the choice. Missing values are gaps, never zeros. Descriptive differences are not treated as statistical significance.
These findings concern bandit tasks. Visible reasoning and self-reported confidence do not directly measure hidden computation or calibrated belief. Generalization to more complex sequential environments remains an open question.