Skip to content
TVIJO / RESEARCH ATLAS 2.0.0-candidate

From candidate to your own measurement

Choose a testable configuration for a task. Connect projects, acceptance criteria, benchmarks, and a plan for your own evaluation.

3 projects8 evaluation lenses0 own snapshotsCatalog: 2026-11-09

Own result snapshots

0

No measurements yet

The catalog contains candidates and protocols. A plan or an educational calculator does not create a measured result. A snapshot requires a run, independent acceptance, and trace verification.

Local plan · no data submission

Design an experiment

Always status: planned

Export creates a JSON file on your device. Fields do not request credentials or secrets, and the plan contains no measured results.

Full identity of the selected configuration
Executor / harness
Typed decision adapter · proposed
Model — full label
GLiDE v1.2
API model ID
Exact ID unresolved
Provider / route
Unpinned / не закреплён
Reasoning effort
unverified
Harness version
unverified
Runtime version — separate axis
Unverified
Tools / skills
read-only state · structured choices · probability vector

Research candidate from the decision-model family; exact API, revision and availability are unverified.

Corpus, split, and repetitions

An unset hash is an explicit placeholder. A seed does not guarantee determinism. Repetitions of one task are not independent tasks; freeze their aggregation rule before running.

Budget and limits

Local action. Nothing is written to the server.

Protocol calculators

Manual educational inputs. These numbers are not published as model results.

WILSON 95% / n, k

Acceptance rate and uncertainty

n counts independent unique tasks; k counts accepted tasks. LLM calls and repetitions do not increase n.

Enter n and k to calculate.

The interval assumes independent Bernoulli trials with a common success probability. Related tasks and repeated runs need a different uncertainty model. An interval does not correct taskset bias.

TOTAL COST / ACCEPTED

Cost per accepted result

The numerator includes all cohort costs, including failures, retries, and auxiliary models.

Until cost completeness is confirmed, the result is unknown.

Actual bills, API-equivalent estimates, and allocated subscription costs use different bases. Choose and record one basis for a comparison.

EDUCATIONAL EXAMPLE · NOT A MODEL RESULT

Brier · chosen-answer

p is the stated probability that the chosen answer is correct. y = 1 if independent verification confirms it; otherwise y = 0.

0.09(0.7 − 1)²

This is the binary confidence score of a chosen answer, ranging from 0 to 1; lower is better. Average it over a set. One example does not establish calibration; multiclass Brier is a different formulation.

Evidence status matters.

A documented feature, a test candidate, and a measured result are different states. Session cost does not measure success, and coding-agent performance does not establish trading profitability.

Local fixture evidence · 2026-11-10

What our local harness trials establish

Tvijo is evaluating its own harness. This engineering pilot covers local T1/T2/T3 fixtures, not SWE-bench or Terminal-Bench. This versioned reconciliation retains unknown task costs and acceptance. A completed worker turn does not establish independently accepted work.

Provider exports reconcile to USD 0.895596960 across two nonoverlapping account/key windows. This is neither a task cost nor an invoice or campaign settlement. Task-attributed cost, independent acceptance and allocated subscription seat cost remain unknown (null).

Configuration identity is partial. Returned names and aliases do not attest the model, provider or harness revision. Exact settings and exact patch attribution are unavailable for these observations. No quality index, ranking, stable multiplier or savings guarantee follows.

Canonical experiment ID: tvijo-harness-local-fixtures-20261109-10. This is a source-linked copy, separate from the Atlas proposals: no measured snapshot was imported.

Attempt observations, with explicit denominators

Of 9 recorded observations: 5 completed turns, 2 failed attempts, 1 operator-stopped attempt and 1 unknown/in-flight snapshot. Independent acceptance is unknown for all 9/9; this is not a success rate or a sample of independent tasks. Paid retries remain separate. Usage representations are stream/final copies, not extra responses.

T1 · local fixture
durable queue, claims and effects
T2 · local fixture
cursor pagination contract/store/API
T3 · local fixture
terminal input, paste, grapheme, reflow and viewport
Nine observations; execution status is distinct from independent acceptance and cost.
Attempt / evidenceExecution stateProvider responsesUsage representationsAccepted / task USD
dshrem-T1-01Failed: max tokens48Unknown / unknown
dshrem-T1-02Failed: max tokens48Unknown / unknown
dshrem-T1-03Completed turn; acceptance unknown1428Unknown / unknown
dshrem-T1-patchedCompleted turn; acceptance unknown3060Unknown / unknown
T2-dshrem-01Completed turn; acceptance unknown1428Unknown / unknown
dshrem-T2-patchedCompleted turn; acceptance unknown1530Unknown / unknown
T3-dshrem-01Operator stopped177354Unknown / unknown
dshrem-T3-patchedCompleted turn; acceptance unknown102204Unknown / unknown
dshrem-T3-patched-02Unknown or in flight139278Unknown / unknown

The patched T3 summary is unverified

The supplied report claims 53 calls, 199 seconds, USD 0.028666 and 19 gold cases. These are reported, UNVERIFIED figures. The frozen completed-turn journal has 102 responses; another unfinished snapshot has 139. Their correspondence to the exact attempt and patch is missing. The original released T3 was operator-stopped. No second-place claim is established.

Redacted evidence references and source provenance

These accessible excerpts come from the versioned record. Hashes identify retained private sources; the raw journals and billing exports are withheld. These summaries are not independent verification or an invoice. No prompts, credentials, customer records or third-party benchmark rows are distributed.

Canonical source SHA-256: d7f214ae752ebc27955599ed41c138413f620b3e85fe31f37d914d918d5fe1be

  • dshrem-T1-01 · REDACTED journal excerpt

    Failed: max tokens; 4 response identities; 8 usage representations; independent acceptance and attributed USD: null.

    SHA-256 2ab199094e096742b257386a9c93a1be4679adc1c975843f2cd4c7ae05e53aa8

  • dshrem-T1-02 · REDACTED journal excerpt

    Failed: max tokens; 4 response identities; 8 usage representations; independent acceptance and attributed USD: null.

    SHA-256 0b43c31a9398e2184407419fc2270f39b00687edc2fbf9a2b7e6681de21959f5

  • dshrem-T1-03 · REDACTED journal excerpt

    Completed turn; acceptance unknown; 14 response identities; 28 usage representations; independent acceptance and attributed USD: null.

    SHA-256 3d84188377be1f17a006cc37b40d7361070aa924779a729266579b8a27397e15

  • dshrem-T1-patched · REDACTED journal excerpt

    Completed turn; acceptance unknown; 30 response identities; 60 usage representations; independent acceptance and attributed USD: null.

    SHA-256 a9ce92ba59445e8e1dbeb8b9199c9444f17c1b3c245155dccfc68530cae4a5e4

  • T2-dshrem-01 · REDACTED journal excerpt

    Completed turn; acceptance unknown; 14 response identities; 28 usage representations; independent acceptance and attributed USD: null.

    SHA-256 9f7df82cb8bd97f3518632505747c4b70a92886740e41046def9e4daeec66bfe

  • dshrem-T2-patched · REDACTED journal excerpt

    Completed turn; acceptance unknown; 15 response identities; 30 usage representations; independent acceptance and attributed USD: null.

    SHA-256 5b047d4ae68eb9ad33346fd75b4408f181396c2bfe6403dc14065d68adeedf70

  • T3-dshrem-01 · REDACTED journal excerpt

    Operator stopped; 177 response identities; 354 usage representations; independent acceptance and attributed USD: null.

    SHA-256 62f4959c7ced38068e129e66c408d444bc4e63dc1dfdbd891378d13991878f05

  • dshrem-T3-patched · REDACTED journal excerpt

    Completed turn; acceptance unknown; 102 response identities; 204 usage representations; independent acceptance and attributed USD: null.

    SHA-256 287a0f0a6c705238f415504c6911857b67012ac6bda0bb020e7314b99c52fbb4

  • dshrem-T3-patched-02 · REDACTED journal excerpt

    Unknown or in flight; 139 response identities; 278 usage representations; independent acceptance and attributed USD: null.

    SHA-256 494ea9ac053aeb50e42800f768067f4f2f780fbe32e717074668cb7552a27c06

  • REDACTED export: to ; USD 0.2682621180000000; pricing delta 0E-16. Price export SHA-256: 7ee1fe8ca4058dfdfd14066ca2f919fa7af0d490f68d716da23147d5a96f00fa.
  • REDACTED export: to ; USD 0.6273348420000000; pricing delta 0E-16. Price export SHA-256: 7ee1fe8ca4058dfdfd14066ca2f919fa7af0d490f68d716da23147d5a96f00fa.

Original journals identify 213 responses against 214 requests in their key window: 1 request remains unattributed. This original scope excludes later patched attempts. Overlapping export revisions are not additive; equal token counts do not identify a retry.

When this record cannot answer your question

  • Explicit input revision order selects overlapping export snapshots; their totals are not additive.
  • Provider billing exports do not allocate pooled key-window charges to individual tasks.
  • Normalized journal usage omits authoritative cache-category settlement; no invented task USD.
  • A response identity deduplicates representations, never distinct paid requests with equal tokens.
  • Growing or unterminated journals remain unknown/in flight; no automatic retry or settlement.
  • A worker turn ending does not establish independent grading or acceptance.
  • No stable overall ranking, causal attribution, absolute cap or customer-saving guarantee.

Subscription execution consumes a paid seat and quota; a zero marginal API invoice is not free API work. Earlier closed campaigns retain their own scopes. This record does not establish production readiness, a new integrity proof or final independent acceptance.

DECISION MATH LAB

Decision mathematics: loss and probability updating

Two educational scenarios with manual inputs. All amounts are arbitrary units and all probabilities are example assumptions. These are not API prices, AA data, or return forecasts.

EDUCATIONAL SCENARIO 01

Expected loss: processing or review

Unit: arbitrary amount

p is the assumed probability of a correct result. Error loss and the residual error probability after review are separate inputs. The comparison applies only under these assumptions.

reviewCost includes the complete review workflow. If that workflow starts with automatic processing, include its cost here to avoid missing or double-counting it.

E(auto) = autoCost + (1 − p) × errorLossE(review) = reviewCost + reviewResidualError × errorLoss
Automatic option21
Review option10

In this scenario, the review option has lower expected loss.

Break-even thresholdp* = 1 − (E(review) − autoCost) / errorLoss

0.91 (91%)

For p > p*, the automatic option has lower loss; for p < p*, review has lower loss; p = p* is a tie. This is an educational model boundary, not an instruction to act.

Educational expected-loss curves across p
Automatic optionReview option● current probabilityp* · break-even threshold
Expected losses of two optionsThe horizontal axis shows correct-result probability from 0 to 100 percent; the vertical axis shows arbitrary loss units. Current and checkpoint values are provided in the table below.Expected loss, arbitrary units054.5109.10%25%50%75%100%Probability of a correct result p
Exact values and accessible table
Educational expected loss in arbitrary units; display values are rounded.
pE(auto)E(review)
0%10110
25%7610
50%5110
75%2610
80% · current2110
100%110

The model assumes a common error loss and a constant residual error probability after review. It does not represent every consequence, task heterogeneity, or error dependence. No actions are executed.

EDUCATIONAL SCENARIO 02

Beta–binomial: how observations update probability

Before observing data, p has a Beta(α, β) prior. Then we observe k successes in n independent Bernoulli trials sharing p. All four numbers below are manual educational inputs.

Repeated attempts on one task and related documents are not automatically independent observations. LLM call counts do not increase n. Varying success probabilities require a more suitable model.

p | data ~ Beta(α + k, β + n − k)Beta(8; 4)
0.5α / (α + β)
0.6667(α + k) / (α + β + n)

The posterior mean combines prior assumptions with observations. Prior influence depends on α + β; this is a model parameter, not the count of already measured tasks.

This panel shows distribution parameters and the mean. No credible interval is computed. The mean does not guarantee the next outcome, and an educational update does not establish real-system reliability.