Skip to content
TVIJO / RESEARCH ATLAS 2.0.0-candidate

How to read benchmarks

Choose a testable configuration for a task. Connect projects, acceptance criteria, benchmarks, and a plan for your own evaluation.

3 projects8 evaluation lenses0 own snapshotsCatalog: 2026-11-09

From an English label to a practical decision

Definitions and primary sources. Third-party raw scores and prices are not reproduced here.

Decision

Choice correctness, confidence, and the cost of error. Evaluate separately from speed and explanation quality.

Openness

Weights availability, licensing, and transparency. Hardware requirements and deployment feasibility require separate verification; openness is not an intelligence score.

Matching terms: 88

Decision — decision quality

Decision — decision quality / 0–100

AA Decision Index v2

Индекс принятия решений AA v2

What it measures
A weighted assessment of structured decisions, confidence and invariance.
Why it matters
A research lead for routing, classification and extraction. Freeze the component version.
Limitation
Five Decision Scores plus 15% Symmetry. It is not a project success probability or the Jev scale.
Decision — decision quality / 0–100

Decision Score

Оценка правильности и уверенности

What it measures
For a matching evaluation: 50 × (1 + accuracy − mean((p − c)²)); c is chosen-answer correctness.
Why it matters
Penalizes confident mistakes; accuracy is a fraction from 0 to 1.
Limitation
This component formula does not reconstruct the full v2 index. Ground truth is required.
Decision — decision quality / versioned score

Jev Decision Index 0.4.1

Индекс решений Jev: Public и Full

What it measures
Separate public reproduction and private Full evaluation; Public adjusts for chance.
Why it matters
Use the public kit to test exact engines with pinned data and protocol.
Limitation
Full uses 60% private same-skills and 40% private new-domain tests. Public is not a reproduction of Full.

Openness — openness and transparency

Openness — openness and transparency / 0–100

AA Openness Index

Индекс открытости ИИ-модели

What it measures
Access to weights, licenses and transparency of data, methodology and training code.
Why it matters
Helps assess self-hosting, reproducibility and supplier dependence.
Limitation
V1.0 normalizes raw points by 18. Openness does not establish accuracy, hardware fit or commercial permission by itself.

Agents and task execution

Agents and task execution / % objectives

AutomationBench-AA Score

Выполнение целей с соблюдением ограничений

What it measures
Average objective completion per task; a task with a guardrail breach receives zero.
Why it matters
Relevant to multi-application AIOS workflows and controlled document processing.
Limitation
Partial objectives are not fully accepted tasks. Simulated apps do not certify your production tools.
Agents and task execution / % objectives

Objectives Completed

Выполненные цели

What it measures
Completed objectives, including tasks with violations.
Why it matters
Read together with acceptance and violation metrics.
Limitation
The difference from Score is not itself a violation rate; aggregation can differ.
Agents and task execution / % tasks

Tasks Completed

Полностью завершённые задачи

What it measures
Tasks with every objective met and no guardrail violations.
Why it matters
Closer to acceptance of a complete business result.
Limitation
Do not substitute an objective completion percentage.
Agents and task execution / versioned index

AA Coding Agent Index

Сравнение агентных конфигураций для кода

What it measures
Composite software-engineering evaluation of an agent with its model and execution settings.
Why it matters
Compare the full model–harness pair and its measured resource budget.
Limitation
Versioned components matter. API-equivalent task cost is not a subscription invoice or business-success cost.
Agents and task execution / % exercises

Aider polyglot benchmark

Правки кода в нескольких языках

What it measures
Coding exercise results under Aider-specific editing and test conditions.
Why it matters
Useful when selecting a model and edit format for an Aider workflow.
Limitation
Do not transfer its rank to another executor, repository or retry policy.
Agents and task execution / % resolved

SWE-bench Verified

Исправление задач в репозиториях

What it measures
Repository issue resolution judged with a specified evaluation harness.
Why it matters
A useful engineering probe before testing patches in your own repositories.
Limitation
Pin dataset, agent, model, tools and attempt budget. A pass is not production deployment readiness.
Agents and task execution / % tasks

Terminal-Bench

Задачи в терминале

What it measures
Completion of versioned tasks in a terminal environment.
Why it matters
A probe for tools, environment handling and task completion.
Limitation
Versions, timeouts, compute and harnesses change the meaning of the score.
Agents and task execution / benchmark-specific

IFBench

Следование проверяемым инструкциям

What it measures
Adherence to specified instructions under the benchmark evaluator.
Why it matters
Helps design tests for formats, constraints and structured outputs.
Limitation
Instruction compliance does not establish factual correctness or authorization.
Agents and task execution

Model × executor configuration

Конфигурация «модель × исполнитель»

What it measures
The exact model, provider, effort, harness, tools, memory and permissions used for one run.
Why it matters
Makes a result reproducible and attributable.
Limitation
A short alias or a provider logo is not a complete identity.
Agents and task execution

Harness

Среда исполнения и проверки

What it measures
Software that manages the agent loop, tool interfaces, context and evaluation.
Why it matters
The same model may behave differently under another harness.
Limitation
DSH Harness and DeepSeek Harness are distinct identities.
Agents and task execution

Turns / tool calls

Шаги модели / вызовы инструментов

What it measures
Model interaction steps and individual tool invocations; record them separately.
Why it matters
Explain time, orchestration overhead and recovery behavior.
Limitation
Fewer calls may mean efficient work or premature failure; inspect accepted outcomes.
Agents and task execution

Authorized scope

Разрешённый объём действий

What it measures
The tools, resources, data and actions explicitly available to a run.
Why it matters
Defines what autonomy can be tested and promoted.
Limitation
A benchmark score does not authorize tender submission, live trading or production deployment.

Data and extraction quality

Data and extraction quality / benchmark-specific

AA-AnalystAgent

Работа аналитика с данными

What it measures
Quantitative questions answered using spreadsheet and document workflows.
Why it matters
A probe for analytical assistants in all three projects.
Limitation
Check source lineage, units, numerical oracles and reproducibility locally.
Data and extraction quality / benchmark-specific

Long-context reasoning

Рассуждение по длинному контексту

What it measures
Correct answers grounded in long supplied material.
Why it matters
Useful for tender packs, research logs and specifications.
Limitation
Maximum context capacity does not establish reliable retrieval at every position.
Data and extraction quality

Out-of-sample / walk-forward

Новые данные / последовательная проверка

What it measures
Evaluate on untouched future periods with only information available at the decision time.
Why it matters
Tests whether a result survives outside its tuning period.
Limitation
Repeatedly selecting on the same holdout turns it into development data.

Research and reproducibility

Research and reproducibility / versioned index

AA Intelligence Index

Сводный индекс возможностей

What it measures
A versioned aggregate across several capability evaluations.
Why it matters
Use individual components to build a shortlist for a task.
Limitation
A broad ranking is not a calibrated probability of project success.
Research and reproducibility

Relative heatmap

Относительная тепловая карта

What it measures
Colors rescaled within the selected comparison group.
Why it matters
Helps inspect patterns across task groups.
Limitation
Green is not an absolute success probability; changing selected models changes colors.
Research and reproducibility

Model × harness interaction

Взаимодействие модели и harness

What it measures
A factorial comparison separates model, executor and their interaction on matched tasks.
Why it matters
Reveals which pairing works best for a particular task family.
Limitation
A strong model does not guarantee the best pair; sparse cells cannot support reliable interaction estimates.
Research and reproducibility

Champion / challenger

Текущая версия / кандидат

What it measures
A frozen baseline compared with one versioned candidate under declared gates.
Why it matters
Turns automatic generation into a controlled improvement process.
Limitation
Keep a protected holdout, all failed trials, a budget, a stopping rule and rollback evidence.
Research and reproducibility

Accuracy

Доля правильных ответов

What it measures
correct answers / all scored questions
Why it matters
Read with the same taskset, unit and budget.
Limitation
Class imbalance can hide missed rare events.
Research and reproducibility

Precision

Точность положительных находок

What it measures
true positive findings / all positive findings
Why it matters
Read with the same taskset, unit and budget.
Limitation
Precision alone does not show what was missed.
Research and reproducibility

Recall

Полнота обнаружения

What it measures
found true positives / all true positives
Why it matters
Read with the same taskset, unit and budget.
Limitation
Recall alone does not show false alarms.
Research and reproducibility

F1

Среднее гармоническое точности и полноты

What it measures
2 × precision × recall / (precision + recall)
Why it matters
Read with the same taskset, unit and budget.
Limitation
State the macro/micro weighting and zero-denominator convention.
Research and reproducibility

pass@1

Успех первой попытки

What it measures
success under a single declared attempt
Why it matters
Read with the same taskset, unit and budget.
Limitation
Pin the evaluator and the attempt budget.
Research and reproducibility

pass@k

Хотя бы один успех за k попыток

What it measures
at least one accepted solution among k attempts
Why it matters
Read with the same taskset, unit and budget.
Limitation
A higher k changes cost and does not demonstrate repeat reliability.
Research and reproducibility

pass^k / all-K

Успех во всех k повторах

What it measures
all of the fixed k repeats accepted for a task
Why it matters
Read with the same taskset, unit and budget.
Limitation
Use the task as the cluster; do not assume p^k without independence.
Research and reproducibility

TTFT / tokens per second

До первого токена / токенов в секунду

What it measures
response-start delay / generation throughput
Why it matters
Read with the same taskset, unit and budget.
Limitation
Fast output is not fast accepted completion; queues, tools and retries remain.
Research and reproducibility

Percentage points

Процентные пункты

What it measures
absolute difference between two percentages
Why it matters
Read with the same taskset, unit and budget.
Limitation
80% to 85% is +5 points and +6.25% relative, not the same change.
Research and reproducibility

Log scale

Логарифмическая шкала

What it measures
equal distances correspond to equal ratios
Why it matters
Read with the same taskset, unit and budget.
Limitation
Zero and negative values are not defined on an ordinary logarithmic axis.

Cost and resources

Cost and resources

Cost per session

Стоимость сессии

What it measures
The cost of a session under a stated length, pricing and cache condition.
Why it matters
Useful for cost planning when the session mix is known.
Limitation
A 1-turn session is not comparable with a 50-turn task; no quality is implied.
Cost and resources

API-equivalent cost

Эквивалент стоимости по API

What it measures
A token-based estimate at a stated provider tariff.
Why it matters
Supports reproducible price scenarios.
Limitation
Keep estimate, billed API amount, subscription allocation and full cost separate.
Cost and resources

Input / output / reasoning / cache tokens

Входные / выходные / reasoning / cache-токены

What it measures
Distinct billing and usage categories whose definitions depend on the provider.
Why it matters
Count retries, assistants and auxiliary models in the same cohort.
Limitation
Do not count reasoning twice when it is already included in output tokens.

Economics and efficiency

Economics and efficiency

Pareto frontier

Граница Парето

What it measures
Candidates not dominated on every chosen objective within a comparable experiment.
Why it matters
Shows quality–cost–latency trade-offs before choosing business priorities.
Limitation
Missing costs, different tasksets and statistical uncertainty can invalidate a claimed frontier.
Economics and efficiency

Point-in-time macro vintage

Историческая версия макроданных

What it measures
The value and publication date that were actually known at a past decision time.
Why it matters
Keeps macro scenarios and FinEdg research free of revised-data leakage.
Limitation
The value displayed today may differ from what the market knew then.
Economics and efficiency

Contribution / cash flow

Маржинальная прибыль / денежный поток

What it measures
Revenue minus variable cost differs from the timing of actual receipts and payments.
Why it matters
Supports viable pricing, bid/no-bid choices and budget stress scenarios.
Limitation
Business margin, trading PnL and benchmark quality have different units and must remain separate.

Probability and confidence

Probability and confidence

Wilson 95% interval

95%-й интервал Уилсона

What it measures
An interval for a Bernoulli success proportion under its sampling assumptions.
Why it matters
Shows the uncertainty hidden by a single observed percentage.
Limitation
Repeated calls on one task are not independent observations; this is not a Bayesian posterior probability.
Probability and confidence

Beta–binomial model

Бета-биномиальная модель

What it measures
Beta(a,b) prior plus k successes in n independent trials gives Beta(a+k,b+n-k).
Why it matters
Makes prior beliefs and their update explicit.
Limitation
State and test the prior; dependence and changing task difficulty require a richer model.
Probability and confidence

Expected loss

Ожидаемые потери решения

What it measures
Sum of outcome probabilities times their losses, including action and review costs.
Why it matters
Connects a calibrated prediction to a project-specific action.
Limitation
Loss assumptions must be explicit; a confidence threshold never creates permission.
Probability and confidence

Paired comparison / cluster bootstrap

Парное сравнение / кластерный bootstrap

What it measures
Compare systems on the same cases; resample independent task, company or time clusters.
Why it matters
Reduces task-mix noise and respects repeated observations.
Limitation
Separate comparisons across many candidates need a declared selection and multiplicity policy.
Probability and confidence / squared_probability

Chosen-answer correctness Brier

Brier правильности выбранного ответа

What it measures
mean_i (p_correct_i - c_i)^2; c=1 iff the chosen answer is correct
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Probability and confidence / squared_probability_sum

Multiclass vector Brier

Brier полного вектора классов

What it measures
mean_i sum_k (p_ik - 1[y_i=k])^2; sum_k p_ik=1
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Probability and confidence / bin_probabilities

Reliability diagram

Калибровочная диаграмма

What it measures
For frozen bins: count, mean forecast, observed outcome frequency and uncertainty
Why it matters
Interpret jointly with the other metrics; no universal direction.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Probability and confidence / fraction

Critical-event upper 95% bound

Верхняя граница частоты критичных ошибок

What it measures
One-sided exact binomial upper bound on independent representative episodes; zero errors: 1-alpha^(1/n)
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.

Our project metrics

Our project metrics / fraction

Accepted task rate

Доля принятых задач

What it measures
accepted unique task episodes / assigned eligible task episodes
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Accepted without intervention rate

Принято без вмешательства

What it measures
accepted episodes with no human intervention / assigned eligible episodes
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

All-K repeat reliability

Успех во всех K повторах

What it measures
mean_i product_r accepted[i,r], fixed K
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Critical violation count

Критичные нарушения

What it measures
count of predefined forbidden state transitions / critical invariant failures
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Replay integrity rate

Восстановление без потерь и повторных эффектов

What it measures
fault cases restored with same intended state and exactly-once business effect / all injected fault cases
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Trace coverage

Полнота связанного журнала

What it measures
required event links present / required event links
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Numerical oracle pass rate

Точность расчётных примеров

What it measures
cases passing all unit-specific tolerances / all cases
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Feasible solution rate

Соблюдение ограничений

What it measures
solutions satisfying every declared constraint / all fixtures
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Certified relative optimality gap

Разрыв с доказанным оптимумом

What it measures
For minimization: (candidate objective - certified optimum)/abs(certified optimum)
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Critical-field exact match

Точное извлечение критичных полей

What it measures
correct critical field values / all required critical field values
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Macro field F1

Средний F1 типов полей

What it measures
mean over declared field types of 2*precision*recall/(precision+recall)
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Entity-link precision

Точность связей компаний

What it measures
correct proposed links / all proposed links
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Entity-link recall

Полнота связей компаний

What it measures
recovered true links / all true links in adjudicated set
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Claim support precision

Подтверждённость утверждений

What it measures
claims semantically supported by cited versioned source / all asserted factual claims
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Required-fact recall

Полнота обязательных фактов

What it measures
correctly reported required facts / all required facts in gold
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Data-lineage coverage

Происхождение данных

What it measures
required fields with source version/hash/location/as-of / all required fields
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Data-leakage count

Случаи утечки данных

What it measures
violations of train/calibration/test or as-of boundaries
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Qualified conversion rate

Конверсия в целевое действие

What it measures
eligible assigned accounts with matured qualified outcome / eligible assigned accounts
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / currency/account

Contribution per assigned account

Маржинальный вклад на назначенный аккаунт

What it measures
net cohort revenue minus variable acquisition/service/refund costs, divided by all eligible assigned accounts
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / probability

Sample Ratio Mismatch p-value

Диагностика распределения групп

What it measures
goodness-of-fit p-value for observed vs planned assignment counts
Why it matters
Interpret jointly with the other metrics; no universal direction.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Unsupported material-claim count

Неподтверждённые обещания

What it measures
material promises without evidence / prohibited claim instances
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / currency/accepted_task

Cost per accepted unique task

Стоимость принятой задачи

What it measures
all relevant costs of cohort / accepted unique task episodes
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Required cost-item coverage

Полнота учёта расходов

What it measures
cost components backed by receipts or explicit allocations / required components
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / minor_currency_units

Money reconciliation error

Расхождение денежных расчётов

What it measures
max absolute residual after prescribed currency-specific rounding
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / currency/business_unit

Contribution margin per unit

Маржинальная прибыль на единицу

What it measures
net revenue excluding chosen indirect-tax treatment - variable cost
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / currency

Minimum projected cash balance

Минимальный денежный остаток сценария

What it measures
min_t(initial cash + cumulative dated receipts - cumulative dated payments)
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / currency/business_unit

Worst declared-scenario contribution

Вклад в худшем заданном сценарии

What it measures
min over declared stress scenarios of contribution
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Experiment manifest coverage

Полнота манифестов

What it measures
episodes with complete frozen config/data/criteria/cost lineage / episodes
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / minutes/accepted_task

Human review minutes per accepted task

Время участия человека

What it measures
all logged intervention and review minutes / accepted unique tasks
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / seconds

p95 time to accepted completion

95-й процентиль завершения

What it measures
p95 wall-clock elapsed time including queue/retries/review; report timeout mass separately
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Missed critical-deadline count

Пропущенные дедлайны

What it measures
missed gold critical deadlines with explicit timezone/version
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Orphaned-episode count

Задачи без конечного состояния

What it measures
finished/expired episodes lacking correlated terminal result
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Protected-holdout exposure count

Раскрытия контрольной выборки

What it measures
unauthorized development access to protected inputs/labels before final evaluation
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Independent reproduction rate

Независимое воспроизведение

What it measures
artifacts independently rerun with equivalent outputs within frozen tolerances / required reruns
Why it matters
Higher, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Material regression count

Критичные регрессии

What it measures
new failures on predeclared critical regression cases
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

Budget-contract violation count

Нарушения согласованного лимита

What it measures
episodes exceeding their explicit monetary/resource/action limits
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Out-of-sample net return

Доходность вне периода настройки

What it measures
equity_end/equity_start - 1, after recorded fees/funding/slippage under declared simulation
Why it matters
Interpret jointly with the other metrics; no universal direction.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / fraction

Maximum drawdown

Максимальная просадка

What it measures
max_t(1 - equity_t / running_peak_equity_t)
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Our project metrics / count

PAPER execution invariant failures

Ошибки PAPER-исполнения

What it measures
duplicate orders, stale prices, invalid sizing or violated approved controls in PAPER tests
Why it matters
Lower, subject to the acceptance gates.
Limitation
A proposed local metric. Freeze its denominator, scope and missing-data policy before running the experiment.
Evidence status matters.

A documented feature, a test candidate, and a measured result are different states. Session cost does not measure success, and coding-agent performance does not establish trading profitability.