---
title: "ADR-220: Evaluation results name their generator"
manual: "LLM Foundation"
version: "0.41"
permalink: "https://docs.typo3.org/permalink/netresearch/nr-llm:adr-220@0.41"
source: "Adr/Adr220EvaluationResultsNameTheirGenerator.rst"
rendered: "2026-10-20T08:48:47+00:00"
---

# ADR-220: Evaluation results name their generator {#adr-220}

-   *Status:* Accepted
-   *Date:* 2026-10-10
-   *Amends:*

    [ADR-060](https://docs.typo3.org/permalink/netresearch/nr-llm:adr-060@0.41) (model-specific results require one
    consistently reported generator identity)

-   *Authors:* Netresearch DTT GmbH

## Context {#context}

A golden-set run may use a fallback model. EvaluationService previously
overwrote its run model with each response and stored every prompt's score
under the final model. The real quality-routing consumer then attributed a
primary model's failure to the fallback that answered another prompt.

The grading yardstick and the generator are separate facts. Deterministic
assertions can fairly grade answers from several models, while their
aggregate is not a measurement of any one model on the complete set.

## Decision {#decision}

1.  Distinguish the DB provider instance from the adapter key returned in
    `CompletionResponse.provider`. After a configuration-driven terminal
    succeeds, its actual resolved DB provider, final outbound model alias
    and response-reported model become reserved additive response metadata.
    The actual fallback terminal supplies its own identity. Requested or
    primary configuration values are not serving evidence.

    The terminal overwrites any adapter-supplied reserved value and preserves
    the rest of the response. Cache/idempotency replay preserves the original
    served identity. Adapter-only paths without a DB instance remain unknown.
    The existing public response constructor remains unchanged.
1.  Keep this provenance in prompt details and identify a run only when all
    prompts share its DB provider instance, actual outbound alias and
    reported model. The outbound alias is the homogeneous run's model key;
    its reported model is a separate fact, including for snapshot names.
    Mixed, partly unknown and empty runs keep an empty run model, their
    grades and aggregate. No subset becomes a full-set measurement.
1.  Keep grading comparability independent. `sharesOneYardstick()` still
    checks graders. A run without one generator is recorded but cannot
    provide a model-specific regression or quality-routing signal. The
    strict CLI gate fails when such a comparison cannot be made.
1.  Keep versioned, content-free generator provenance in a dedicated column
    of the existing result table. It survives metadata privacy separately
    from the filtered details. Absent, malformed, unsupported-version or
    row-inconsistent provenance supplies no eligibility evidence. Retention
    removes it with the result. Retrieval provenance retains its own column.
1.  Core quality reads and generator baselines require verified provenance,
    including provider instance and outbound model alias. Baselines also
    require the same reported model, explicitly passed to the scoped read
    and filtered before choosing its latest row. A newer different reported
    snapshot cannot mask an earlier matching baseline. Add optional scoped
    capabilities beside the existing repository and quality-provider interfaces,
    without adding
    required members to those contracts. Routing uses the capability when
    present; custom model-ID-only sources retain their existing contract.
    The core adapter treats old repositories without verified reads as an
    unknown signal. An unscoped core read is unknown when several verified
    providers share the same model name.
1.  Preserve historical aggregate/history reads. Legacy records are
    unverified and cannot drive core quality routing or a new generator
    baseline. Do not backfill invented evidence. Trailing optional internal
    DTO fields preserve existing constructors; no frozen `@api` contract
    changes.

## Consequences {#consequences}

Homogeneous runs retain their established behavior. Mixed runs remain
useful aggregate measurements but cannot be presented as the final model's
quality. Reported identifiers describe the provider response; mutable
model aliases do not establish fixed weights.

Old rows kept only the collapsed model identity. They remain readable but
stop contributing to core quality routing because they cannot be corrected
reliably from that value, including when details were removed by privacy.
Operators update the database schema and re-evaluate relevant model/set
series to restore verified quality signals.

The specification is `specs/018-evaluation-generator-attribution/spec.md`.