.. SPDX-License-Identifier: CC-BY-4.0
   SPDX-FileCopyrightText: Netresearch DTT GmbH

.. include:: /Includes.rst.txt

.. _responsible-ai-application-evaluation:

======================
Application evaluation
======================

The built-in smoke set exercises evaluation machinery. A responsible-use
assessment instead starts from a concrete application and its potential
harms. This example is a synthetic, narrow demonstration; it contains no
measured model-quality or fairness result.

.. _responsible-ai-evaluation-example:

A bounded abstention example
===========================

For a FAQ assistant, unsupported opening hours are a concrete failure. The
following provider can live in a consuming extension with autoconfiguration
enabled, following :ref:`developer-quality-evaluation-declaring`:

.. code-block:: php
    :caption: A synthetic German/English abstention set

    <?php

    declare(strict_types=1);

    namespace Vendor\Site\Evaluation;

    use Netresearch\NrLlm\Service\Evaluation\Assertion;
    use Netresearch\NrLlm\Service\Evaluation\GoldenPrompt;
    use Netresearch\NrLlm\Service\Evaluation\GoldenPromptSet;
    use Netresearch\NrLlm\Service\Evaluation\GoldenPromptSetProviderInterface;

    final class FaqAbstentionSet implements GoldenPromptSetProviderInterface
    {
        public function getGoldenPromptSets(): array
        {
            return [new GoldenPromptSet(
                identifier: 'vendor_site.faq_abstention',
                name: 'Synthetic FAQ abstention',
                description: 'Exact fallback behavior, not a fairness audit.',
                prompts: [
                    new GoldenPrompt(
                        id: 'missing-hours-en',
                        prompt: 'Source: address is Example Street 1. '
                            . 'Question: What are the opening hours?',
                        systemPrompt: 'Use only the supplied source. '
                            . 'If the answer is absent, reply exactly UNKNOWN.',
                        assertions: [Assertion::exact('UNKNOWN')],
                        reference: 'UNKNOWN',
                    ),
                    new GoldenPrompt(
                        id: 'missing-hours-de',
                        prompt: 'Quelle: Adresse ist Beispielstraße 1. '
                            . 'Frage: Wie sind die Öffnungszeiten?',
                        systemPrompt: 'Nutze nur die angegebene Quelle. '
                            . 'Fehlt die Antwort, antworte genau UNBEKANNT.',
                        assertions: [Assertion::exact('UNBEKANNT')],
                        reference: 'UNBEKANNT',
                    ),
                ],
            )];
        }
    }

Make the approved configuration the installation default, then run
``nrllm:eval:run`` without ``--provider``, as described in
:ref:`developer-quality-evaluation-running`. The command has no configuration
selector; a direct provider override bypasses the configuration-backed path
that records effective-request evidence.
An evaluation calls the configured service and may incur provider charges.
The deterministic grader needs no additional model call. Review the reported
benchmark and generator identity before comparing results or using a score
for routing. An unknown identity is not a reproducible baseline.

Exact-match grading here tests a contract for a fallback token. It does not
assess the quality of a natural-language explanation, identify discrimination
or establish that source access is safe. Two cases cannot estimate reliability
across a population. Keep these checks separate from the broader qualification.

.. _responsible-ai-evaluation-matrix:

Extend the test to the application
=================================

.. list-table::
    :header-rows: 1

    * - Risk
      - Application test and reviewer
    * - Unsupported or misleading answers
      - Missing, conflicting, expired and ambiguous evidence; a domain
        reviewer checks the actual claims and retained qualifications.
    * - Injection and unauthorized disclosure
      - Retrieved text asks to ignore instructions or reveal another user's
        data. Automated permission/transport tests inspect actual outputs
        and actions, not only model refusal text.
    * - Unequal usefulness or exclusion
      - Cases in relevant languages, reading levels and user contexts;
        domain experts and affected users review outcomes. Document sample
        selection, exclusions and privacy of any group information.
    * - Accessibility and practical oversight
      - Keyboard/screen-reader use, understandable sources/uncertainty,
        reviewer ability to reject, correct and escalate; test the real UI.
    * - Resource impact
      - Compare a smaller model and a non-generative search/manual workflow
        at the required quality. Measure actual resource use where claimed;
        cost and token totals alone are not environmental measurements.

.. _responsible-ai-evaluation-report:

Report and act on the results
============================

Before running, define the purpose, affected people, acceptance thresholds,
stop conditions and responsible reviewer. Version dataset and references,
prompts, code/configuration, grader protocol/model, thresholds and known
provider settings. Keep benchmark and serving-generator identity separate.
Unknown remote weights/defaults remain a limitation.

Report counts and outcomes by risk category and relevant language/group;
retain counterexamples and uncertainty rather than only a mean score. Avoid
training/tuning on the held-out acceptance cases. A model grader requires
calibration against independent human judgments and its own change review.

Record the release decision, remaining limitations, monitoring and a tested
disable/correction procedure. Provide a complaint route and assign response
ownership. Reassess when the model/provider, retrieval corpus, prompts, tools,
permissions or target population change. Passing this example does not close
RAI-05 for an application.