---
sidebar_label: Answer Relevance
description: 'Score LLM response relevance and completeness against user queries using sophisticated AI-powered evaluation metrics'
---

# Answer Relevance

The `answer-relevance` assertion evaluates whether an LLM's output is relevant to the original query. It uses a combination of embedding similarity and LLM evaluation to determine relevance.

### How to use it

To use the `answer-relevance` assertion type, add it to your test configuration like this:

```yaml
assert:
  - type: answer-relevance
    threshold: 0.7 # Score between 0 and 1
```

If `threshold` is omitted, it defaults to `0.5`.

### How it works

The answer relevance checker:

1. Uses three independent LLM calls to generate potential questions that the output could be answering. Each sample is cached separately and reused on later runs. Deterministic graders can return the same question for multiple samples.
2. Compares these questions with the original query using embedding similarity
3. Calculates a relevance score based on the similarity scores

A higher threshold requires the output to be more closely related to the original query.

### Example Configuration

Here's a complete example showing how to use answer relevance:

```yaml
prompts:
  - 'Tell me about {{topic}}'
providers:
  - openai:gpt-5
tests:
  - vars:
      topic: quantum computing
    assert:
      - type: answer-relevance
        threshold: 0.8
```

### Overriding the Providers

Answer relevance uses two types of providers:

- A text provider for generating questions
- An embedding provider for calculating similarity

You can override either or both:

```yaml
defaultTest:
  options:
    provider:
      text:
        id: gpt-5
        config:
          temperature: 0
      embedding:
        id: openai:text-embedding-ada-002
```

You can also override providers at the assertion level:

```yaml
assert:
  - type: answer-relevance
    threshold: 0.8
    provider:
      text: anthropic:claude-sonnet-5
      embedding: cohere:embedding:embed-english-v3.0
```

### Customizing the Prompt

You can customize the question generation prompt using the `rubricPrompt` property:

```yaml
defaultTest:
  options:
    rubricPrompt: |
      Given this answer: {{output}}

      Generate 3 questions that this answer would be appropriate for.
      Make the questions specific and directly related to the content.
```

# Further reading

See [model-graded metrics](/docs/configuration/expected-outputs/model-graded) for more options.