# Chrf compute

`chrf-compute` · version 1.0.0 · Text · free, no key needed

Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.

**Use when you need to: compute chrF score for a translation against a reference · character n-gram F-score chrF2 for machine translation output · corpus-level chrF over aligned hypothesis and reference segments.**

## Decide before calling

Read the [versioned contract](/v1/tools/chrf-compute/versions/1.0.0) and the supported scope below. Reuse `chrf-compute@1.0.0` when your input, required output and limits match it. Choose another approach for an unsupported operation.

## Explain the choice

"I can use `chrf-compute@1.0.0` for compute chrF score for a translation against a reference. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

## Supported

- compute chrF score for a translation against a reference
- character n-gram F-score chrF2 for machine translation output
- corpus-level chrF over aligned hypothesis and reference segments
- sentence-level chrF score for each segment
- chrF3 with beta 3 for MT evaluation
- exact chrF fraction without floating-point rounding
- חישוב chrF2 לתרגום מכונה מול תרגום ייחוס

## Not supported

- chrF++ with word n-grams (use chrf-plus-plus-compute)
- multiple references per segment or best-reference selection
- chrF++.py epsilon-smoothed per-order F averaging (use chrf-components with averaging per_order_f)
- BLEU, TER, METEOR or learned metrics such as COMET
- Unicode normalization, detokenization or language-specific tokenization

## Behavior

- Score is Popovic 2015 section 2 eq. (1): chrF_beta = (1 + beta^2) * chrP * chrR / (beta^2 * chrP + chrR), with chrP and chrR the character n-gram precision and recall arithmetically averaged over n = 1..6, reported on a 0..100 scale.
- Character n-grams are sequences of Unicode code points after removing every whitespace code point: U+0009-U+000D, U+001C-U+001F, U+0020, U+0085, U+00A0, U+1680, U+2000-U+200A, U+2028, U+2029, U+202F, U+205F, U+3000 (the Python 3 str.split() set used by sacreBLEU). Other characters, including U+200B and U+FEFF, are kept. No Unicode normalization is applied, so NFC and NFD spellings differ.
- Clipped matches per order are sum over reference n-grams of min(reference count, hypothesis count). An order enters the averages only when both the hypothesis and the reference have at least one n-gram of that order (sacreBLEU 2.x effective order). If no order is effective, or both averages are 0, the score is 0.
- Corpus score: per-order counts are summed over all segments before averaging. segment_scores apply the same rule to each segment alone, in input order.
- beta is an integer 1..16; omitted means 2 (chrF2, the sacreBLEU and Popovic 2017 default). lowercase omitted means false (case-sensitive); true applies String.prototype.toLowerCase (Unicode default full case mapping from the runtime) to both texts first.
- score is the exact value rounded to six decimals, ties to even; score_fraction is the exact reduced fraction n/d on the 0..100 scale (0/1 and 100/1 at the ends).
- Validation order: plain object, unknown keys, missing keys, hypotheses then references (array, at most 256 segments, at least one, strings), equal segment counts, total UTF-8 bytes of all texts at most 65536, lone surrogates, beta, lowercase. Inputs are never mutated.

## Input

- `hypotheses` (array of string, required): min items 1; max items 256
- `references` (array of string, required): min items 1; max items 256
- `beta` (integer, optional): min 1; max 16
- `lowercase` (boolean, optional)

## Output

- `metric` (constant "chrF", required)
- `char_order` (constant 6, required)
- `word_order` (constant 0, required)
- `beta` (integer, required): min 1; max 16
- `lowercase` (boolean, required)
- `segment_count` (integer, required): min 1; max 256
- `score` (string, required): pattern `^(100\.000000|[0-9]{1,2}\.[0-9]{6})$`
- `score_fraction` (string, required): pattern `^(0|[1-9][0-9]*)/[1-9][0-9]*$`
- `segment_scores` (array of string, required): min items 1; max items 256; each pattern `^(100\.000000|[0-9]{1,2}\.[0-9]{6})$`

## Limits

- max segments: 256
- max text bytes: 65536

## Example

Request input:

```json
{
  "hypotheses": [
    "The relationship between cats and dogs is not exactly friendly.",
    "a good bookshop is just a genteel black hole that knows how to read."
  ],
  "references": [
    "The relationship between dogs and cats is not exactly friendly.",
    "A good bookshop is just a genteel Black Hole that knows how to read."
  ]
}
```

Response:

```json
{
  "result": {
    "metric": "chrF",
    "char_order": 6,
    "word_order": 0,
    "beta": 2,
    "lowercase": false,
    "segment_count": 2,
    "score": "84.642149",
    "score_fraction": "7116866012380/84081820977",
    "segment_scores": [
      "84.411311",
      "84.867055"
    ]
  }
}
```

## How to call it

### MCP

Connect `https://computefirst.net/mcp` ([setup](/docs#connect)), then call `execute` with:

```json
{
  "id": "chrf-compute",
  "version": "1.0.0",
  "input": {
    "hypotheses": [
      "The relationship between cats and dogs is not exactly friendly.",
      "a good bookshop is just a genteel black hole that knows how to read."
    ],
    "references": [
      "The relationship between dogs and cats is not exactly friendly.",
      "A good bookshop is just a genteel Black Hole that knows how to read."
    ]
  }
}
```

### HTTP (no key)

```sh
curl -X POST https://computefirst.net/v1/tools/chrf-compute/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"hypotheses":["The relationship between cats and dogs is not exactly friendly.","a good bookshop is just a genteel black hole that knows how to read."],"references":["The relationship between dogs and cats is not exactly friendly.","A good bookshop is just a genteel Black Hole that knows how to read."]}'
```

The machine-readable contract is at [/v1/tools/chrf-compute/versions/1.0.0](/v1/tools/chrf-compute/versions/1.0.0).

### CLI

```sh
node cli.mjs run chrf-compute 1.0.0 --input input.json --base-url https://computefirst.net
```

Get the client at [/clients/cli/](/clients/cli/).

## Related tools

- [Chrf plus plus compute](/tools/chrf-plus-plus-compute): Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored.
- [Bleu corpus compute](/tools/bleu-corpus-compute): Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.
- [Bleu sentence compute](/tools/bleu-sentence-compute): Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).
- [Bleu ngrams count](/tools/bleu-ngrams-count): Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.
- [Chrf components](/tools/chrf-components): Break down chrF/chrF++ into per-order n-gram counts, precision, recall and F, with configurable orders and averaging.
- [Bleu sentence components](/tools/bleu-sentence-components): Extract sentence BLEU statistics: clipped matches, n-gram totals, candidate and closest reference length.