Chrf components

chrf-components · version 1.0.0 · Text · free, no key needed

Break down chrF/chrF++ into per-order n-gram counts, precision, recall and F, with configurable orders and averaging.

Use when you need to: chrF per-order n-gram precision and recall breakdown · show character and word n-gram match counts behind a chrF score · compute chrF with custom character and word n-gram orders.

Decide before calling

Read the versioned contract and the supported scope below. Reuse chrf-components@1.0.0 when your input, required output and limits match it. Choose another approach for an unsupported operation.

Explain the choice

"I can use chrf-components@1.0.0 for chrF per-order n-gram precision and recall breakdown. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

Supported

• chrF per-order n-gram precision and recall breakdown
• show character and word n-gram match counts behind a chrF score
• compute chrF with custom character and word n-gram orders
• chrF with whitespace kept as a character
• reproduce chrF++.py per-order F averaging exactly
• explain why a chrF++ score is low by n-gram order
• פירוק ציון chrF לפי סדר n-gram עם דיוק והחזרה

Not supported

• multiple references per segment or best-reference selection
• epsilon (1e-16) terms added by sacreBLEU eps_smoothing for orders without matches
• non-integer beta such as 0.5
• BLEU, TER or other metrics
• Unicode normalization or language-specific tokenization

Behavior

• Orders are character n = 1..char_order (default 6, range 0..10) then word n = 1..word_order (default 0, range 0..4); both 0 is invalid_input. Counts per order are summed over all segments; matching_ngrams is the clipped count sum over reference n-grams of min(reference count, hypothesis count).
• Character n-grams are Unicode code point sequences; when include_whitespace is false (default) every code point in U+0009-U+000D, U+001C-U+001F, U+0020, U+0085, U+00A0, U+1680, U+2000-U+200A, U+2028, U+2029, U+202F, U+205F, U+3000 is removed first; when true the text is used as given. No Unicode normalization.
• Word tokens follow Popovic chrF++.py separate_punctuation: split on runs of the same whitespace set; a one-code-point word is kept; else one trailing, otherwise one leading, ASCII string.punctuation mark is split off. Word n-grams are adjacent tokens.
• Per order: precision = matching/hypothesis_ngrams (null if 0 hypothesis n-grams), recall = matching/reference_ngrams (null if 0 reference n-grams), f_score = (1+beta^2)PR/(beta^2 P+R) on a 0..1 scale (0/1 when nothing matches, null when either side has no n-grams). effective is true when both counts are positive.
• average_precision and average_recall are the arithmetic means over effective orders (0/1 if none). averaging precision_recall (default) scores 100 * F_beta(average_precision, average_recall), Popovic 2015/2017 eq. (1) with sacreBLEU 2.x effective order; per_order_f scores 100 * the mean of per-order f_score over all char_order + word_order orders with null and no-match orders counted as 0, the arithmetic of Popovic chrF++.py.
• beta omitted means 2; lowercase omitted means false (true applies String.prototype.toLowerCase with the runtime Unicode case mapping). score is rounded to six decimals, ties to even; all fractions are exact and reduced.
• Validation order: plain object, unknown keys, missing keys, hypotheses then references (array, at most 256 segments, at least one, strings), equal segment counts, total UTF-8 bytes of all texts at most 65536, lone surrogates, char_order, word_order, beta, lowercase, include_whitespace, averaging. Inputs are never mutated.

Input

• hypotheses (array of string, required): min items 1; max items 256
• references (array of string, required): min items 1; max items 256
• char_order (integer, optional): min 0; max 10
• word_order (integer, optional): min 0; max 4
• beta (integer, optional): min 1; max 16
• lowercase (boolean, optional)
• include_whitespace (boolean, optional)
• averaging (one of "precision_recall", "per_order_f", optional)

Output

• char_order (integer, required): min 0; max 10
• word_order (integer, required): min 0; max 4
• beta (integer, required): min 1; max 16
• lowercase (boolean, required)
• include_whitespace (boolean, required)
• averaging (one of "precision_recall", "per_order_f", required)
• segment_count (integer, required): min 1; max 256
• orders (array of object, required): min items 1; max items 14
• effective_order_count (integer, required): min 0; max 14
• average_precision (string, required): pattern ^(0|[1-9][0-9]*)/[1-9][0-9]*$
• average_recall (string, required): pattern ^(0|[1-9][0-9]*)/[1-9][0-9]*$
• score (string, required): pattern ^(100\.000000|[0-9]{1,2}\.[0-9]{6})$
• score_fraction (string, required): pattern ^(0|[1-9][0-9]*)/[1-9][0-9]*$

Limits

• max segments: 256
• max text bytes: 65536

Example

Request input:

{
  "hypotheses": [
    "aa"
  ],
  "references": [
    "ab"
  ],
  "beta": 3
}

Response:

{
  "result": {
    "char_order": 6,
    "word_order": 0,
    "beta": 3,
    "lowercase": false,
    "include_whitespace": false,
    "averaging": "precision_recall",
    "segment_count": 1,
    "orders": [
      {
        "type": "char",
        "n": 1,
        "hypothesis_ngrams": 2,
        "reference_ngrams": 2,
        "matching_ngrams": 1,
        "effective": true,
        "precision": "1/2",
        "recall": "1/2",
        "f_score": "1/2"
      },
      {
        "type": "char",
        "n": 2,
        "hypothesis_ngrams": 1,
        "reference_ngrams": 1,
        "matching_ngrams": 0,
        "effective": true,
        "precision": "0/1",
        "recall": "0/1",
        "f_score": "0/1"
      },
      {
        "type": "char",
        "n": 3,
        "hypothesis_ngrams": 0,
        "reference_ngrams": 0,
        "matching_ngrams": 0,
        "effective": false,
        "precision": null,
        "recall": null,
        "f_score": null
      },
      {
        "type": "char",
        "n": 4,
        "hypothesis_ngrams": 0,
        "reference_ngrams": 0,
        "matching_ngrams": 0,
        "effective": false,
        "precision": null,
        "recall": null,
        "f_score": null
      },
      {
        "type": "char",
        "n": 5,
        "hypothesis_ngrams": 0,
        "reference_ngrams": 0,
        "matching_ngrams": 0,
        "effective": false,
        "precision": null,
        "recall": null,
        "f_score": null
      },
      {
        "type": "char",
        "n": 6,
        "hypothesis_ngrams": 0,
        "reference_ngrams": 0,
        "matching_ngrams": 0,
        "effective": false,
        "precision": null,
        "recall": null,
        "f_score": null
      }
    ],
    "effective_order_count": 2,
    "average_precision": "1/4",
    "average_recall": "1/4",
    "score": "25.000000",
    "score_fraction": "25/1"
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "chrf-components",
  "version": "1.0.0",
  "input": {
    "hypotheses": [
      "aa"
    ],
    "references": [
      "ab"
    ],
    "beta": 3
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/chrf-components/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"hypotheses":["aa"],"references":["ab"],"beta":3}'

The machine-readable contract is at /v1/tools/chrf-components/versions/1.0.0.

CLI

node cli.mjs run chrf-components 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Related tools

• Chrf compute: Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.
• Chrf plus plus compute: Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored.
• Bleu ngrams count: Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.
• Bleu corpus compute: Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.
• Bleu smoothing compute: Apply none or Chen-Cherry smoothing 1-3 to BLEU match/total counts; return smoothed counts, BP and BLEU.
• Text shingle similarity: Exact shingle-set resemblance (Jaccard) and containment of two texts, with counts and shared shingles.