# Chrf plus plus compute `chrf-plus-plus-compute` · version 1.0.0 · Text · free, no key needed Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored. **Use when you need to: compute chrF++ score for machine translation output · chrF++ with character 6-grams and word bigrams against a reference · chrF2++ corpus score over aligned segments.** ## Decide before calling Read the [versioned contract](/v1/tools/chrf-plus-plus-compute/versions/1.0.0) and the supported scope below. Reuse `chrf-plus-plus-compute@1.0.0` when your input, required output and limits match it. Choose another approach for an unsupported operation. ## Explain the choice "I can use `chrf-plus-plus-compute@1.0.0` for compute chrF++ score for machine translation output. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured." ## Supported - compute chrF++ score for machine translation output - chrF++ with character 6-grams and word bigrams against a reference - chrF2++ corpus score over aligned segments - sentence-level chrF++ score per segment - show the chrF++ word tokens after punctuation splitting - case-insensitive chrF++ with lowercasing - ציון chrF++ עם bigrams של מילים לתרגום ## Not supported - plain chrF without word n-grams (use chrf-compute) - configurable n-gram orders or keeping whitespace (use chrf-components) - multiple references per segment (chrF++.py "*#" syntax) or best-reference selection - language-specific word tokenization such as Moses, 13a or CJK segmentation - non-ASCII punctuation splitting such as U+3002 or U+00BF ## Behavior - Score is Popovic 2017 section 2 eq. (1): F_beta = (1 + beta^2) * P * R / (beta^2 * P + R), where P and R are the precisions and recalls of character n-grams n = 1..6 and word n-grams n = 1..2 arithmetically averaged together (8 orders), reported on a 0..100 scale. - Character n-grams are Unicode code point sequences of the text with every whitespace code point removed: U+0009-U+000D, U+001C-U+001F, U+0020, U+0085, U+00A0, U+1680, U+2000-U+200A, U+2028, U+2029, U+202F, U+205F, U+3000. No Unicode normalization is applied. - Word tokens follow separate_punctuation in Popovic chrF++.py (sacreBLEU _remove_punctuation): split the text on runs of the same whitespace set; a word of one code point is kept; otherwise if its last code point is one of the 32 ASCII punctuation characters !"#$%&'()*+,-./:;<=>?@[\]^_`{|}~ it becomes [rest, mark]; else if its first code point is one, [mark, rest]; else it is kept. At most one mark is split per word, so "(hi)" gives "(hi" and ")", and "..." gives ".." and ".". - Clipped matches per order are sum over reference n-grams of min(reference count, hypothesis count); word bigrams are adjacent token pairs. An order enters the averages only when both sides have at least one n-gram of that order (sacreBLEU 2.x effective order); if none does, or both averages are 0, the score is 0. - Corpus score sums per-order counts over all segments; segment_scores and word_tokens are per segment in input order. beta omitted means 2; lowercase omitted means false, and true applies String.prototype.toLowerCase (runtime Unicode default case mapping) before tokenization, so word_tokens are lowercased too. - score is the exact value rounded to six decimals, ties to even; score_fraction is the exact reduced fraction n/d on the 0..100 scale. - Validation order: plain object, unknown keys, missing keys, hypotheses then references (array, at most 256 segments, at least one, strings), equal segment counts, total UTF-8 bytes of all texts at most 65536, lone surrogates, beta, lowercase. Inputs are never mutated. ## Input - `hypotheses` (array of string, required): min items 1; max items 256 - `references` (array of string, required): min items 1; max items 256 - `beta` (integer, optional): min 1; max 16 - `lowercase` (boolean, optional) ## Output - `metric` (constant "chrF++", required) - `char_order` (constant 6, required) - `word_order` (constant 2, required) - `beta` (integer, required): min 1; max 16 - `lowercase` (boolean, required) - `segment_count` (integer, required): min 1; max 256 - `score` (string, required): pattern `^(100\.000000|[0-9]{1,2}\.[0-9]{6})$` - `score_fraction` (string, required): pattern `^(0|[1-9][0-9]*)/[1-9][0-9]*$` - `segment_scores` (array of string, required): min items 1; max items 256; each pattern `^(100\.000000|[0-9]{1,2}\.[0-9]{6})$` - `word_tokens` (array of object, required): min items 1; max items 256 ## Limits - max segments: 256 - max text bytes: 65536 ## Example Request input: ```json { "hypotheses": [ "The relationship between cats and dogs is not exactly friendly.", "a good bookshop is just a genteel black hole that knows how to read." ], "references": [ "The relationship between dogs and cats is not exactly friendly.", "A good bookshop is just a genteel Black Hole that knows how to read." ] } ``` Response: ```json { "result": { "metric": "chrF++", "char_order": 6, "word_order": 2, "beta": 2, "lowercase": false, "segment_count": 2, "score": "82.872637", "score_fraction": "40260030811255/485806076756", "segment_scores": [ "83.308483", "82.578862" ], "word_tokens": [ { "hypothesis": [ "The", "relationship", "between", "cats", "and", "dogs", "is", "not", "exactly", "friendly", "." ], "reference": [ "The", "relationship", "between", "dogs", "and", "cats", "is", "not", "exactly", "friendly", "." ] }, { "hypothesis": [ "a", "good", "bookshop", "is", "just", "a", "genteel", "black", "hole", "that", "knows", "how", "to", "read", "." ], "reference": [ "A", "good", "bookshop", "is", "just", "a", "genteel", "Black", "Hole", "that", "knows", "how", "to", "read", "." ] } ] } } ``` ## How to call it ### MCP Connect `https://computefirst.net/mcp` ([setup](/docs#connect)), then call `execute` with: ```json { "id": "chrf-plus-plus-compute", "version": "1.0.0", "input": { "hypotheses": [ "The relationship between cats and dogs is not exactly friendly.", "a good bookshop is just a genteel black hole that knows how to read." ], "references": [ "The relationship between dogs and cats is not exactly friendly.", "A good bookshop is just a genteel Black Hole that knows how to read." ] } } ``` ### HTTP (no key) ```sh curl -X POST https://computefirst.net/v1/tools/chrf-plus-plus-compute/versions/1.0.0/execute \ -H "Content-Type: application/json" \ -d '{"hypotheses":["The relationship between cats and dogs is not exactly friendly.","a good bookshop is just a genteel black hole that knows how to read."],"references":["The relationship between dogs and cats is not exactly friendly.","A good bookshop is just a genteel Black Hole that knows how to read."]}' ``` The machine-readable contract is at [/v1/tools/chrf-plus-plus-compute/versions/1.0.0](/v1/tools/chrf-plus-plus-compute/versions/1.0.0). ### CLI ```sh node cli.mjs run chrf-plus-plus-compute 1.0.0 --input input.json --base-url https://computefirst.net ``` Get the client at [/clients/cli/](/clients/cli/). ## Related tools - [Chrf compute](/tools/chrf-compute): Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment. - [Bleu corpus compute](/tools/bleu-corpus-compute): Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts. - [Bleu sentence compute](/tools/bleu-sentence-compute): Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3). - [Bleu ngrams count](/tools/bleu-ngrams-count): Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables. - [Chrf components](/tools/chrf-components): Break down chrF/chrF++ into per-order n-gram counts, precision, recall and F, with configurable orders and averaging. - [Bleu brevity penalty](/tools/bleu-brevity-penalty): Compute the BLEU corpus brevity penalty from candidate and reference lengths using closest reference lengths.