{"id":"chrf-components","version":"1.0.0","description":"Break down chrF/chrF++ into per-order n-gram counts, precision, recall and F, with configurable orders and averaging.","supported_operations":["chrF per-order n-gram precision and recall breakdown","show character and word n-gram match counts behind a chrF score","compute chrF with custom character and word n-gram orders","chrF with whitespace kept as a character","reproduce chrF++.py per-order F averaging exactly","explain why a chrF++ score is low by n-gram order","פירוק ציון chrF לפי סדר n-gram עם דיוק והחזרה"],"unsupported_operations":["multiple references per segment or best-reference selection","epsilon (1e-16) terms added by sacreBLEU eps_smoothing for orders without matches","non-integer beta such as 0.5","BLEU, TER or other metrics","Unicode normalization or language-specific tokenization"],"semantics":["Orders are character n = 1..char_order (default 6, range 0..10) then word n = 1..word_order (default 0, range 0..4); both 0 is invalid_input. Counts per order are summed over all segments; matching_ngrams is the clipped count sum over reference n-grams of min(reference count, hypothesis count).","Character n-grams are Unicode code point sequences; when include_whitespace is false (default) every code point in U+0009-U+000D, U+001C-U+001F, U+0020, U+0085, U+00A0, U+1680, U+2000-U+200A, U+2028, U+2029, U+202F, U+205F, U+3000 is removed first; when true the text is used as given. No Unicode normalization.","Word tokens follow Popovic chrF++.py separate_punctuation: split on runs of the same whitespace set; a one-code-point word is kept; else one trailing, otherwise one leading, ASCII string.punctuation mark is split off. Word n-grams are adjacent tokens.","Per order: precision = matching/hypothesis_ngrams (null if 0 hypothesis n-grams), recall = matching/reference_ngrams (null if 0 reference n-grams), f_score = (1+beta^2)PR/(beta^2 P+R) on a 0..1 scale (0/1 when nothing matches, null when either side has no n-grams). effective is true when both counts are positive.","average_precision and average_recall are the arithmetic means over effective orders (0/1 if none). averaging precision_recall (default) scores 100 * F_beta(average_precision, average_recall), Popovic 2015/2017 eq. (1) with sacreBLEU 2.x effective order; per_order_f scores 100 * the mean of per-order f_score over all char_order + word_order orders with null and no-match orders counted as 0, the arithmetic of Popovic chrF++.py.","beta omitted means 2; lowercase omitted means false (true applies String.prototype.toLowerCase with the runtime Unicode case mapping). score is rounded to six decimals, ties to even; all fractions are exact and reduced.","Validation order: plain object, unknown keys, missing keys, hypotheses then references (array, at most 256 segments, at least one, strings), equal segment counts, total UTF-8 bytes of all texts at most 65536, lone surrogates, char_order, word_order, beta, lowercase, include_whitespace, averaging. Inputs are never mutated."],"limits":{"max_segments":256,"max_text_bytes":65536},"pricing":{"status":"unpriced","charge_usd":null},"input_schema":{"type":"object","additionalProperties":false,"properties":{"hypotheses":{"type":"array","minItems":1,"maxItems":256,"items":{"type":"string"}},"references":{"type":"array","minItems":1,"maxItems":256,"items":{"type":"string"}},"char_order":{"type":"integer","minimum":0,"maximum":10},"word_order":{"type":"integer","minimum":0,"maximum":4},"beta":{"type":"integer","minimum":1,"maximum":16},"lowercase":{"type":"boolean"},"include_whitespace":{"type":"boolean"},"averaging":{"type":"string","enum":["precision_recall","per_order_f"]}},"required":["hypotheses","references"]},"output_schema":{"type":"object","additionalProperties":false,"properties":{"char_order":{"type":"integer","minimum":0,"maximum":10},"word_order":{"type":"integer","minimum":0,"maximum":4},"beta":{"type":"integer","minimum":1,"maximum":16},"lowercase":{"type":"boolean"},"include_whitespace":{"type":"boolean"},"averaging":{"type":"string","enum":["precision_recall","per_order_f"]},"segment_count":{"type":"integer","minimum":1,"maximum":256},"orders":{"type":"array","minItems":1,"maxItems":14,"items":{"type":"object","additionalProperties":false,"properties":{"type":{"type":"string","enum":["char","word"]},"n":{"type":"integer","minimum":1,"maximum":10},"hypothesis_ngrams":{"type":"integer","minimum":0},"reference_ngrams":{"type":"integer","minimum":0},"matching_ngrams":{"type":"integer","minimum":0},"effective":{"type":"boolean"},"precision":{"type":["string","null"]},"recall":{"type":["string","null"]},"f_score":{"type":["string","null"]}},"required":["type","n","hypothesis_ngrams","reference_ngrams","matching_ngrams","effective","precision","recall","f_score"]}},"effective_order_count":{"type":"integer","minimum":0,"maximum":14},"average_precision":{"type":"string","pattern":"^(0|[1-9][0-9]*)/[1-9][0-9]*$"},"average_recall":{"type":"string","pattern":"^(0|[1-9][0-9]*)/[1-9][0-9]*$"},"score":{"type":"string","pattern":"^(100\\.000000|[0-9]{1,2}\\.[0-9]{6})$"},"score_fraction":{"type":"string","pattern":"^(0|[1-9][0-9]*)/[1-9][0-9]*$"}},"required":["char_order","word_order","beta","lowercase","include_whitespace","averaging","segment_count","orders","effective_order_count","average_precision","average_recall","score","score_fraction"]},"examples":[{"input":{"hypotheses":["aa"],"references":["ab"],"beta":3},"output":{"char_order":6,"word_order":0,"beta":3,"lowercase":false,"include_whitespace":false,"averaging":"precision_recall","segment_count":1,"orders":[{"type":"char","n":1,"hypothesis_ngrams":2,"reference_ngrams":2,"matching_ngrams":1,"effective":true,"precision":"1/2","recall":"1/2","f_score":"1/2"},{"type":"char","n":2,"hypothesis_ngrams":1,"reference_ngrams":1,"matching_ngrams":0,"effective":true,"precision":"0/1","recall":"0/1","f_score":"0/1"},{"type":"char","n":3,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":4,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":5,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":6,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null}],"effective_order_count":2,"average_precision":"1/4","average_recall":"1/4","score":"25.000000","score_fraction":"25/1"}},{"input":{"hypotheses":["a"],"references":["a"],"beta":3,"averaging":"per_order_f"},"output":{"char_order":6,"word_order":0,"beta":3,"lowercase":false,"include_whitespace":false,"averaging":"per_order_f","segment_count":1,"orders":[{"type":"char","n":1,"hypothesis_ngrams":1,"reference_ngrams":1,"matching_ngrams":1,"effective":true,"precision":"1/1","recall":"1/1","f_score":"1/1"},{"type":"char","n":2,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":3,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":4,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":5,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null},{"type":"char","n":6,"hypothesis_ngrams":0,"reference_ngrams":0,"matching_ngrams":0,"effective":false,"precision":null,"recall":null,"f_score":null}],"effective_order_count":1,"average_precision":"1/1","average_recall":"1/1","score":"16.666667","score_fraction":"50/3"}}],"execute_url":"/v1/tools/chrf-components/versions/1.0.0/execute"}