{"id":"bleu-ngrams-count","version":"1.0.0","description":"Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.","supported_operations":["modified n-gram precision numerator and denominator","clipped n-gram match counts against several references","BLEU sufficient statistics per n-gram order","max reference count clipping table for each n-gram","how many bigrams of the hypothesis match the references","per-segment n-gram match counts for a translation corpus","ספירת רצפי מילים חופפים מול תרגומי ייחוס"],"unsupported_operations":["computing the BLEU score or brevity penalty (use bleu-corpus-compute or bleu-brevity-penalty)","tokenizing, lowercasing or normalizing raw text before counting","recall-oriented or chrF character n-gram statistics","skip-grams or n-grams that cross segment boundaries"],"semantics":["Input is pre-tokenized; tokens are compared by exact string equality with no case folding or Unicode normalization. N-grams never cross segment boundaries.","For each order n = 1..max_order (omitted means 4): count(g) is the occurrences of n-gram g in the hypothesis, max_reference_count(g) the largest count of g in any single reference, clipped_count(g) = min(count, max_reference_count) (Papineni 2002 section 2.1, footnote 2).","matches[n-1] sums clipped counts and totals[n-1] sums max(0, len - n + 1) over all segments (section 2.1.1); a hypothesis shorter than n contributes 0 to both.","detail \"totals\" (default when omitted) returns only the corpus sums; \"segments\" adds one row per segment in input order; \"ngrams\" also adds one row per distinct hypothesis n-gram, ordered by segment, then order, then first occurrence in the hypothesis.","Output larger than 1,000,000 UTF-8 bytes of JSON is limit_exceeded; ask for a coarser detail level or split the corpus."],"limits":{"max_segments":1000,"max_references_per_segment":8,"max_total_tokens":50000,"max_token_bytes":128,"max_total_token_bytes":1048576,"max_output_bytes":1000000},"pricing":{"status":"unpriced","charge_usd":null},"input_schema":{"type":"object","additionalProperties":false,"required":["hypotheses","references"],"properties":{"hypotheses":{"type":"array","minItems":1,"maxItems":1000,"items":{"type":"array","items":{"type":"string","minLength":1,"maxLength":128}}},"references":{"type":"array","minItems":1,"maxItems":1000,"items":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"array","items":{"type":"string","minLength":1,"maxLength":128}}}},"max_order":{"type":"integer","minimum":1,"maximum":8},"detail":{"type":"string","enum":["totals","segments","ngrams"]}}},"output_schema":{"type":"object","additionalProperties":false,"required":["max_order","matches","totals"],"properties":{"max_order":{"type":"integer","minimum":1,"maximum":8},"matches":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"integer","minimum":0}},"totals":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"integer","minimum":0}},"segments":{"type":"array","items":{"type":"object","additionalProperties":false,"required":["segment","candidate_length","matches","totals"],"properties":{"segment":{"type":"integer","minimum":0},"candidate_length":{"type":"integer","minimum":0},"matches":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"integer","minimum":0}},"totals":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"integer","minimum":0}}}}},"ngrams":{"type":"array","items":{"type":"object","additionalProperties":false,"required":["segment","order","ngram","count","max_reference_count","clipped_count"],"properties":{"segment":{"type":"integer","minimum":0},"order":{"type":"integer","minimum":1,"maximum":8},"ngram":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"string"}},"count":{"type":"integer","minimum":1},"max_reference_count":{"type":"integer","minimum":0},"clipped_count":{"type":"integer","minimum":0}}}}}},"examples":[{"input":{"hypotheses":[["of","the"]],"references":[[["It","is","a","guide","to","action","that","ensures","that","the","military","will","forever","heed","Party","commands"],["It","is","the","guiding","principle","which","guarantees","the","military","forces","always","being","under","the","command","of","the","Party"],["It","is","the","practical","guide","for","the","army","always","to","heed","the","directions","of","the","party"]]],"max_order":2},"output":{"max_order":2,"matches":[2,1],"totals":[2,1]}},{"input":{"hypotheses":[["the","the","the","the","the","the","the"]],"references":[[["the","cat","is","on","the","mat"],["there","is","a","cat","on","the","mat"]]],"max_order":2,"detail":"ngrams"},"output":{"max_order":2,"matches":[2,0],"totals":[7,6],"segments":[{"segment":0,"candidate_length":7,"matches":[2,0],"totals":[7,6]}],"ngrams":[{"segment":0,"order":1,"ngram":["the"],"count":7,"max_reference_count":2,"clipped_count":2},{"segment":0,"order":2,"ngram":["the","the"],"count":6,"max_reference_count":0,"clipped_count":0}]}}],"execute_url":"/v1/tools/bleu-ngrams-count/versions/1.0.0/execute"}