{"id":"bleu-sentence-compute","version":"1.0.0","description":"Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).","supported_operations":["sentence BLEU score for one translation","smoothed sentence-level BLEU with Chen and Cherry smoothing","BLEU of a single hypothesis against reference sentences","sentence BLEU with floor epsilon smoothing","NIST exponential smoothing BLEU for one segment","add-one smoothed BLEU for a short sentence","ציון איכות תרגום למשפט בודד עם החלקה"],"unsupported_operations":["tokenizing or lowercasing raw sentences (supply tokens)","Chen and Cherry smoothing 4, 5, 6 and 7 (unsupported_input with the reason)","corpus-level BLEU over many segments (deep10-bleu-corpus)","custom n-gram weights, add-k with k other than 1, chrF or TER"],"semantics":["hypothesis is one token list and references 1 to 8 token lists; tokens are compared by exact string equality (no case folding, no Unicode normalization). max_order omitted means 4.","smoothing is required: \"none\" (no smoothing), \"method1\" (m_n = 0 becomes epsilon; epsilon 0-1, omitted means 0.1), \"method2\" (add 1 to matches and totals for n >= 2, so a 0/0 order becomes 1/1), \"method3\" (NIST: the k-th zero-match order gets 1/2^k). epsilon is accepted only with method1.","effective_order omitted means false: every order 1..max_order enters the geometric mean and an order with no hypothesis n-grams (l_n = 0 after smoothing) makes BLEU 0. With true, only the leading orders with l_n > 0 are used (sacreBLEU effective order).","BLEU is 0 when the hypothesis is empty or has no matching unigram, for every method (NLTK corpus_bleu and sacreBLEU return 0 before smoothing); otherwise bleu = BP * exp(mean of ln(m_n / l_n)) over the used orders, BP = 1 if c > r, exp(1 - r/c) otherwise, r = closest reference length (ties to the shorter).","decimals (0-15) rounds bleu with Number.prototype.toFixed (nearest, exact binary value); omitted means full double precision. Scores are on the 0-1 scale (sacreBLEU prints 100 * bleu)."],"limits":{"max_references":8,"max_total_tokens":10000,"max_token_bytes":128,"max_total_token_bytes":262144},"pricing":{"status":"unpriced","charge_usd":null},"input_schema":{"type":"object","additionalProperties":false,"required":["hypothesis","references","smoothing"],"properties":{"hypothesis":{"type":"array","items":{"type":"string","minLength":1,"maxLength":128}},"references":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"array","items":{"type":"string","minLength":1,"maxLength":128}}},"smoothing":{"type":"string","enum":["none","method1","method2","method3"]},"epsilon":{"type":"number","minimum":0,"maximum":1},"max_order":{"type":"integer","minimum":1,"maximum":8},"effective_order":{"type":"boolean"},"decimals":{"type":"integer","minimum":0,"maximum":15}}},"output_schema":{"type":"object","additionalProperties":false,"required":["bleu","smoothing","epsilon","max_order","effective_order"],"properties":{"bleu":{"type":"number","minimum":0,"maximum":1},"smoothing":{"type":"string","enum":["none","method1","method2","method3"]},"epsilon":{"type":["number","null"],"minimum":0,"maximum":1},"max_order":{"type":"integer","minimum":1,"maximum":8},"effective_order":{"type":"boolean"}}},"examples":[{"input":{"hypothesis":["this","is","a","fest"],"references":[["this","is","a","test"]],"smoothing":"method1","epsilon":0.01},"output":{"bleu":0.223606797749979,"smoothing":"method1","epsilon":0.01,"max_order":4,"effective_order":false}},{"input":{"hypothesis":["a"],"references":[["a"]],"smoothing":"method2"},"output":{"bleu":1,"smoothing":"method2","epsilon":null,"max_order":4,"effective_order":false}},{"input":{"hypothesis":["a","little","test"],"references":[["a","test"]],"smoothing":"method3","effective_order":true,"decimals":6},"output":{"bleu":0.346681,"smoothing":"method3","epsilon":null,"max_order":4,"effective_order":true}}],"execute_url":"/v1/tools/bleu-sentence-compute/versions/1.0.0/execute"}