{"id":"bleu-smoothing-compute","version":"1.0.0","description":"Apply none or Chen-Cherry smoothing 1-3 to BLEU match/total counts; return smoothed counts, BP and BLEU.","supported_operations":["BLEU score from sufficient statistics","smooth zero n-gram precisions with Chen and Cherry method 1, 2 or 3","compute BLEU from matches totals and lengths","apply add-one smoothing to higher order n-gram counts","mteval NIST geometric smoothing of zero matches","what epsilon floor smoothing does to BLEU counts","החלקת ספירות חופפות לחישוב ציון תרגום"],"unsupported_operations":["counting n-grams from text (use bleu-sentence-components first)","Chen and Cherry smoothing 4-7 (unsupported_input with the reason)","add-k with k other than 1, or custom n-gram weights","confidence intervals or significance tests"],"semantics":["matches[n-1] = m_n and totals[n-1] = l_n for n = 1..N (N = array length, 1-8), with m_n <= l_n; if m_1 = 0 every m_n must be 0. candidate_length c and reference_length r are taken as given (corpus-summed statistics are allowed).","Smoothed counts per Chen and Cherry 2014 section 2.2: none keeps them; method1 sets m_n = epsilon when m_n = 0 (epsilon 0-1, omitted means 0.1, accepted only with method1); method2 adds 1 to m_n and l_n for n >= 2 (so 0/0 becomes 1/1); method3 gives the k-th zero-match order m_n = 1/2^k. smoothed_matches and smoothed_totals report them for every order.","brevity_penalty = 0 when c = 0, 1 when c > r (including r = 0 < c), else exp(1 - r/c).","orders_used = N, or with effective_order true the leading orders whose smoothed total is > 0 (sacreBLEU effective order); effective_order omitted means false.","bleu = 0 when c = 0, m_1 = 0 (NLTK and sacreBLEU return 0 before smoothing), orders_used = 0, or a used order has smoothed total or smoothed match 0; otherwise BP * exp(mean of ln(m_n/l_n) over the used orders), summed in order n = 1..orders_used.","decimals (0-15) rounds bleu and brevity_penalty with Number.prototype.toFixed; omitted means full double precision. Smoothed counts are exact (integers, epsilon as given, or 1/2^k)."],"limits":{"max_orders":8,"max_count_value":1000000000000},"pricing":{"status":"unpriced","charge_usd":null},"input_schema":{"type":"object","additionalProperties":false,"required":["matches","totals","candidate_length","reference_length","smoothing"],"properties":{"matches":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"integer","minimum":0,"maximum":1000000000000}},"totals":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"integer","minimum":0,"maximum":1000000000000}},"candidate_length":{"type":"integer","minimum":0,"maximum":1000000000000},"reference_length":{"type":"integer","minimum":0,"maximum":1000000000000},"smoothing":{"type":"string","enum":["none","method1","method2","method3"]},"epsilon":{"type":"number","minimum":0,"maximum":1},"effective_order":{"type":"boolean"},"decimals":{"type":"integer","minimum":0,"maximum":15}}},"output_schema":{"type":"object","additionalProperties":false,"required":["bleu","brevity_penalty","orders_used","smoothed_matches","smoothed_totals"],"properties":{"bleu":{"type":"number","minimum":0,"maximum":1},"brevity_penalty":{"type":"number","minimum":0,"maximum":1},"orders_used":{"type":"integer","minimum":0,"maximum":8},"smoothed_matches":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"number","minimum":0}},"smoothed_totals":{"type":"array","minItems":1,"maxItems":8,"items":{"type":"number","minimum":0}}}},"examples":[{"input":{"matches":[9,7,5,3],"totals":[10,8,6,4],"candidate_length":11,"reference_length":11,"smoothing":"none","decimals":10},"output":{"bleu":0.8375922397,"brevity_penalty":1,"orders_used":4,"smoothed_matches":[9,7,5,3],"smoothed_totals":[10,8,6,4]}},{"input":{"matches":[1,0,0,0],"totals":[1,0,0,0],"candidate_length":1,"reference_length":1,"smoothing":"method2"},"output":{"bleu":1,"brevity_penalty":1,"orders_used":4,"smoothed_matches":[1,1,1,1],"smoothed_totals":[1,1,1,1]}},{"input":{"matches":[2,1,0,0],"totals":[2,1,0,0],"candidate_length":2,"reference_length":2,"smoothing":"method3","effective_order":true},"output":{"bleu":1,"brevity_penalty":1,"orders_used":2,"smoothed_matches":[2,1,0.5,0.25],"smoothed_totals":[2,1,0,0]}}],"execute_url":"/v1/tools/bleu-smoothing-compute/versions/1.0.0/execute"}