Bleu smoothing compute

bleu-smoothing-compute · version 1.0.0 · Text · free, no key needed

Apply none or Chen-Cherry smoothing 1-3 to BLEU match/total counts; return smoothed counts, BP and BLEU.

Use when you need to: BLEU score from sufficient statistics · smooth zero n-gram precisions with Chen and Cherry method 1, 2 or 3 · compute BLEU from matches totals and lengths.

Decide before calling

Read the versioned contract and the supported scope below. Reuse bleu-smoothing-compute@1.0.0 when your input, required output and limits match it. Choose another approach for an unsupported operation.

Explain the choice

"I can use bleu-smoothing-compute@1.0.0 for BLEU score from sufficient statistics. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

Supported

• BLEU score from sufficient statistics
• smooth zero n-gram precisions with Chen and Cherry method 1, 2 or 3
• compute BLEU from matches totals and lengths
• apply add-one smoothing to higher order n-gram counts
• mteval NIST geometric smoothing of zero matches
• what epsilon floor smoothing does to BLEU counts
• החלקת ספירות חופפות לחישוב ציון תרגום

Not supported

• counting n-grams from text (use bleu-sentence-components first)
• Chen and Cherry smoothing 4-7 (unsupported_input with the reason)
• add-k with k other than 1, or custom n-gram weights
• confidence intervals or significance tests

Behavior

• matches[n-1] = m_n and totals[n-1] = l_n for n = 1..N (N = array length, 1-8), with m_n <= l_n; if m_1 = 0 every m_n must be 0. candidate_length c and reference_length r are taken as given (corpus-summed statistics are allowed).
• Smoothed counts per Chen and Cherry 2014 section 2.2: none keeps them; method1 sets m_n = epsilon when m_n = 0 (epsilon 0-1, omitted means 0.1, accepted only with method1); method2 adds 1 to m_n and l_n for n >= 2 (so 0/0 becomes 1/1); method3 gives the k-th zero-match order m_n = 1/2^k. smoothed_matches and smoothed_totals report them for every order.
• brevity_penalty = 0 when c = 0, 1 when c > r (including r = 0 < c), else exp(1 - r/c).
• orders_used = N, or with effective_order true the leading orders whose smoothed total is > 0 (sacreBLEU effective order); effective_order omitted means false.
• bleu = 0 when c = 0, m_1 = 0 (NLTK and sacreBLEU return 0 before smoothing), orders_used = 0, or a used order has smoothed total or smoothed match 0; otherwise BP * exp(mean of ln(m_n/l_n) over the used orders), summed in order n = 1..orders_used.
• decimals (0-15) rounds bleu and brevity_penalty with Number.prototype.toFixed; omitted means full double precision. Smoothed counts are exact (integers, epsilon as given, or 1/2^k).

Input

• matches (array of integer, required): min items 1; max items 8; each min 0; each max 1000000000000
• totals (array of integer, required): min items 1; max items 8; each min 0; each max 1000000000000
• candidate_length (integer, required): min 0; max 1000000000000
• reference_length (integer, required): min 0; max 1000000000000
• smoothing (one of "none", "method1", "method2", "method3", required)
• epsilon (number, optional): min 0; max 1
• effective_order (boolean, optional)
• decimals (integer, optional): min 0; max 15

Output

• bleu (number, required): min 0; max 1
• brevity_penalty (number, required): min 0; max 1
• orders_used (integer, required): min 0; max 8
• smoothed_matches (array of number, required): min items 1; max items 8; each min 0
• smoothed_totals (array of number, required): min items 1; max items 8; each min 0

Limits

• max orders: 8
• max count value: 1000000000000

Example

Request input:

{
  "matches": [
    9,
    7,
    5,
    3
  ],
  "totals": [
    10,
    8,
    6,
    4
  ],
  "candidate_length": 11,
  "reference_length": 11,
  "smoothing": "none",
  "decimals": 10
}

Response:

{
  "result": {
    "bleu": 0.8375922397,
    "brevity_penalty": 1,
    "orders_used": 4,
    "smoothed_matches": [
      9,
      7,
      5,
      3
    ],
    "smoothed_totals": [
      10,
      8,
      6,
      4
    ]
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "bleu-smoothing-compute",
  "version": "1.0.0",
  "input": {
    "matches": [
      9,
      7,
      5,
      3
    ],
    "totals": [
      10,
      8,
      6,
      4
    ],
    "candidate_length": 11,
    "reference_length": 11,
    "smoothing": "none",
    "decimals": 10
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/bleu-smoothing-compute/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"matches":[9,7,5,3],"totals":[10,8,6,4],"candidate_length":11,"reference_length":11,"smoothing":"none","decimals":10}'

The machine-readable contract is at /v1/tools/bleu-smoothing-compute/versions/1.0.0.

CLI

node cli.mjs run bleu-smoothing-compute 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Related tools

• Bleu sentence compute: Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).
• Bleu corpus compute: Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.
• Bleu ngrams count: Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.
• Bleu sentence components: Extract sentence BLEU statistics: clipped matches, n-gram totals, candidate and closest reference length.
• Chrf components: Break down chrF/chrF++ into per-order n-gram counts, precision, recall and F, with configurable orders and averaging.
• Chrf compute: Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.