Bleu sentence compute

bleu-sentence-compute · version 1.0.0 · Text · free, no key needed

Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).

Use when you need to: sentence BLEU score for one translation · smoothed sentence-level BLEU with Chen and Cherry smoothing · BLEU of a single hypothesis against reference sentences.

Decide before calling

Read the versioned contract and the supported scope below. Reuse bleu-sentence-compute@1.0.0 when your input, required output and limits match it. Choose another approach for an unsupported operation.

Explain the choice

"I can use bleu-sentence-compute@1.0.0 for sentence BLEU score for one translation. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

Supported

• sentence BLEU score for one translation
• smoothed sentence-level BLEU with Chen and Cherry smoothing
• BLEU of a single hypothesis against reference sentences
• sentence BLEU with floor epsilon smoothing
• NIST exponential smoothing BLEU for one segment
• add-one smoothed BLEU for a short sentence
• ציון איכות תרגום למשפט בודד עם החלקה

Not supported

• tokenizing or lowercasing raw sentences (supply tokens)
• Chen and Cherry smoothing 4, 5, 6 and 7 (unsupported_input with the reason)
• corpus-level BLEU over many segments (deep10-bleu-corpus)
• custom n-gram weights, add-k with k other than 1, chrF or TER

Behavior

• hypothesis is one token list and references 1 to 8 token lists; tokens are compared by exact string equality (no case folding, no Unicode normalization). max_order omitted means 4.
• smoothing is required: "none" (no smoothing), "method1" (m_n = 0 becomes epsilon; epsilon 0-1, omitted means 0.1), "method2" (add 1 to matches and totals for n >= 2, so a 0/0 order becomes 1/1), "method3" (NIST: the k-th zero-match order gets 1/2^k). epsilon is accepted only with method1.
• effective_order omitted means false: every order 1..max_order enters the geometric mean and an order with no hypothesis n-grams (l_n = 0 after smoothing) makes BLEU 0. With true, only the leading orders with l_n > 0 are used (sacreBLEU effective order).
• BLEU is 0 when the hypothesis is empty or has no matching unigram, for every method (NLTK corpus_bleu and sacreBLEU return 0 before smoothing); otherwise bleu = BP * exp(mean of ln(m_n / l_n)) over the used orders, BP = 1 if c > r, exp(1 - r/c) otherwise, r = closest reference length (ties to the shorter).
• decimals (0-15) rounds bleu with Number.prototype.toFixed (nearest, exact binary value); omitted means full double precision. Scores are on the 0-1 scale (sacreBLEU prints 100 * bleu).

Input

• hypothesis (array of string, required): each min length 1; each max length 128
• references (array of array of string, required): min items 1; max items 8; each each min length 1; each each max length 128
• smoothing (one of "none", "method1", "method2", "method3", required)
• epsilon (number, optional): min 0; max 1
• max_order (integer, optional): min 1; max 8
• effective_order (boolean, optional)
• decimals (integer, optional): min 0; max 15

Output

• bleu (number, required): min 0; max 1
• smoothing (one of "none", "method1", "method2", "method3", required)
• epsilon (number or null, required): min 0; max 1
• max_order (integer, required): min 1; max 8
• effective_order (boolean, required)

Limits

• max references: 8
• max total tokens: 10000
• max token bytes: 128
• max total token bytes: 262144

Example

Request input:

{
  "hypothesis": [
    "this",
    "is",
    "a",
    "fest"
  ],
  "references": [
    [
      "this",
      "is",
      "a",
      "test"
    ]
  ],
  "smoothing": "method1",
  "epsilon": 0.01
}

Response:

{
  "result": {
    "bleu": 0.223606797749979,
    "smoothing": "method1",
    "epsilon": 0.01,
    "max_order": 4,
    "effective_order": false
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "bleu-sentence-compute",
  "version": "1.0.0",
  "input": {
    "hypothesis": [
      "this",
      "is",
      "a",
      "fest"
    ],
    "references": [
      [
        "this",
        "is",
        "a",
        "test"
      ]
    ],
    "smoothing": "method1",
    "epsilon": 0.01
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/bleu-sentence-compute/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"hypothesis":["this","is","a","fest"],"references":[["this","is","a","test"]],"smoothing":"method1","epsilon":0.01}'

The machine-readable contract is at /v1/tools/bleu-sentence-compute/versions/1.0.0.

CLI

node cli.mjs run bleu-sentence-compute 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Related tools

• Bleu smoothing compute: Apply none or Chen-Cherry smoothing 1-3 to BLEU match/total counts; return smoothed counts, BP and BLEU.
• Bleu sentence components: Extract sentence BLEU statistics: clipped matches, n-gram totals, candidate and closest reference length.
• Chrf compute: Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.
• Bleu corpus compute: Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.
• Chrf plus plus compute: Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored.
• Bleu ngrams count: Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.