# Bleu corpus compute

`bleu-corpus-compute` · version 1.0.0 · Text · free, no key needed

Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.

**Use when you need to: corpus BLEU score from tokenized segments · system-level BLEU for a machine translation test set · compute BLEU-4 over a whole corpus without smoothing.**

## Decide before calling

Read the [versioned contract](/v1/tools/bleu-corpus-compute/versions/1.0.0) and the supported scope below. Reuse `bleu-corpus-compute@1.0.0` when your input, required output and limits match it. Choose another approach for an unsupported operation.

## Explain the choice

"I can use `bleu-corpus-compute@1.0.0` for corpus BLEU score from tokenized segments. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

## Supported

- corpus BLEU score from tokenized segments
- system-level BLEU for a machine translation test set
- compute BLEU-4 over a whole corpus without smoothing
- Papineni corpus BLEU with multiple references per segment
- micro-averaged BLEU across many hypothesis segments
- document-level BLEU with brevity penalty and n-gram precisions
- חישוב ציון איכות תרגום לקורפוס שלם

## Not supported

- tokenizing raw text (13a, intl, char or any other tokenizer); tokens must be supplied
- sentence-level smoothing (Chen and Cherry methods) or effective-order rescaling
- averaging per-sentence BLEU scores (macro average)
- custom non-uniform n-gram weights, chrF, TER, METEOR or case-insensitive matching

## Behavior

- Input is pre-tokenized: hypotheses[i] is the token list of segment i and references[i] lists 1 to 8 reference token lists for it. Tokens are compared by exact string equality: no case folding, no Unicode normalization, no trimming.
- For each order n = 1..max_order (max_order omitted means 4) the clipped matches m_n and totals l_n are summed over all segments before dividing (Papineni 2002 section 2.1.1); clipping uses the largest count of the n-gram in any single reference.
- candidate_length c is the total hypothesis token count; reference_length r sums, per segment, the reference length closest to the hypothesis length, ties going to the shorter reference.
- brevity_penalty is 1 when c > r, 0 when c = 0 (even if r = 0), otherwise exp(1 - r/c).
- bleu = brevity_penalty * exp((1/N) * sum of ln(m_n / l_n)) with N = max_order, summed in order n = 1..N; it is exactly 0 when brevity_penalty is 0 or any order has m_n = 0 (including l_n = 0). No smoothing is applied.
- decimals (0-15) rounds bleu and brevity_penalty with Number.prototype.toFixed (nearest, computed on the exact binary value); omitted means full IEEE-754 double precision. Counts are exact integers.
- Scores are on the 0-1 scale; multiply by 100 for the sacreBLEU percentage scale.

## Input

- `hypotheses` (array of array of string, required): min items 1; max items 1000; each each min length 1; each each max length 128
- `references` (array of array of array of string, required): min items 1; max items 1000; each min items 1; each max items 8; each each each min length 1; each each each max length 128
- `max_order` (integer, optional): min 1; max 8
- `decimals` (integer, optional): min 0; max 15

## Output

- `bleu` (number, required): min 0; max 1
- `brevity_penalty` (number, required): min 0; max 1
- `candidate_length` (integer, required): min 0
- `reference_length` (integer, required): min 0
- `max_order` (integer, required): min 1; max 8
- `matches` (array of integer, required): min items 1; max items 8; each min 0
- `totals` (array of integer, required): min items 1; max items 8; each min 0

## Limits

- max segments: 1000
- max references per segment: 8
- max total tokens: 50000
- max token bytes: 128
- max total token bytes: 1048576

## Example

Request input:

```json
{
  "hypotheses": [
    [
      "this",
      "is",
      "a",
      "test"
    ]
  ],
  "references": [
    [
      [
        "this",
        "is",
        "a",
        "test"
      ]
    ]
  ]
}
```

Response:

```json
{
  "result": {
    "bleu": 1,
    "brevity_penalty": 1,
    "candidate_length": 4,
    "reference_length": 4,
    "max_order": 4,
    "matches": [
      4,
      3,
      2,
      1
    ],
    "totals": [
      4,
      3,
      2,
      1
    ]
  }
}
```

## How to call it

### MCP

Connect `https://computefirst.net/mcp` ([setup](/docs#connect)), then call `execute` with:

```json
{
  "id": "bleu-corpus-compute",
  "version": "1.0.0",
  "input": {
    "hypotheses": [
      [
        "this",
        "is",
        "a",
        "test"
      ]
    ],
    "references": [
      [
        [
          "this",
          "is",
          "a",
          "test"
        ]
      ]
    ]
  }
}
```

### HTTP (no key)

```sh
curl -X POST https://computefirst.net/v1/tools/bleu-corpus-compute/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"hypotheses":[["this","is","a","test"]],"references":[[["this","is","a","test"]]]}'
```

The machine-readable contract is at [/v1/tools/bleu-corpus-compute/versions/1.0.0](/v1/tools/bleu-corpus-compute/versions/1.0.0).

### CLI

```sh
node cli.mjs run bleu-corpus-compute 1.0.0 --input input.json --base-url https://computefirst.net
```

Get the client at [/clients/cli/](/clients/cli/).

## Related tools

- [Chrf compute](/tools/chrf-compute): Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.
- [Chrf plus plus compute](/tools/chrf-plus-plus-compute): Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored.
- [Bleu brevity penalty](/tools/bleu-brevity-penalty): Compute the BLEU corpus brevity penalty from candidate and reference lengths using closest reference lengths.
- [Bleu ngrams count](/tools/bleu-ngrams-count): Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.
- [Bleu sentence compute](/tools/bleu-sentence-compute): Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).
- [Bleu sentence components](/tools/bleu-sentence-components): Extract sentence BLEU statistics: clipped matches, n-gram totals, candidate and closest reference length.