Bleu ngrams count

bleu-ngrams-count · version 1.0.0 · Text · free, no key needed

Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.

Use when you need to: modified n-gram precision numerator and denominator · clipped n-gram match counts against several references · BLEU sufficient statistics per n-gram order.

Decide before calling

Read the versioned contract and the supported scope below. Reuse bleu-ngrams-count@1.0.0 when your input, required output and limits match it. Choose another approach for an unsupported operation.

Explain the choice

"I can use bleu-ngrams-count@1.0.0 for modified n-gram precision numerator and denominator. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

Supported

modified n-gram precision numerator and denominator
clipped n-gram match counts against several references
BLEU sufficient statistics per n-gram order
max reference count clipping table for each n-gram
how many bigrams of the hypothesis match the references
per-segment n-gram match counts for a translation corpus
ספירת רצפי מילים חופפים מול תרגומי ייחוס

Not supported

computing the BLEU score or brevity penalty (use bleu-corpus-compute or bleu-brevity-penalty)
tokenizing, lowercasing or normalizing raw text before counting
recall-oriented or chrF character n-gram statistics
skip-grams or n-grams that cross segment boundaries

Behavior

Input is pre-tokenized; tokens are compared by exact string equality with no case folding or Unicode normalization. N-grams never cross segment boundaries.
For each order n = 1..max_order (omitted means 4): count(g) is the occurrences of n-gram g in the hypothesis, max_reference_count(g) the largest count of g in any single reference, clipped_count(g) = min(count, max_reference_count) (Papineni 2002 section 2.1, footnote 2).
matches[n-1] sums clipped counts and totals[n-1] sums max(0, len - n + 1) over all segments (section 2.1.1); a hypothesis shorter than n contributes 0 to both.
detail "totals" (default when omitted) returns only the corpus sums; "segments" adds one row per segment in input order; "ngrams" also adds one row per distinct hypothesis n-gram, ordered by segment, then order, then first occurrence in the hypothesis.
Output larger than 1,000,000 UTF-8 bytes of JSON is limit_exceeded; ask for a coarser detail level or split the corpus.

Input

hypotheses (array of array of string, required): min items 1; max items 1000; each each min length 1; each each max length 128
references (array of array of array of string, required): min items 1; max items 1000; each min items 1; each max items 8; each each each min length 1; each each each max length 128
max_order (integer, optional): min 1; max 8
detail (one of "totals", "segments", "ngrams", optional)

Output

max_order (integer, required): min 1; max 8
matches (array of integer, required): min items 1; max items 8; each min 0
totals (array of integer, required): min items 1; max items 8; each min 0
segments (array of object, optional)
ngrams (array of object, optional)

Limits

max segments: 1000
max references per segment: 8
max total tokens: 50000
max token bytes: 128
max total token bytes: 1048576
max output bytes: 1000000

Example

Request input:

{
  "hypotheses": [
    [
      "of",
      "the"
    ]
  ],
  "references": [
    [
      [
        "It",
        "is",
        "a",
        "guide",
        "to",
        "action",
        "that",
        "ensures",
        "that",
        "the",
        "military",
        "will",
        "forever",
        "heed",
        "Party",
        "commands"
      ],
      [
        "It",
        "is",
        "the",
        "guiding",
        "principle",
        "which",
        "guarantees",
        "the",
        "military",
        "forces",
        "always",
        "being",
        "under",
        "the",
        "command",
        "of",
        "the",
        "Party"
      ],
      [
        "It",
        "is",
        "the",
        "practical",
        "guide",
        "for",
        "the",
        "army",
        "always",
        "to",
        "heed",
        "the",
        "directions",
        "of",
        "the",
        "party"
      ]
    ]
  ],
  "max_order": 2
}

Response:

{
  "result": {
    "max_order": 2,
    "matches": [
      2,
      1
    ],
    "totals": [
      2,
      1
    ]
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "bleu-ngrams-count",
  "version": "1.0.0",
  "input": {
    "hypotheses": [
      [
        "of",
        "the"
      ]
    ],
    "references": [
      [
        [
          "It",
          "is",
          "a",
          "guide",
          "to",
          "action",
          "that",
          "ensures",
          "that",
          "the",
          "military",
          "will",
          "forever",
          "heed",
          "Party",
          "commands"
        ],
        [
          "It",
          "is",
          "the",
          "guiding",
          "principle",
          "which",
          "guarantees",
          "the",
          "military",
          "forces",
          "always",
          "being",
          "under",
          "the",
          "command",
          "of",
          "the",
          "Party"
        ],
        [
          "It",
          "is",
          "the",
          "practical",
          "guide",
          "for",
          "the",
          "army",
          "always",
          "to",
          "heed",
          "the",
          "directions",
          "of",
          "the",
          "party"
        ]
      ]
    ],
    "max_order": 2
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/bleu-ngrams-count/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"hypotheses":[["of","the"]],"references":[[["It","is","a","guide","to","action","that","ensures","that","the","military","will","forever","heed","Party","commands"],["It","is","the","guiding","principle","which","guarantees","the","military","forces","always","being","under","the","command","of","the","Party"],["It","is","the","practical","guide","for","the","army","always","to","heed","the","directions","of","the","party"]]],"max_order":2}'

The machine-readable contract is at /v1/tools/bleu-ngrams-count/versions/1.0.0.

CLI

node cli.mjs run bleu-ngrams-count 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Bleu sentence components: Extract sentence BLEU statistics: clipped matches, n-gram totals, candidate and closest reference length.
Bleu corpus compute: Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.
Bleu brevity penalty: Compute the BLEU corpus brevity penalty from candidate and reference lengths using closest reference lengths.
Chrf compute: Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.
Chrf plus plus compute: Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored.
Bleu sentence compute: Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).