Text sentence offsets

text-sentence-offsets · version 1.0.0 · Text · free, no key needed

Return UAX #29 sentence spans as start/end offsets in code points, UTF-16 units or UTF-8 bytes, with the ending rule.

Use when you need to: sentence start and end offsets · sentence boundary positions in utf-8 bytes · character offsets of each sentence for highlighting.

Decide before calling

Read the versioned contract and the supported scope below. Reuse text-sentence-offsets@1.0.0 when your input, required output and limits match it. Choose another approach for an unsupported operation.

Explain the choice

"I can use text-sentence-offsets@1.0.0 for sentence start and end offsets. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

Supported

• sentence start and end offsets
• sentence boundary positions in utf-8 bytes
• character offsets of each sentence for highlighting
• utf-16 sentence spans for javascript strings
• which rule ended each sentence
• map sentences to code point ranges

Not supported

• returning the sentence text (use text-sentences-split)
• grapheme cluster, word or line break offsets
• offsets after Unicode normalization of the input
• language-tailored sentence boundaries

Behavior

• Boundaries are the default UAX #29 (revision 47) sentence boundaries over Unicode 17.0.0 Sentence_Break data (data_version unicode-17.0.0), identical to text-sentences-split with trim false.
• unit (optional, default "code_point"): "code_point" counts Unicode scalar values (Python str indices), "utf16" counts UTF-16 code units (JavaScript string indices), "utf8" counts UTF-8 bytes.
• spans are half-open [start, end), contiguous, cover the whole text in order, and the last end equals length.
• break_rule names the UAX #29 rule that created the break ending the span: "SB4" after a paragraph separator (Sep, CR, LF; CR LF is one), "SB11" after a sentence terminator sequence SATerm Close* Sp*, "SB2" at the end of text. These match the [4.0], [11.0] and [0.3] markers of SentenceBreakTest-17.0.0.txt.
• Empty text returns no spans and length 0; text is limited to 262144 UTF-8 bytes and 65536 sentences.

Input

• text (string, required): max length 262144; Text to segment (at most 262144 UTF-8 bytes).
• unit (one of "code_point", "utf16", "utf8", optional): Optional; default "code_point".

Output

• data_version (constant "unicode-17.0.0", required)
• unit (one of "code_point", "utf16", "utf8", required)
• length (integer, required): min 0
• count (integer, required): min 0; max 65536
• spans (array of object, required): max items 65536

Limits

• max text bytes: 262144
• max sentences: 65536

Example

Request input:

{
  "text": "a. (3) b. Ça va?\nOui."
}

Response:

{
  "result": {
    "data_version": "unicode-17.0.0",
    "unit": "code_point",
    "length": 21,
    "count": 3,
    "spans": [
      {
        "start": 0,
        "end": 10,
        "break_rule": "SB11"
      },
      {
        "start": 10,
        "end": 17,
        "break_rule": "SB4"
      },
      {
        "start": 17,
        "end": 21,
        "break_rule": "SB2"
      }
    ]
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "text-sentence-offsets",
  "version": "1.0.0",
  "input": {
    "text": "a. (3) b. Ça va?\nOui."
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/text-sentence-offsets/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"text":"a. (3) b. Ça va?\nOui."}'

The machine-readable contract is at /v1/tools/text-sentence-offsets/versions/1.0.0.

CLI

node cli.mjs run text-sentence-offsets 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Related tools

• Citation anchor check: Check a quote anchor (exact text plus prefix/suffix) matches exactly one place in a supplied source version.
• Text paragraphs split: Split text into paragraphs separated by blank lines (CommonMark line rules) with code point spans and line numbers.
• Chrf compute: Compute exact chrF (character 6-gram F-beta, Popovic 2015) for hypothesis/reference segments, corpus and per segment.
• Markdown sections chunk: Chunk a CommonMark document into sections at its top-level ATX and setext headings, with levels, paths and spans.
• Text delimiter chunk: Split text on an explicit literal delimiter (Python str.split rules, optional max_splits) with spans and line numbers.
• Text sentences chunk: Pack whole UAX #29 sentences into chunks of at most max_size code points, UTF-16 units or UTF-8 bytes.