# Text sentences chunk

`text-sentences-chunk` · version 1.0.0 · Text · free, no key needed

Pack whole UAX #29 sentences into chunks of at most max_size code points, UTF-16 units or UTF-8 bytes.

**Use when you need to: chunk text by sentences · split a document into sentence-aligned chunks for embeddings · group sentences under a size budget.**

## Decide before calling

Read the [versioned contract](/v1/tools/text-sentences-chunk/versions/1.0.0) and the supported scope below. Reuse `text-sentences-chunk@1.0.0` when your input, required output and limits match it. Choose another approach for an unsupported operation.

## Explain the choice

"I can use `text-sentences-chunk@1.0.0` for chunk text by sentences. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

## Supported

- chunk text by sentences
- split a document into sentence-aligned chunks for embeddings
- group sentences under a size budget
- sentence chunking with overlap
- rag chunker that never cuts a sentence
- pack sentences into utf-8 byte limited chunks

## Not supported

- model tokenizer token budgets (tiktoken, sentencepiece)
- cutting a long sentence into smaller pieces
- semantic or embedding-similarity chunking
- heading, paragraph or markdown structure aware chunking

## Behavior

- Sentences are the default UAX #29 (revision 47) sentence segments over Unicode 17.0.0 data (data_version unicode-17.0.0), exactly as text-sentences-split with trim false; chunk text is the exact slice of the input from its first to its last sentence.
- unit (optional, default "code_point") measures sizes and offsets: "code_point", "utf16" (UTF-16 code units) or "utf8" (bytes). max_size is required, an integer from 1 to 1048576 in that unit.
- overlap (optional, default 0, at most 16) is the number of whole sentences from the end of the previous chunk to repeat at the start of the next chunk.
- Greedy packing: a chunk starts with the largest k <= min(overlap, sentences in the previous chunk) previous sentences such that they plus the first uncovered sentence fit in max_size, then adds following sentences while the total stays <= max_size. Every chunk contains at least one new sentence.
- A single sentence larger than max_size becomes its own chunk with oversized true (it is never cut and gets no overlap).
- Offsets are half-open [start, end) in the chosen unit; size = end - start; first_sentence is the 0-based index of the chunk's first sentence.
- Text is limited to 262144 UTF-8 bytes and 65536 sentences, and the JSON result to 4194304 UTF-8 bytes; more is limit_exceeded.

## Input

- `text` (string, required): max length 262144; Text to chunk (at most 262144 UTF-8 bytes).
- `max_size` (integer, required): min 1; max 1048576; Maximum chunk size in unit.
- `unit` (one of "code_point", "utf16", "utf8", optional): Optional; default "code_point".
- `overlap` (integer, optional): min 0; max 16; Optional; default 0. Whole sentences repeated from the previous chunk.

## Output

- `data_version` (constant "unicode-17.0.0", required)
- `unit` (one of "code_point", "utf16", "utf8", required)
- `max_size` (integer, required): min 1; max 1048576
- `overlap` (integer, required): min 0; max 16
- `sentence_count` (integer, required): min 0; max 65536
- `count` (integer, required): min 0
- `chunks` (array of object, required)

## Limits

- max text bytes: 262144
- max sentences: 65536
- max output bytes: 4194304

## Example

Request input:

```json
{
  "text": "One. Two two. Three three three.",
  "max_size": 14
}
```

Response:

```json
{
  "result": {
    "data_version": "unicode-17.0.0",
    "unit": "code_point",
    "max_size": 14,
    "overlap": 0,
    "sentence_count": 3,
    "count": 2,
    "chunks": [
      {
        "text": "One. Two two. ",
        "start": 0,
        "end": 14,
        "size": 14,
        "first_sentence": 0,
        "sentence_count": 2,
        "oversized": false
      },
      {
        "text": "Three three three.",
        "start": 14,
        "end": 32,
        "size": 18,
        "first_sentence": 2,
        "sentence_count": 1,
        "oversized": true
      }
    ]
  }
}
```

## How to call it

### MCP

Connect `https://computefirst.net/mcp` ([setup](/docs#connect)), then call `execute` with:

```json
{
  "id": "text-sentences-chunk",
  "version": "1.0.0",
  "input": {
    "text": "One. Two two. Three three three.",
    "max_size": 14
  }
}
```

### HTTP (no key)

```sh
curl -X POST https://computefirst.net/v1/tools/text-sentences-chunk/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"text":"One. Two two. Three three three.","max_size":14}'
```

The machine-readable contract is at [/v1/tools/text-sentences-chunk/versions/1.0.0](/v1/tools/text-sentences-chunk/versions/1.0.0).

### CLI

```sh
node cli.mjs run text-sentences-chunk 1.0.0 --input input.json --base-url https://computefirst.net
```

Get the client at [/clients/cli/](/clients/cli/).

## Related tools

- [Text paragraphs split](/tools/text-paragraphs-split): Split text into paragraphs separated by blank lines (CommonMark line rules) with code point spans and line numbers.
- [Markdown sections chunk](/tools/markdown-sections-chunk): Chunk a CommonMark document into sections at its top-level ATX and setext headings, with levels, paths and spans.
- [Text sentences split](/tools/text-sentences-split): Split text into sentences using the default Unicode UAX #29 sentence boundary rules (Unicode 17.0.0).
- [Text minhash cluster](/tools/text-minhash-cluster): Cluster near-duplicate texts by fixed-seed MinHash signatures, with estimated Jaccard per pair.
- [Text sentence offsets](/tools/text-sentence-offsets): Return UAX #29 sentence spans as start/end offsets in code points, UTF-16 units or UTF-8 bytes, with the ending rule.
- [Bleu sentence compute](/tools/bleu-sentence-compute): Compute sentence-level BLEU for one tokenized hypothesis with a chosen smoothing method (none or Chen-Cherry 1-3).