# Text delimiter chunk

`text-delimiter-chunk` · version 1.0.0 · Text · free, no key needed

Split text on an explicit literal delimiter (Python str.split rules, optional max_splits) with spans and line numbers.

**Use when you need to: split text on a custom delimiter · chunk a document at every separator marker · split on a multi character separator and keep empty pieces.**

## Decide before calling

Read the [versioned contract](/v1/tools/text-delimiter-chunk/versions/1.0.0) and the supported scope below. Reuse `text-delimiter-chunk@1.0.0` when your input, required output and limits match it. Choose another approach for an unsupported operation.

## Explain the choice

"I can use `text-delimiter-chunk@1.0.0` for split text on a custom delimiter. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

## Supported

- split text on a custom delimiter
- chunk a document at every separator marker
- split on a multi character separator and keep empty pieces
- split at most n times on a delimiter
- character offsets of each piece between delimiters
- cut text at explicit split markers like <!-- split -->

## Not supported

- regular-expression or multiple alternative delimiters (one literal string only)
- whitespace-run splitting without a delimiter (Python str.split() with no sep)
- splitting from the right (rsplit) or dropping empty pieces

## Behavior

- Follows Python 3 str.split(sep, maxsplit) with an explicit sep: the delimiter is a non-empty literal string matched by exact UTF-16 code units (no case folding or Unicode normalization); occurrences are found left to right without overlap, each search resuming right after the previous occurrence; consecutive delimiters delimit empty strings; text "" gives one empty chunk.
- max_splits omitted means no limit (Python maxsplit=-1). max_splits = k >= 0 performs at most k splits, so the last chunk keeps the remaining text including any further delimiters; k = 0 returns the whole text as one chunk.
- Each chunk reports start and end as Unicode code point offsets into text (end exclusive) and its exact text; the delimiters themselves belong to no chunk. delimiter_count is the number of delimiter occurrences consumed (chunk_count - 1).
- Line numbers use CommonMark 0.31.2 line endings (LF, CR, CRLF) of the whole text: line(pos) = 1 + number of line endings that end at or before pos; start_line = line(start), end_line = line(end - 1), or start_line for an empty chunk.
- An empty delimiter (Python: ValueError "empty separator"), a lone surrogate in text or delimiter, or a negative or non-integer max_splits throws invalid_input.

## Input

- `text` (string, required): The text to split (UTF-8, at most max_input_bytes).
- `delimiter` (string, required): min length 1; The literal separator string (at most max_delimiter_bytes).
- `max_splits` (integer, optional): min 0; max 9007199254740991; At most this many splits; omit for no limit.

## Output

- `chunk_count` (integer, required): min 1
- `delimiter_count` (integer, required): min 0
- `chunks` (array of object, required): min items 1; max items 10000

## Limits

- max input bytes: 262144
- max delimiter bytes: 256
- max chunks: 10000
- max output bytes: 1048576

## Example

Request input:

```json
{
  "text": "1,2,,3,",
  "delimiter": ","
}
```

Response:

```json
{
  "result": {
    "chunk_count": 5,
    "delimiter_count": 4,
    "chunks": [
      {
        "index": 0,
        "start": 0,
        "end": 1,
        "start_line": 1,
        "end_line": 1,
        "text": "1"
      },
      {
        "index": 1,
        "start": 2,
        "end": 3,
        "start_line": 1,
        "end_line": 1,
        "text": "2"
      },
      {
        "index": 2,
        "start": 4,
        "end": 4,
        "start_line": 1,
        "end_line": 1,
        "text": ""
      },
      {
        "index": 3,
        "start": 5,
        "end": 6,
        "start_line": 1,
        "end_line": 1,
        "text": "3"
      },
      {
        "index": 4,
        "start": 7,
        "end": 7,
        "start_line": 1,
        "end_line": 1,
        "text": ""
      }
    ]
  }
}
```

## How to call it

### MCP

Connect `https://computefirst.net/mcp` ([setup](/docs#connect)), then call `execute` with:

```json
{
  "id": "text-delimiter-chunk",
  "version": "1.0.0",
  "input": {
    "text": "1,2,,3,",
    "delimiter": ","
  }
}
```

### HTTP (no key)

```sh
curl -X POST https://computefirst.net/v1/tools/text-delimiter-chunk/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"text":"1,2,,3,","delimiter":","}'
```

The machine-readable contract is at [/v1/tools/text-delimiter-chunk/versions/1.0.0](/v1/tools/text-delimiter-chunk/versions/1.0.0).

### CLI

```sh
node cli.mjs run text-delimiter-chunk 1.0.0 --input input.json --base-url https://computefirst.net
```

Get the client at [/clients/cli/](/clients/cli/).

## Related tools

- [Markdown sections chunk](/tools/markdown-sections-chunk): Chunk a CommonMark document into sections at its top-level ATX and setext headings, with levels, paths and spans.
- [Text paragraphs split](/tools/text-paragraphs-split): Split text into paragraphs separated by blank lines (CommonMark line rules) with code point spans and line numbers.
- [Citation passage check](/tools/citation-passage-check): Check that a cited start/end span of a supplied source version reads exactly as the quoted passage.
- [Text sentence offsets](/tools/text-sentence-offsets): Return UAX #29 sentence spans as start/end offsets in code points, UTF-16 units or UTF-8 bytes, with the ending rule.
- [Text shingle similarity](/tools/text-shingle-similarity): Exact shingle-set resemblance (Jaccard) and containment of two texts, with counts and shared shingles.
- [Bleu ngrams count](/tools/bleu-ngrams-count): Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.