Text delimiter chunk

text-delimiter-chunk · version 1.0.0 · Text · free, no key needed

Split text on an explicit literal delimiter (Python str.split rules, optional max_splits) with spans and line numbers.

Use when you need to: split text on a custom delimiter · chunk a document at every separator marker · split on a multi character separator and keep empty pieces.

Decide before calling

Read the versioned contract and the supported scope below. Reuse text-delimiter-chunk@1.0.0 when your input, required output and limits match it. Choose another approach for an unsupported operation.

Explain the choice

"I can use text-delimiter-chunk@1.0.0 for split text on a custom delimiter. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

Supported

• split text on a custom delimiter
• chunk a document at every separator marker
• split on a multi character separator and keep empty pieces
• split at most n times on a delimiter
• character offsets of each piece between delimiters
• cut text at explicit split markers like <!-- split -->

Not supported

• regular-expression or multiple alternative delimiters (one literal string only)
• whitespace-run splitting without a delimiter (Python str.split() with no sep)
• splitting from the right (rsplit) or dropping empty pieces

Behavior

• Follows Python 3 str.split(sep, maxsplit) with an explicit sep: the delimiter is a non-empty literal string matched by exact UTF-16 code units (no case folding or Unicode normalization); occurrences are found left to right without overlap, each search resuming right after the previous occurrence; consecutive delimiters delimit empty strings; text "" gives one empty chunk.
• max_splits omitted means no limit (Python maxsplit=-1). max_splits = k >= 0 performs at most k splits, so the last chunk keeps the remaining text including any further delimiters; k = 0 returns the whole text as one chunk.
• Each chunk reports start and end as Unicode code point offsets into text (end exclusive) and its exact text; the delimiters themselves belong to no chunk. delimiter_count is the number of delimiter occurrences consumed (chunk_count - 1).
• Line numbers use CommonMark 0.31.2 line endings (LF, CR, CRLF) of the whole text: line(pos) = 1 + number of line endings that end at or before pos; start_line = line(start), end_line = line(end - 1), or start_line for an empty chunk.
• An empty delimiter (Python: ValueError "empty separator"), a lone surrogate in text or delimiter, or a negative or non-integer max_splits throws invalid_input.

Input

• text (string, required): The text to split (UTF-8, at most max_input_bytes).
• delimiter (string, required): min length 1; The literal separator string (at most max_delimiter_bytes).
• max_splits (integer, optional): min 0; max 9007199254740991; At most this many splits; omit for no limit.

Output

• chunk_count (integer, required): min 1
• delimiter_count (integer, required): min 0
• chunks (array of object, required): min items 1; max items 10000

Limits

• max input bytes: 262144
• max delimiter bytes: 256
• max chunks: 10000
• max output bytes: 1048576

Example

Request input:

{
  "text": "1,2,,3,",
  "delimiter": ","
}

Response:

{
  "result": {
    "chunk_count": 5,
    "delimiter_count": 4,
    "chunks": [
      {
        "index": 0,
        "start": 0,
        "end": 1,
        "start_line": 1,
        "end_line": 1,
        "text": "1"
      },
      {
        "index": 1,
        "start": 2,
        "end": 3,
        "start_line": 1,
        "end_line": 1,
        "text": "2"
      },
      {
        "index": 2,
        "start": 4,
        "end": 4,
        "start_line": 1,
        "end_line": 1,
        "text": ""
      },
      {
        "index": 3,
        "start": 5,
        "end": 6,
        "start_line": 1,
        "end_line": 1,
        "text": "3"
      },
      {
        "index": 4,
        "start": 7,
        "end": 7,
        "start_line": 1,
        "end_line": 1,
        "text": ""
      }
    ]
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "text-delimiter-chunk",
  "version": "1.0.0",
  "input": {
    "text": "1,2,,3,",
    "delimiter": ","
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/text-delimiter-chunk/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"text":"1,2,,3,","delimiter":","}'

The machine-readable contract is at /v1/tools/text-delimiter-chunk/versions/1.0.0.

CLI

node cli.mjs run text-delimiter-chunk 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Related tools

• Markdown sections chunk: Chunk a CommonMark document into sections at its top-level ATX and setext headings, with levels, paths and spans.
• Text paragraphs split: Split text into paragraphs separated by blank lines (CommonMark line rules) with code point spans and line numbers.
• Citation passage check: Check that a cited start/end span of a supplied source version reads exactly as the quoted passage.
• Text sentence offsets: Return UAX #29 sentence spans as start/end offsets in code points, UTF-16 units or UTF-8 bytes, with the ending rule.
• Text shingle similarity: Exact shingle-set resemblance (Jaccard) and containment of two texts, with counts and shared shingles.
• Bleu ngrams count: Count BLEU clipped n-gram matches and totals per order for tokenized segments, with optional clipping tables.