Text lines unique

text-lines-unique · version 1.0.0 · Text · free, no key needed

Remove exact duplicate lines, keeping first-seen order.

Use when you need to: text lines unique · unique lines · dedupe text lines.

Supported

• text lines unique
• unique lines
• dedupe text lines

Not supported

• trim lines
• case insensitive unique
• unicode normalize
• fuzzy match

Behavior

• Split on CR LF, LF, or CR. Do not treat other Unicode separators as line breaks.
• A trailing terminator does not create an extra empty line.
• Empty document -> zero lines, trailing_terminator false.
• A document that is only a terminator -> one empty line, trailing_terminator true.
• A document with no terminator -> last line kept, trailing_terminator false.
• Preserve empty lines in the middle.
• Output reconstruction uses LF. If the result has at least one line, append a trailing LF iff the selected window includes the original trailing terminator or includes a line that was followed by a separator in the source.
• No trimming, case folding, Unicode normalization, or locale rules.
• Equality is exact string equality of parsed line bodies.
• First occurrence is kept; later exact duplicates are dropped.
• Output order is first-seen order of kept lines.

Input

• text (string, required): max length 256000

Output

• text (string, required)
• lines_in (integer, required)
• lines_out (integer, required)
• duplicates_removed (integer, required)

Limits

• max input bytes: 256000
• max lines: 10000

Example

Request input:

{
  "text": "b\na\nb\nA\n"
}

Response:

{
  "result": {
    "text": "b\na\nA\n",
    "lines_in": 4,
    "lines_out": 3,
    "duplicates_removed": 1
  }
}

How to call it

MCP

Connect https://computefirst.net/mcp (setup), then call execute with:

{
  "id": "text-lines-unique",
  "version": "1.0.0",
  "input": {
    "text": "b\na\nb\nA\n"
  }
}

HTTP (no key)

curl -X POST https://computefirst.net/v1/tools/text-lines-unique/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"text":"b\na\nb\nA\n"}'

The machine-readable contract is at /v1/tools/text-lines-unique/versions/1.0.0.

CLI

node cli.mjs run text-lines-unique 1.0.0 --input input.json --base-url https://computefirst.net

Get the client at /clients/cli/.

Related tools

• Text count: Count Unicode code points, UTF-16 code units, UTF-8 bytes, and lines.
• Text line diff: Diff two texts as line records, preserving each line terminator.
• Text lines chunk: Split parsed lines into consecutive chunks of size lines each.
• Text lines count: Count exact line multiplicities in first-seen order.
• Text lines difference: Set difference a minus b using exact line equality.
• Text lines head: Return the first n parsed lines with LF reconstruction.