# Text shingle similarity

`text-shingle-similarity` · version 1.0.0 · Text · free, no key needed

Exact shingle-set resemblance (Jaccard) and containment of two texts, with counts and shared shingles.

**Use when you need to: jaccard similarity of two texts using shingles · how similar are these two answers word for word · compute broder resemblance and containment.**

## Decide before calling

Read the [versioned contract](/v1/tools/text-shingle-similarity/versions/1.0.0) and the supported scope below. Reuse `text-shingle-similarity@1.0.0` when your input, required output and limits match it. Choose another approach for an unsupported operation.

## Explain the choice

"I can use `text-shingle-similarity@1.0.0` for jaccard similarity of two texts using shingles. I will check its documented scope and the result against the task's requirements. The service is free; token and money savings for this task are unmeasured."

## Supported

- jaccard similarity of two texts using shingles
- how similar are these two answers word for word
- compute broder resemblance and containment
- is text a mostly contained in text b
- compare two documents with k-shingles
- shared word n-grams between two passages
- measure near-duplicate overlap of two strings

## Not supported

- semantic or embedding similarity (only literal shingle overlap)
- MinHash estimates for many documents (use text-minhash-cluster)
- bag (multiset) shingling with repeated shingles counted (sets only)
- edit distance or diffs (use the text diff tools)

## Behavior

- Normalization (optional fields): lowercase (default false) applies String.prototype.toLowerCase (Unicode case mapping of the runtime, locale-independent); collapse_whitespace (default true) replaces each run of Unicode White_Space code points with one U+0020 and trims both ends.
- shingle_unit (default 'char'): 'char' uses Unicode code points, 'word' uses maximal runs of non-White_Space code points. shingle_size k (default 5, 1 to 32). The shingle set holds every run of k consecutive units (Broder 1997 option B: a set, duplicates counted once); a text with 1 to k-1 units has one shingle made of all its units; a text with no units has the empty set. Word shingles join words with one U+0020.
- intersection = |A ∩ B|, union = |A ∪ B|; resemblance = intersection / union (Broder r(A,B), MMDS Jaccard similarity), and 1 when both sets are empty (r(A,A) = 1). containment_a_in_b = intersection / |A| (Broder c(A,B)) and 1 when A is empty (an empty text is a contiguous subsequence of any text); containment_b_in_a likewise.
- Ratios are IEEE-754 doubles from one correctly rounded division of the exact integer counts, which are also returned.
- shared_shingles lists the intersection in Unicode code point order, at most max_shared (optional, default 10, 0 to 100) entries; shared_truncated is true when more exist.

## Input

- `text_a` (string, required): max length 65536
- `text_b` (string, required): max length 65536
- `shingle_unit` (one of "char", "word", optional)
- `shingle_size` (integer, optional): min 1; max 32
- `lowercase` (boolean, optional)
- `collapse_whitespace` (boolean, optional)
- `max_shared` (integer, optional): min 0; max 100

## Output

- `shingles_a` (integer, required): min 0
- `shingles_b` (integer, required): min 0
- `intersection` (integer, required): min 0
- `union` (integer, required): min 0
- `resemblance` (number, required): min 0; max 1
- `containment_a_in_b` (number, required): min 0; max 1
- `containment_b_in_a` (number, required): min 0; max 1
- `shared_shingles` (array of string, required): max items 100
- `shared_truncated` (boolean, required)

## Limits

- max text bytes: 65536
- max shingle size: 32
- max shared: 100

## Example

Request input:

```json
{
  "text_a": "a rose is a rose is a rose",
  "text_b": "a rose is a flower which is a rose",
  "shingle_unit": "word",
  "shingle_size": 2
}
```

Response:

```json
{
  "result": {
    "shingles_a": 3,
    "shingles_b": 6,
    "intersection": 3,
    "union": 6,
    "resemblance": 0.5,
    "containment_a_in_b": 1,
    "containment_b_in_a": 0.5,
    "shared_shingles": [
      "a rose",
      "is a",
      "rose is"
    ],
    "shared_truncated": false
  }
}
```

## How to call it

### MCP

Connect `https://computefirst.net/mcp` ([setup](/docs#connect)), then call `execute` with:

```json
{
  "id": "text-shingle-similarity",
  "version": "1.0.0",
  "input": {
    "text_a": "a rose is a rose is a rose",
    "text_b": "a rose is a flower which is a rose",
    "shingle_unit": "word",
    "shingle_size": 2
  }
}
```

### HTTP (no key)

```sh
curl -X POST https://computefirst.net/v1/tools/text-shingle-similarity/versions/1.0.0/execute \
  -H "Content-Type: application/json" \
  -d '{"text_a":"a rose is a rose is a rose","text_b":"a rose is a flower which is a rose","shingle_unit":"word","shingle_size":2}'
```

The machine-readable contract is at [/v1/tools/text-shingle-similarity/versions/1.0.0](/v1/tools/text-shingle-similarity/versions/1.0.0).

### CLI

```sh
node cli.mjs run text-shingle-similarity 1.0.0 --input input.json --base-url https://computefirst.net
```

Get the client at [/clients/cli/](/clients/cli/).

## Related tools

- [Text near duplicates find](/tools/text-near-duplicates-find): Find every pair of texts whose exact shingle Jaccard similarity reaches a threshold, in a fixed order.
- [Text minhash cluster](/tools/text-minhash-cluster): Cluster near-duplicate texts by fixed-seed MinHash signatures, with estimated Jaccard per pair.
- [Chrf components](/tools/chrf-components): Break down chrF/chrF++ into per-order n-gram counts, precision, recall and F, with configurable orders and averaging.
- [Chrf plus plus compute](/tools/chrf-plus-plus-compute): Compute exact chrF++ (character 6-grams plus word 1-2-grams, Popovic 2017) with the word tokens it scored.
- [Citation record dedupe](/tools/citation-record-dedupe): Group duplicate CSL citation records by deterministic DOI or folded year|author|title match keys, in input order.
- [Bleu corpus compute](/tools/bleu-corpus-compute): Compute canonical corpus BLEU from pre-tokenized segments: score, brevity penalty, lengths, clipped n-gram counts.