01 · The Problem
Most online diff tools answer the wrong question. They report line additions and deletions, scoring a one-word edit inside a reflowed paragraph as a total replacement.
Goal: Accurate character-level similarity ratios and fine-grained word diffing.
02 · Myers + Word Pairing
Line alignment runs Myers' greedy edit-graph walk (from git diff).
Delete/insert runs are paired into modified blocks and diffed at the word level.
Optimization: Common prefix/suffix stripping + line interning for millisecond speed.
03 · Semantic Gap
Lexical similarity and semantic meaning diverge on paraphrases and negations. The semantic engine compares TF-IDF or model embeddings to highlight meaning shifts.
Actionable: Alignment table maps paragraph-to-paragraph shifts.
Operational Limits & Edge Cases
- · 2,000 Edit Threshold: Past 2,000 edits, minimal alignment stops and falls back to block replacement. Similarity figures remain exact.
- · No Move Tracking: Relocating a block appears as an insertion and deletion without cross-position linking.
- · Embedding Negation Blindspot: Embedding models often score negations as similar subject matter. Check the lexical diff for flipped words.
- · Cosine Calibration: Unrelated embeddings cluster around 0.65. Use the interpreted bands or angular distance for realistic scale.