Remove Duplicate Lines
Paste a list to remove duplicate lines, with control over case sensitivity, surrounding whitespace and whether blank lines are kept.
How it works
Each line is compared against the ones already seen, and the first occurrence is kept. Order is preserved, so the list comes back in the sequence you pasted it rather than sorted — sort afterwards if you need that.
The comparison options matter more than they look. With case sensitivity off, “Apple” and “apple” are the same line; with trimming on, “ apple” and “apple” are too. For email lists and exported data, leaving both off-and-on as they default is usually what you want.
The formula
Keep rule
keep a line if its comparison key has not been seen before
Comparison key
line → optionally trimmed → optionally lowercased
Worked examples
| Scenario | Working | Result |
|---|---|---|
| apple, banana, Apple | Case insensitive | apple, banana |
| apple, banana, Apple | Case sensitive | apple, banana, Apple |
| “ apple” and “apple” | Trim on | One line kept |
When you'd use it
- Cleaning an exported email or subscriber list
- Removing repeats from a copied spreadsheet column
- Tidying a list of URLs, SKUs or IDs before import
- Finding the distinct values in a log extract
Common questions
Does it keep the first or the last duplicate?
The first. Everything after it that matches is dropped, and the surviving lines stay in their original order.
Should I turn case sensitivity on?
Usually not. Email addresses, tags and most identifiers are the same value regardless of case, so leaving it off catches more genuine duplicates. Turn it on when case actually distinguishes two values, such as case-sensitive codes.
Is my text uploaded anywhere?
No. The whole tool runs in your browser, so what you paste never leaves your device. That matters when the text is a draft, a transcript or anything covered by an NDA.