Remove Duplicate Lines from Your Text

When duplicate lines get in the way

Repeated lines creep into a list almost every time it has been through more than one pair of hands. Exports get merged, spreadsheets get pasted together, and suddenly the same entry is in there four times.

It shows up everywhere:

• Email and subscriber lists
• Keyword and search term lists
• URL and redirect lists
• Log files and CSV exports
• Anything assembled from several sources

Three ways to handle repeats

Removing duplicates can mean two quite different things, so you get to pick which one you meant:

• Keep the first of each line gives you one copy of everything, in the original order.
• Keep only lines that appear once throws away every line that was repeated, including its first copy. Useful when a repeat means something went wrong.
• Show me the duplicates instead flips it round and returns only the lines that occurred more than once, so you can see what was doubled up before you decide.

The counter under the result tells you how many lines went in, how many came out, and how many were removed.

How to use the tool

It takes about ten seconds:

• Paste your text into the left box, one entry per line.
• Choose what should happen to the repeats.
• Adjust the matching if you need to: ignore casing, ignore spaces around a line, or drop blank lines.
• The result appears as you type, and the button copies it.

Everything runs in your browser, so nothing is uploaded anywhere. If your list needs tidying first, clean up the whitespace or sort it alphabetically.