Change language

Text

Remove duplicate lines from a list

Keep only unique lines, optionally sorted alphabetically.

Published by SolucionesAhora.com How we verify

Calculator inputs

Ignore letter case

Sort alphabetically

Results

Enter your values and press “Calculate” to see the result.

Free, no sign-up! What if you could do this without a calculator? A five-minute game with challenges, lives and points. It clicks even when you cannot be bothered.

In short

What it calculates
Keep only unique lines, optionally sorted alphabetically.
Formula used
The first occurrence of each line is kept
Example
From the list apple, pear, apple, grape:

Remove duplicate lines

Lists of emails, product references, keywords or identifiers almost always arrive with repeats.

Paste the list and keep only what appears once.

How it works

Lines are scanned keeping the first occurrence of each value and discarding the rest. Optionally letter case is ignored and the result sorted.

Formula

The first occurrence of each line is kept

Removed = original − unique

Worked example

From the list apple, pear, apple, grape:

Original: 4
Unique: 3
Removed: 1

Explanation

Duplicates are almost never identical

A trailing space, a different capital, a mistyped accent or an invisible line break make two entries the eye reads as the same look different to a computer. Before removing duplicates it pays to normalise: trim spaces, unify case and decide whether accents should count.

Case: it depends on the data

In an email list, MARIA@EXAMPLE.COM and maria@example.com are the same person, because domains are case-insensitive. In a list of product codes or passwords, that distinction can be significant. There is no universal answer: it depends what the data represent.

Preserving order matters

When deduplicating a list ordered by priority, date or relevance, keeping the first occurrence of each value preserves that information. Sorting alphabetically to deduplicate and returning the sorted result destroys the original order, which was often the valuable part.

In databases, prevent rather than clean

A unique index stops duplicates entering in the first place. Cleaning afterwards is always costlier and riskier, especially if related records point at the rows about to be deleted.

Frequently asked questions

Is the original order preserved?

Yes, unless you turn on alphabetical sorting. Each value keeps the position of its first occurrence.

Is it case sensitive?

By default yes. With the ignore-case option, “London” and “london” count as the same line.

Why do duplicates still appear after cleaning?

Almost always trailing spaces, differing capitals or invisible characters. Normalising the text before comparing resolves most cases.

Is the original order preserved?

Yes. The first occurrence of each value is kept, which preserves the priority or date order of the original list.

Need to calculate something else?

These tools are often used alongside this calculator.

Change case

Turn text into uppercase, lowercase, title case, snake_case or camelCase.

Word count

Words, characters, sentences and paragraphs in any text.

Slug generator

Turn a title into a clean URL with no accents or odd characters.

Character count

Characters with and without spaces, lines and bytes in a text.

Reading time

How long a text takes to read, silently or aloud.