how to delete the repeat lines in emacs

Question

I have a text with a lots of lines, my question is how to delete the repeat lines in emacs? using the command in emacs or elisp packages without external utils.

for example:

this is line a
this is line b
this is line a

to remove the 3rd line (same as 1st line)

this is line a
this is line b

Answer 1

If you have Emacs 24.4 or newer, the cleanest way to do it would be the new delete-duplicate-lines function. Note that

• this works on a region, not a buffer, so select the desired text first
• it maintains the relative order of the originals, killing the duplicates

For example, if your input is

test
dup
dup
one
two
one
three
one
test
five

M-x delete-duplicate-lines would make it

test
dup
one
two
three
five

You've the option of searching from backwards by prefixing it with the universal argument (C-u). The result would then be

dup
two
three
one
test
five

Credit goes to emacsredux.com.

Other roundabout options, not giving quite the same result, available via Eshell:

1. sort -u; doesn't maintain the relative order of the originals
2. uniq; worse it needs its input to be sorted