FWIW, here's my UNEEK.BAT. (It works for me.
@echo off > uneek.txt
if %1'==' echo Syntax: %0 filename.ext && goto :eof
if not exist %1 echo %1 not found && goto :eof
echo Removing duplicate lines from "%1" ...
for /f "tokens=* delims= " %%x in (%1) do (
find "%%x" < uneek.txt > nul
if errorlevel 1 echo %%x >> uneek.txt
echo Done. Results are in "uneek.txt".
Just tried it using the command:
and it took it roughly 10 minutes on this aging laptop running XP Pro (w/all updates). The original HOSTS file contains 3,254 lines and UNEEK.BAT stripped out 26 duplicate lines.
The biggest reason for it taking so long is because it does NOT sort anything, (which could screw up the HOSTS file). It leaves everything as is and just searches for a duplicate line, a line at a time.
Note that your original file is NOT altered in any way. The output from UNEEK.BAT will always be in a file named UNEEK.TXT in your current directory (folder).