Remove duplicate lines in file

gawk " !x[$0]++" inputfilename.txt > outputfilename.txt