1.

Solve : delete duplicates in text file?

Answer»

I have a text file with hundreds of e-mail addresses in it (separated by semi-colons) - no, NOT for spamming! I want to get rid of duplicate e-mail addresses within the file so if I broadcast a message to those on my list, I won't annoy anybody by sending it two or three times. Any suggestions would be welcomed, as I have tried everything including a script from MS Technet which I could not get to run.Are you saying the file has MULTIPLE email addresses separated by semi-colons on each line or one address per line with a semi-colon at the end.

In either case, one solution would be Excel (if installed). You can use it to filter unique records, put them on another sheet, then save the sheet as a text file. Excel would require a lot of cut & paste if more than one address per line.

If you know any scripting language, this can be scripted without too much hassle.

Generally the scripts from MS Techet teach technique and require some editing before they will run on the LOCAL machine.

If you still have problems, post the script you got from Technet and someone can probably help you.

Many thanks for the REPLY. The e-mail addresses are all on a "continuous line" in notepad separated by a semi-colon. I was looking to SIMPLY copy & paste into an outlook e-mail message when done. Excel limits the import to about 65 out of hundreds, so that doesn't work. The following script which I can't get to work is from Technet at:

www.microsoft.com/technet/scriptcenter/resources/qanda/apr05/hey0413.mspx


Const ForReading = 1
Const ForWriting = 2

Set objDictionary = CreateObject("Scripting.Dictionary")

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFile = objFSO.OpenTextFile c:\newletter-duplicates-.txt
   ("c:\scripts\namelist.txt", ForReading)

Do Until objFile.AtEndOfStream
   strName = objFile.ReadLine
   If Not objDictionary.Exists(strName) Then
       objDictionary.Add strName, strName
   End If
Loop

objFile.Close

Set objFile = objFSO.OpenTextFile c:\newletter-clean.txt
   ("c:\scripts\namelist.txt", ForWriting)

For Each strKey in objDictionary.Keys
   objFile.WriteLine strKey
Next

objFile.Close

Well you could open the file in Word.  Do a search and replace on the ";" and replace it with ";" plus the carriage return character.  This would give you a list in a single column that you could open and sort with Excel.  Then you could manually remove the duplicates from the sorted list.  Excel will then allow you to save the file as a text file.

If you wanted it back to a single line list you could reverse the search and replace using Word.Good idea, but I tried it with no success. There appears to be no way of geting an "enter" or "carriage return" into the replace cell in the word repalce window. You can insert the ; but if you hit enter it starts the replace processSure there is.  When you do the replace look on the bottom of the replace window, you will see a "more" button, click it, then click the "special" button, and select "manual line break"

See your other post on the Windows board for an automated  solution for this.

Gussery - thanks, I never saw the drop down. I will try again.

Sidewinder - I posted to you on the other board.

Many, many thanks to you both for your input and your good natures!!Here is a good tool to delete duplicate FILES
www.duplicateFilesDeleter.comHe left 5 Years ago....i doubt he'll see your Post. Quote from: patio on November 28, 2010, 08:00:53 AM

He left 5 Years ago....i doubt he'll see your Post.

And he wasn't even asking about duplicate files. And the link posted was for "free-trial" payware.



Discussion

No Comment Found