Solve : Cataloging/Removing Duplicate Photos?

1.	Solve : Cataloging/Removing Duplicate Photos?
Answer» I need to clean up my hard drive - I have MADE copies of folders filled with pictures, as well as folders with compressed and cropped versions. I'd like to build an archive of just the original, raw pics. I know I can grunt through and compare folders, etc., by hand, but I'm lazy. I'd rather spend four hours researching an easy solution than three hours of work. That said, I ran across this article: http://www.howtogeek.com/200962/how-to-find-and-remove-duplicate-files-on-windows/ The "VisiPics" OPTION looks interesting. Has anyone ever used any software to do something like this? What tips or tricks might anyone have?Before using a tool that will clean up to remove duplicates, I would perform a backup of your data, just in case you have files named the same in different folders and same file size which are different photos. Its rare that a file will be different and be such a close match, but its safest to back up the data prior to putting the decision making into an automated program that does the cleanup. I just run a batch script to run across my computer and grab all JPG images and store them to an external device such as USB stick at the E: drive. I run the following instructions which targets the C: drive and grabs a copy of only JPG files, and RETAINS their directory structure so that if there are images of same file name, but images are different, images are then not lost to which file is newest or written last of the same file name. Code: [Select]c: cd\. xcopy .jpg E:\.* /s/d/y/h echo. echo. JPG Image Grab from C to E drive Complete echo. echo. pause Here is a list of what each switch does: Quote /S Copies directories and subdirectories except empty ones. /D:m-d-y Copies files changed on or after the specified date. If no date is given, copies only those files whose source time is newer than the destination time. /Y Suppresses prompting to confirm you want to overwrite an existing destination file. /H Copies hidden and system files also. My problem with my photos is that my camera takes pictures such as Photo001, Photo002, Photo003 and so on. And each time the camera is erased of images and new pictures are taken, it starts back at Photo001 and names the pictures the same as others even though the pictures are completely different. So I went with this method of getting all pictures to a USB stick, I can then go through and collapse the folders to GET all images into one folder, but I have to manually confirm that I wont be stomping a Photo001 that is in say E:\Data\Photo001.jpg with E:\DOCUMENTS and Settings\Dave\Pictures\Photo001.jpg I havent used any of the tools at that link yet. Thanks, Dave. For a backup, I used SmartCopy to do basically what your xcopy script did. I will save that, though, because I've needed to run backups like that for work docs, etc. I then made a ZIP copy of that backup - just to be safe! I created a worksheet and I'm just going to catalog - subdirectory, number of files, starting file (akin to your Photo001 problem), and size of subdirectory. I'm hopeful I have entire subdirectories that are dupes, and I don't have too many intermingled. I noticed ccleaner has a duplicate file finder, but it only works off name and size. Some of the ones in that link claim to be able to "read" the photo and determine dupes based off image. I suspect both would be a good scrub. I'll take any other advice! Just a word of advice...makin a zip is a good idea...i've seen dupe file finder apps completely bork a Win install waay too many times. Yeah, I figure even though it doesn't save much space - it's a "Safe Space" from any utilities. Duplicate finders that compute an md5 hash will find duplicate images. The hashes will match only if the files are identical on the binary level. There is a free "Duplicate Files Finder" on Sourceforge https://sourceforge.net/projects/doubles/?source=directory Instead of doing a hash, there is an old utility in DOS that compares two files. File Compare or FC as we will refer to is from here on out, is a simple program that will compare the contents of text or binary files and is capable of comparing both ASCII and Unicode text. You can use this tool to display any lines from two files or two sets of files that do not match up with the others. - from Google search. Quote from: Salmon Trout on December 08, 2016, 01:01:51 PM Duplicate finders that compute an md5 hash will find duplicate images. The hashes will match only if the files are identical on the binary level. There is a free "Duplicate Files Finder" on Sourceforge This sez it does NOT computer a hash... Running it now, we'll see. I've squirreled away partial archives here and there, I want to make sure I build One Archive to Rule Them All. Quote from: joshua on April 21, 2017, 04:02:06 PM try this program Duplicate Files Deleter Steer clear! Not even a proper link. Post removed...thanx for the heads up Salmon...

Answer»

I need to clean up my hard drive - I have MADE copies of folders filled with pictures, as well as folders with compressed and cropped versions.

I'd like to build an archive of just the original, raw pics.

I know I can grunt through and compare folders, etc., by hand, but I'm lazy. I'd rather spend four hours researching an easy solution than three hours of work.

That said, I ran across this article:
http://www.howtogeek.com/200962/how-to-find-and-remove-duplicate-files-on-windows/

The "VisiPics" OPTION looks interesting.

Has anyone ever used any software to do something like this? What tips or tricks might anyone have?Before using a tool that will clean up to remove duplicates, I would perform a backup of your data, just in case you have files named the same in different folders and same file size which are different photos. Its rare that a file will be different and be such a close match, but its safest to back up the data prior to putting the decision making into an automated program that does the cleanup.

I just run a batch script to run across my computer and grab all JPG images and store them to an external device such as USB stick at the E: drive. I run the following instructions which targets the C: drive and grabs a copy of only JPG files, and RETAINS their directory structure so that if there are images of same file name, but images are different, images are then not lost to which file is newest or written last of the same file name.

Code: [Select]c:
cd\.
xcopy *.jpg E:\*.* /s/d/y/h
echo.
echo. JPG Image Grab from C to E drive Complete
echo.
echo.
pause

Here is a list of what each switch does:

Quote

/S Copies directories and subdirectories except empty ones.

/D:m-d-y Copies files changed on or after the specified date.
If no date is given, copies only those files whose
source time is newer than the destination time.

/Y Suppresses prompting to confirm you want to overwrite an
existing destination file.

/H Copies hidden and system files also.

My problem with my photos is that my camera takes pictures such as Photo001, Photo002, Photo003 and so on. And each time the camera is erased of images and new pictures are taken, it starts back at Photo001 and names the pictures the same as others even though the pictures are completely different. So I went with this method of getting all pictures to a USB stick, I can then go through and collapse the folders to GET all images into one folder, but I have to manually confirm that I wont be stomping a Photo001 that is in say E:\Data\Photo001.jpg with E:\DOCUMENTS and Settings\Dave\Pictures\Photo001.jpg

I havent used any of the tools at that link yet.

Thanks, Dave.

For a backup, I used SmartCopy to do basically what your xcopy script did. I will save that, though, because I've needed to run backups like that for work docs, etc.

I then made a ZIP copy of that backup - just to be safe!

I created a worksheet and I'm just going to catalog - subdirectory, number of files, starting file (akin to your Photo001 problem), and size of subdirectory. I'm hopeful I have entire subdirectories that are dupes, and I don't have too many intermingled.

I noticed ccleaner has a duplicate file finder, but it only works off name and size. Some of the ones in that link claim to be able to "read" the photo and determine dupes based off image. I suspect both would be a good scrub.

I'll take any other advice!

Just a word of advice...makin a zip is a good idea...i've seen dupe file finder apps completely bork a Win install waay too many times.
Yeah, I figure even though it doesn't save much space - it's a "Safe Space" from any utilities.
Duplicate finders that compute an md5 hash will find duplicate images. The hashes will match only if the files are identical on the binary level. There is a free "Duplicate Files Finder" on Sourceforge

https://sourceforge.net/projects/doubles/?source=directory
Instead of doing a hash, there is an old utility in DOS that compares two files.
File Compare or FC as we will refer to is from here on out, is a simple program that will compare the contents of text or binary files and is capable of comparing both ASCII and Unicode text. You can use this tool to display any lines from two files or two sets of files that do not match up with the others.
- from Google search. Quote from: Salmon Trout on December 08, 2016, 01:01:51 PM

Duplicate finders that compute an md5 hash will find duplicate images. The hashes will match only if the files are identical on the binary level. There is a free "Duplicate Files Finder" on Sourceforge

This sez it does NOT computer a hash... Running it now, we'll see.

I've squirreled away partial archives here and there, I want to make sure I build One Archive to Rule Them All. Quote from: joshua on April 21, 2017, 04:02:06 PM

try this program Duplicate Files Deleter

Steer clear! Not even a proper link.
Post removed...thanx for the heads up Salmon...

Solve : Cataloging/Removing Duplicate Photos?

Discussion

No Comment Found

Related InterviewSolutions

Reply to Comment