1.

Solve : Compare folder sizes between 2 externals and list only mismatched folder info?

Answer»

I am thinking this might be beyond batch. I realized this morning checking on my 2 external drives that chugged through the night since 6pm yesterday transferring data that I had made a mistake in how I should have gone about copying data from one to the other.

The mistake was that I had already added a lot of data from my local machines hard drive to the 1.5TB external prior to running an xcopy instruction to transfer all data from my 500GB external to my 1.5TB external. When checking on drive properties to see how many files and folders transferred and data size I realized what i had done.

So other than manually clicking on the properties of each folder and writing down the properties and comparing both drives looking for folder mismatches, I figured I'd post this here to see if anyone knows of a creative way to go about doing this.

I looked at the FC command, but thats mainly File Compare and not a Folder Comparer. I am guessing that a loop may be needed with DIR command to visit each folder between G: and F: and compare properties between them, then if they match do not display, but if they are mismatched display a list of mismatches so I can manually check those out to see why there is a mismatch. This is beyond my batching abilities so I figured I'd run it past you all to see if there is a method to do this that doesnt require a full day of coding it up. I found a piece of software called WinMerge but it still leaves me with a mess of information not FILTERING out the matches, so it displays all. If this is something that is beyond batch I can use whatever tools are suggested as well. Just thinking that a batch might be able to chug through comparing both drives folder sizes comparing and listing only ones that are mismatched properties.

I could always wipe the 1.5TB clean and start over copying the 500GB to the 1.5TB clean, then be able to check properties cleanly, but I'd rather not do that if it can be AVOIDED. Having drives transferring data non stop for in excess of 12 hours worries me that I might over-work a drive and have a crash with total data loss. For the fact that they are bottlenecked across USB 2.0 I dont think they are breaking a sweat as if directly SATA linked for 12 hrs chugging, but if direct SATA linked at least the drive would only be active for a few hours transferring at full speed if that. Unfortunately these externals are USB only connectivity without CRACKING them open to gain direct SATA access.

[year+ old attachment deleted by admin]I am a bit unclear on what it is you mean by a "difference". Do you mean that if G:\Root\Branch\Folder has 23 files and F:\Root\Branch\Folder has 0 files that is 23 "differences"? Or one? What are you looking for?
You said you "realised what you had done". What had you done?

Quote

I am a bit unclear on what it is you mean by a "difference". Do you mean that if G:\Root\Branch\Folder has 23 files and F:\Root\Branch\Folder has 0 files that is 23 "differences"? Or one? What are you looking for?
You said you "realised what you had done". What had you done?

Question #1 - Yes I am looking for a mismatch between the 2, so that if both = 23 files it would move on and not report back, but if one had say 22 and the other 23, that it would possibly write to a logfile this inequality between the 2 locations.

*I would also like to add that I put further thought into this and thought that you could grab the structure of the 500GB drive from a DIR output and use that as a reference to test against the 1.5TB drive, so that its only looking for inequalities between both externals that are directly related to the folder structure of the original 500GB drive. All data added that came from the C: drive of the local machine that is blended in on the 1.5TB would be skipped over so that I wouldnt have those listed as inequalities between the 2 drives.

Further information on that question you asked if I wanted to know about a single problem or 23 problems with the relation of 0 and 23 files. It doesnt have to specify 23 problems at a single location if that involves more work to achieve that. If I had a log output to say mismatch.log that displays just the folder path to an inequality that would be good enough for me to navigate and look to see where the problem is. I m guessing that it would be best for this to execute from C:\ to compare between G: and F: and write back to C:\ the mismatch.log information to avoid the mismatch.log from being added to itself if for example that is written to either external drive.

Question # 2 - I realized what I had done ...

I should have copied all data from the 500GB to the 1.5TB with the 1.5TB empty of data. So that could have just done a simple compare between them via properties and verify that it was an exact copy without data lost in the transfer. BUT the problem I created for myself was that prior to the xcopy g:\*.* f:\*.* /s/d/y to copy all data from one external to the other, I copied a large amount of data from the local machines C: drive to this external so it already had about 300GB of data onto it before copying over all data from G: to F:, so in the end I realized that I shouldnt have copied data to this 1.5TB in that order. I should have copied to a clean 1.5TB drive, done a comparison, then when satisfied that all was well then add data from the local machines C: to this 1.5TB.

I could just dump all data from the 1.5TB and start over in the correct order to compare, but figured I'd post here to see if there was a way to check that the data from G: and F: match in relation to the data at G:, when F: will have extra files and folders that should be ignored in this comparison. There use to be a TREE command that I think would have worked out better for such a comparison in which you could TREE and write the output from TREE from G: to a file and then increment each path from that output to test against the destination of F:, but I think I saw a say to do this with DIR and a FOR loop a while back. And this is batching territory that is beyond what i can code up and I figured I'd check with you here to see if its something simple. Salmon - You have amazed me with the simplicity of batches in the past that perform powerful tasks, so I figured I would throw this out there in case this is something that can be done in which your expertise of batch programming shines over my lack of knowledge on the matter.

THANKS for assisting with this Robocopy may be able to help you achieve your goal.
For instance, to list the files in source_path which are not in target_path:
Code: [Select]Robocopy source_path target_path /e /ndl /l
This command will only list the missing files. If you want the missing files to be copied to the target_path, remove the /l from the command line.
If you want the result written to a file, append /log:path_to_log_file to the command line.
This should give you a list of the missing files. Hidden files will be also listed as missing.

Code: [Select]@echo off
dir g:\ /b /s /a-d>%temp%\list.list

for /f "delims=" %%a in (%temp%\list.list) do (
if not exist "f:%%~pnxa" >>g:\MissingFiles.txt echo %%a
)Thanks Everyone! I ran with what foxidrive posted and it worked perfect. Now to look thru the list and manually transfer what is missing.

BTW I was looking it over and trying to figure out what

Quote
"f:%%~pnxa"
does, the ~pnxa is something i have never seen before. I like to see why it does what it does than to simply just run with it to perform a task. Thanks for explaining You can run it again and make it copy the files - using xcopy it will recopy all the hidden/system files but it should be a full copy.

Code: [Select]@echo off
dir g:\ /b /s /a-d>%temp%\list.list

for /f "delims=" %%a in (%temp%\list.list) do (
if not exist "f:%%~pnxa" xcopy /h/y "%%a" "f:%%~pa\"
)
Or if you kept the file on g: then use this:


Code: [Select]@echo off
for /f "delims=" %%a in (g:\MissingFiles.txt) do (
xcopy /h/y "%%a" "f:%%~pa\"
)
"%%~pnxa" is made up from parts of the %%a variable - p is the path, n is the name of the file, x is the .extension and a is the variable name. f: at the start adds the target drive letter.

See the help for this, at the bottom.
FOR /?Thanks for explaining that. Makes sense now. With the lengthy help instructions of FOR I never seen that one before till now.

Also thank you for showing how to xcopy hidden files in reference to MissingFiles.txt from G: to F: Very Cool!

Quote
@echo off
for /f "delims=" %%a in (g:\MissingFiles.txt) do (
xcopy /h/y "%%a" "f:%%~pa\"
)
Quote from: DaveLembke on September 16, 2012, 06:19:47 PM
Also thank you for showing how to xcopy hidden files in reference to MissingFiles.txt from G: to F: Very Cool!

Your welcome. In case I wasn't clear, it will copy all the missing files - and recopy the hidden files if they exist.


Discussion

No Comment Found

Related InterviewSolutions