Solve : Corrupted file detection methods thru batch & replace mismatch

1.	Solve : Corrupted file detection methods thru batch & replace mismatched files etc?
Answer» So my one computer I thought was protected by a battery backup and the 7 year old battery backup dropped like a stone during a short outage killing the computer mid process of some write processes. I learned that some data on the C: drive got corrupted in both my personal data that is at C:\DATA\ as well as I ended up having to uninstall and REINSTALL the video game RIFT which got corrupted through this. Was wondering if anyone knew of any neat batch tricks to test for corrupted files between an internal HDD which has had some files now become unable to open and an external hard drive that was not connected during this outage and is healthy, and if mismatches are found between C:\Data\ and H:\Backup\ to then copy the files from the C:\Data\ to H:\Mightbecorrupt\ and then copy the same file from H:\Backup\ back to C:\Data\ I then can look at the H:\Mightbecorrupt\ files and see if they were picked up as a mismatch because they are newer date/time stamp to that of the last backup same file older/date time stamp and likely binary mismatch or if these are the ones that are corrupt which wont open and so I know what data I lost or need to fix vs having to find out LATER when needing this information that I lost it I am thinking that I will need to perform a FC /B through a FOR loop through all files between C:\DATA\ and H:\Backup\ and will need to add an IF statement for mismatches to xcopy those mismatches to H:\Mightbecorrupt\ to retain directory path and file location structure, as well as if there is a mismatch to xcopy the could be older file from the H:\backup\ to C:\Data\ and to increment through all files and folders between C:\Data\ and H:\Backup\ that I will need to use a TREE or DIR output to start at the beginning alphanumerically and get to the end of the file/directory structure to test all, and this has put me at a loss on how to accomplish this. If I only had a hand full of files I would just take the time to open each one and test to see if they open ok, but because I have many in the thousands, I feel that this calls for a better automated compare method in which most files will be an exact match and only a choice few hopefully are corrupt or newer than the last backup which both newer as well as corrupted would get flagged under the FC /B process as a mismatch. At a loss on how to achieve this test between the 2 drives and identify the corrupted files because this goes beyond my batch knowledge. Also there may be a better method than that of FC, but I was thinking it is likely needed for testing between files for the slightest differences which means a healthy file, corrupted file, or a newer file than that of the backup data from within 7 days prior. Was playing aroudn with FC as seen below... the first test is with 2 files with a D and d difference, the other compare has exact text data in it and so there is no mismatches. I was thinking that maybe anything other than output of "FC: no differences encountered" would cause the batch to go through a xcopy of the C:\Data\potentially_troubled_file.ext to H:\Mightbecorrupt\ and then copy H:\Backup\whatever_the_path_is_to_this_file\known_good_file.ext to C:\Data\same_path_that_was_used\same_file_name_from_external_HDD.ext and then move on to the next file and skip copying any files that areno differences, but go through this process for files with differences detected in which the /B Binary flag is probably best to be used. Quote Microsoft Windows XP [Version 5.1.2600] (C) COPYRIGHT 1985-2001 Microsoft Corp. C:\Documents and Settings\user1>cd\. C:\>md TEST2 C:\>cd test2 C:\test2>edit test1.txt C:\test2>edit test2.txt C:\test2>fc /b test1.txt test2.txt Comparing files test1.txt and TEST2.TXT 00000000: 64 44 C:\test2>edit test2.txt C:\test2>fc /b test1.txt test2.txt Comparing files test1.txt and TEST2.TXT FC: no differences encountered C:\test2> Just an update... Digging online for methods using FC, I came across Foxidrive's response here http://www.pcreview.co.uk/forums/fc-exe-return-value-t2441497.html With this back in 2006 for another person looking for a batch method use of FC for 2 files ... Code: [Select]echo off fc /b file1 file2\|find /i "no differences">nul if errorlevel 1 echo miscompare if not errorlevel 1 echo compared ok Looks like if I make it like this, but able to test among any number of files in any number of folders between 2 root folder locations, it might work, although I need the checking to be between all files between the C:\Data\ and H:\Backup\ Quote echo off fc /b file1 file2\|find /i "no differences">nul if errorlevel 1 echo Mismatch Detected copy dynamicpath/filename from C:\Data\ to H:\mightbecorrupt\ copy the known good file from H:\Backup\%dyn_path%\%dyn_filename% to C:\Data\%dyn_path%\%dyn_filename% loop back ( goto loop condition or FOR loop? ) to test next file until all files compared among all files and folders between C:\Data\ and H:\Backup\if not errorlevel 1 echo compared ok ... moving on to next file checkecho Process Complete ... Any mismatched files can be found at H:\mightbecorrupt\ In red.... - I am unsure how to accomplish using the path from that of which the file was found and compared against to have this dynamic path for the fact that I cant think of a better way to explain it, get passed into a variable such as %dyn_path% in which I think I have to use SET %dyn_path% = ( the actual path to file, but not sure how to get this). Above I show %dyn_path% and %dyn_filename% .... this might be able to be passed for say C:\Data\Project1\test1.cpp directly into %dyn_path% without having to have C:\Data\Project1\ passed into %dyn_path% and test1.cpp passed directly into %dyn_filename% - Not sure also how to start at first file/folder alphanumerically to test all files in all folders between C:\Data\ and H:\Backup\ until End Of Files & Folders is found. Not sure if the best method to test between C:\Data\ and H:\Backup\ would be to create a copy of the entire directory trees between C:\Data\ and H:\Backup\ by use of an XCOPY TRICK where you can actually have it perform an XCOPY without writing files and have this info that is normally displayed written to file and then have a FOR loop that imports each line 1 by 1 which holds the paths to the files, and by use of a counter, can know to read in the path from say line 44 of this XCOPY text file that holds a copy of the directory tree to test against. This XCOPY tree would be generated from that of the C:\Data\ and to trick xcopy into thinking that it would normally need to copy files to say the root of H:\ to avoid error condition of cyclic copy condition, but instead of copying the files and folders list all paths with files would be to use of the /F ( full paths )and /L ( display only )switches and sending the output via > to TreeLayout.txt which is then read by the loop with the FC instructions and IF THEN ELSE logic of true or false for match or mismatch. Are the trees c:\data and h:\backup the same trees? Just 7 days different? (plus corrupted files) Quote Are the trees c:\data and h:\backup the same trees? Hello foxidrive Yes they are exactly the same trees and up to 7 days different since the backup is performed weekly. Right now I have held off on the backup to that external until i can get this issue resolved and instead am just saving data to a 32GB flash stick.This is untested: It will do a fairly swift compare for files that are missing and files that have different sizes and will create a batch file (which has a .txt extension for you to check first for any obvious errors) fc_compare.C.bat.txt The batch file is designed to compare the files with the same filesize and write a report if there is a binary mismatch. This will take much longer if there is a lot of GB of data involved. All log files will be written to the desktop. Code: [Select]echo off dir "c:\data" /b /s /a-d >"%userprofile%\desktop\c_data.txt" for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\c_data.txt") do ( if not exist "h:\backup\%%c" (>>"%userprofile%\desktop\h_data_missing.txt" echo missing "h:\backup\%%c") if exist "h:\backup\%%c" ( for %%m in ("c:\data\%%c") do for %%n in ("h:\backup\%%c") do ( if %%~zn EQU %%~zm ( >>"%userprofile%\desktop\fc_compare.C.bat.txt" echo fc /b "c:\data\%%c" "h:\backup\%%c" ^>nul ^\|^\| ^(^>^>"%userprofile%\desktop\fc_compare.result_C.txt" echo data mismatch in "c:\data\%%c" and "h:\backup\%%c"^) ) else ( >>"%userprofile%\desktop\fc_compare.C.size_mismatch.txt" echo size mismatch in "c:\data\%%c" and "h:\backup\%%c" ) ) ) dir "h:\backup" /b /s /a-d >"%userprofile%\desktop\h_data.txt" for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\h_data.txt") do ( if not exist "c:\data\%%c" (>>"%userprofile%\desktop\c_data_missing.txt" echo missing "c:\data\%%c") ) echo see if any errors appear on the console above pause hmm... so I am testing this on a Windows 7 64-bit system and it creates a c_data.txt file to desktop, but ends there. I ran this from cmd shell and it runs and exits back to C:\> so it seems to be terminating too early for some reason as for it never gets to the pause. The file on the desktop c_data.txt I have the following files in as copy/pasted from this file here. Quote c:\data\test1.txt c:\data\test2.txt c:\data\test3.txt c:\data\test4.txt c:\data\test5.txt c:\data\test6.txt It seems as though its carrying out this action and ending here: Code: [Select]dir "c:\data" /b /s /a-d >"%userprofile%\desktop\c_data.txt" This is where it leaves me off at after running, back to command prompt. And the directory info is written to file that is passed to desktop. Quote C:\batches>run_compare1.bat C:\batches> I'd forgotten about that task. Did it take 2 weeks to run? It was missing a closing parenthesis IIANM so try this. Code: [Select]echo off dir "c:\data" /b /s /a-d >"%userprofile%\desktop\c_data.txt" for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\c_data.txt") do ( if not exist "h:\backup\%%c" (>>"%userprofile%\desktop\h_data_missing.txt" echo missing "h:\backup\%%c") if exist "h:\backup\%%c" ( for %%m in ("c:\data\%%c") do for %%n in ("h:\backup\%%c") do ( if %%~zn EQU %%~zm ( >>"%userprofile%\desktop\fc_compare.C.bat.txt" echo fc /b "c:\data\%%c" "h:\backup\%%c" ^>nul ^\|^\| ^(^>^>"%userprofile%\desktop\fc_compare.result_C.txt" echo data mismatch in "c:\data\%%c" and "h:\backup\%%c"^) ) else ( >>"%userprofile%\desktop\fc_compare.C.size_mismatch.txt" echo size mismatch in "c:\data\%%c" and "h:\backup\%%c" ) ) )) echo stage one hacked&pause dir "h:\backup" /b /s /a-d >"%userprofile%\desktop\h_data.txt" for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\h_data.txt") do ( if not exist "c:\data\%%c" (>>"%userprofile%\desktop\c_data_missing.txt" echo missing "c:\data\%%c") ) echo see if any errors appear on the console above pause Quote I'd forgotten about that task. Did it take 2 weeks to run? Actually I just finally got around to testing it ... Looking forward to getting back to normal 40 hr work weeks vs these 60s. It wont be another 2 weeks before I test this though. Thanks for your help with this. Sweet this works perfect.... Thanks for your help with this.

Solve : Corrupted file detection methods thru batch & replace mismatched files etc?

Answer»

So my one computer I thought was protected by a battery backup and the 7 year old battery backup dropped like a stone during a short outage killing the computer mid process of some write processes.

I learned that some data on the C: drive got corrupted in both my personal data that is at C:\DATA\ as well as I ended up having to uninstall and REINSTALL the video game RIFT which got corrupted through this.

Was wondering if anyone knew of any neat batch tricks to test for corrupted files between an internal HDD which has had some files now become unable to open and an external hard drive that was not connected during this outage and is healthy, and if mismatches are found between C:\Data\ and H:\Backup\ to then copy the files from the C:\Data\ to H:\Mightbecorrupt\ and then copy the same file from H:\Backup\ back to C:\Data\

I then can look at the H:\Mightbecorrupt\ files and see if they were picked up as a mismatch because they are newer date/time stamp to that of the last backup same file older/date time stamp and likely binary mismatch or if these are the ones that are corrupt which wont open and so I know what data I lost or need to fix vs having to find out LATER when needing this information that I lost it

I am thinking that I will need to perform a FC /B through a FOR loop through all files between C:\DATA\ and H:\Backup\ and will need to add an IF statement for mismatches to xcopy those mismatches to H:\Mightbecorrupt\ to retain directory path and file location structure, as well as if there is a mismatch to xcopy the could be older file from the H:\backup\ to C:\Data\ and to increment through all files and folders between C:\Data\ and H:\Backup\ that I will need to use a TREE or DIR output to start at the beginning alphanumerically and get to the end of the file/directory structure to test all, and this has put me at a loss on how to accomplish this.

If I only had a hand full of files I would just take the time to open each one and test to see if they open ok, but because I have many in the thousands, I feel that this calls for a better automated compare method in which most files will be an exact match and only a choice few hopefully are corrupt or newer than the last backup which both newer as well as corrupted would get flagged under the FC /B process as a mismatch.

At a loss on how to achieve this test between the 2 drives and identify the corrupted files because this goes beyond my batch knowledge. Also there may be a better method than that of FC, but I was thinking it is likely needed for testing between files for the slightest differences which means a healthy file, corrupted file, or a newer file than that of the backup data from within 7 days prior.

Was playing aroudn with FC as seen below... the first test is with 2 files with a D and d difference, the other compare has exact text data in it and so there is no mismatches. I was thinking that maybe anything other than output of "FC: no differences encountered" would cause the batch to go through a xcopy of the C:\Data\potentially_troubled_file.ext to H:\Mightbecorrupt\ and then copy H:\Backup\whatever_the_path_is_to_this_file\known_good_file.ext to C:\Data\same_path_that_was_used\same_file_name_from_external_HDD.ext and then move on to the next file and skip copying any files that areno differences, but go through this process for files with differences detected in which the /B Binary flag is probably best to be used.

Quote

Microsoft Windows XP [Version 5.1.2600]
(C) COPYRIGHT 1985-2001 Microsoft Corp.

C:\Documents and Settings\user1>cd\.

C:\>md TEST2

C:\>cd test2

C:\test2>edit test1.txt

C:\test2>edit test2.txt

C:\test2>fc /b test1.txt test2.txt
Comparing files test1.txt and TEST2.TXT
00000000: 64 44

C:\test2>edit test2.txt

C:\test2>fc /b test1.txt test2.txt
Comparing files test1.txt and TEST2.TXT
FC: no differences encountered

C:\test2>

Just an update... Digging online for methods using FC, I came across Foxidrive's response here http://www.pcreview.co.uk/forums/fc-exe-return-value-t2441497.html

With this back in 2006 for another person looking for a batch method use of FC for 2 files ...

Code: [Select]echo off
fc /b file1 file2|find /i "no differences">nul
if errorlevel 1 echo miscompare
if not errorlevel 1 echo compared ok
Looks like if I make it like this, but able to test among any number of files in any number of folders between 2 root folder locations, it might work, although I need the checking to be between all files between the C:\Data\ and H:\Backup\

Quote

echo off
fc /b file1 file2|find /i "no differences">nul
if errorlevel 1 echo Mismatch Detected
copy dynamicpath/filename from C:\Data\ to H:\mightbecorrupt\
copy the known good file from H:\Backup\%dyn_path%\%dyn_filename% to C:\Data\%dyn_path%\%dyn_filename%
loop back ( goto loop condition or FOR loop? ) to test next file until all files compared among all files and folders between C:\Data\ and H:\Backup\if not errorlevel 1 echo compared ok ... moving on to next file checkecho Process Complete ... Any mismatched files can be found at H:\mightbecorrupt\

In red....

- I am unsure how to accomplish using the path from that of which the file was found and compared against to have this dynamic path for the fact that I cant think of a better way to explain it, get passed into a variable such as %dyn_path% in which I think I have to use SET %dyn_path% = ( the actual path to file, but not sure how to get this). Above I show %dyn_path% and %dyn_filename% .... this might be able to be passed for say C:\Data\Project1\test1.cpp directly into %dyn_path% without having to have C:\Data\Project1\ passed into %dyn_path% and test1.cpp passed directly into %dyn_filename%

- Not sure also how to start at first file/folder alphanumerically to test all files in all folders between C:\Data\ and H:\Backup\ until End Of Files & Folders is found. Not sure if the best method to test between C:\Data\ and H:\Backup\ would be to create a copy of the entire directory trees between C:\Data\ and H:\Backup\ by use of an XCOPY TRICK where you can actually have it perform an XCOPY without writing files and have this info that is normally displayed written to file and then have a FOR loop that imports each line 1 by 1 which holds the paths to the files, and by use of a counter, can know to read in the path from say line 44 of this XCOPY text file that holds a copy of the directory tree to test against. This XCOPY tree would be generated from that of the C:\Data\ and to trick xcopy into thinking that it would normally need to copy files to say the root of H:\ to avoid error condition of cyclic copy condition, but instead of copying the files and folders list all paths with files would be to use of the /F ( full paths )and /L ( display only )switches and sending the output via > to TreeLayout.txt which is then read by the loop with the FC instructions and IF THEN ELSE logic of true or false for match or mismatch.

Are the trees c:\data and h:\backup the same trees?

Just 7 days different? (plus corrupted files) Quote

Are the trees c:\data and h:\backup the same trees?

Hello foxidrive Yes they are exactly the same trees and up to 7 days different since the backup is performed weekly. Right now I have held off on the backup to that external until i can get this issue resolved and instead am just saving data to a 32GB flash stick.This is untested:

It will do a fairly swift compare for files that are missing and files that have different sizes
and will create a batch file (which has a .txt extension for you to check first for any obvious errors)

fc_compare.C.bat.txt

The batch file is designed to compare the files with the same filesize and write a report if there is a binary mismatch.
This will take much longer if there is a lot of GB of data involved.

All log files will be written to the desktop.

Code: [Select]echo off
dir "c:\data" /b /s /a-d >"%userprofile%\desktop\c_data.txt"

for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\c_data.txt") do (
if not exist "h:\backup\%%c" (>>"%userprofile%\desktop\h_data_missing.txt" echo missing "h:\backup\%%c")
if exist "h:\backup\%%c" (
for %%m in ("c:\data\%%c") do for %%n in ("h:\backup\%%c") do (
if %%~zn EQU %%~zm (
>>"%userprofile%\desktop\fc_compare.C.bat.txt" echo fc /b "c:\data\%%c" "h:\backup\%%c" ^>nul ^|^| ^(^>^>"%userprofile%\desktop\fc_compare.result_C.txt" echo data mismatch in "c:\data\%%c" and "h:\backup\%%c"^)
) else (
>>"%userprofile%\desktop\fc_compare.C.size_mismatch.txt" echo size mismatch in "c:\data\%%c" and "h:\backup\%%c"
)
)
)

dir "h:\backup" /b /s /a-d >"%userprofile%\desktop\h_data.txt"

for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\h_data.txt") do (
if not exist "c:\data\%%c" (>>"%userprofile%\desktop\c_data_missing.txt" echo missing "c:\data\%%c")
)
echo see if any errors appear on the console above
pause
hmm... so I am testing this on a Windows 7 64-bit system and it creates a c_data.txt file to desktop, but ends there. I ran this from cmd shell and it runs and exits back to C:\> so it seems to be terminating too early for some reason as for it never gets to the pause.

The file on the desktop c_data.txt I have the following files in as copy/pasted from this file here.

Quote

c:\data\test1.txt
c:\data\test2.txt
c:\data\test3.txt
c:\data\test4.txt
c:\data\test5.txt
c:\data\test6.txt

It seems as though its carrying out this action and ending here:

Code: [Select]dir "c:\data" /b /s /a-d >"%userprofile%\desktop\c_data.txt"
This is where it leaves me off at after running, back to command prompt. And the directory info is written to file that is passed to desktop.

Quote

C:\batches>run_compare1.bat
C:\batches>

I'd forgotten about that task. Did it take 2 weeks to run?

It was missing a closing parenthesis IIANM so try this.

Code: [Select]echo off
dir "c:\data" /b /s /a-d >"%userprofile%\desktop\c_data.txt"

for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\c_data.txt") do (
if not exist "h:\backup\%%c" (>>"%userprofile%\desktop\h_data_missing.txt" echo missing "h:\backup\%%c")
if exist "h:\backup\%%c" (
for %%m in ("c:\data\%%c") do for %%n in ("h:\backup\%%c") do (
if %%~zn EQU %%~zm (
>>"%userprofile%\desktop\fc_compare.C.bat.txt" echo fc /b "c:\data\%%c" "h:\backup\%%c" ^>nul ^|^| ^(^>^>"%userprofile%\desktop\fc_compare.result_C.txt" echo data mismatch in "c:\data\%%c" and "h:\backup\%%c"^)
) else (
>>"%userprofile%\desktop\fc_compare.C.size_mismatch.txt" echo size mismatch in "c:\data\%%c" and "h:\backup\%%c"
)
)
))

echo stage one hacked&pause

dir "h:\backup" /b /s /a-d >"%userprofile%\desktop\h_data.txt"

for /f "usebackq tokens=1,2,* delims=\" %%a in ("%userprofile%\desktop\h_data.txt") do (
if not exist "c:\data\%%c" (>>"%userprofile%\desktop\c_data_missing.txt" echo missing "c:\data\%%c")
)
echo see if any errors appear on the console above
pause Quote

I'd forgotten about that task. Did it take 2 weeks to run?

Actually I just finally got around to testing it ... Looking forward to getting back to normal 40 hr work weeks vs these 60s. It wont be another 2 weeks before I test this though. Thanks for your help with this. Sweet this works perfect.... Thanks for your help with this.

Solve : Corrupted file detection methods thru batch & replace mismatched files etc?

Discussion

No Comment Found

Related InterviewSolutions

Reply to Comment