1.

Solve : Trying to figure out how to do this, should be simple, but hit a wall?

Answer»

So I have an old 80GB external hard drive at E:\ that has data some of it redundant scattered like a mess among many directories and I WANT to copy all files on this external hard drive to a single folder at C:\cleanedup\ but the part that I am stumped on is how to get a copy of all files to C:\cleanedup\ with an xcopy /d/y so that I only get the newest file of redundant file copied to C:\cleanedup\ and not have the directory paths reconstructed at c:\cleanedup\

Attempt to try and use just xcopy /d/y will only copy files in the root at which the command is executed, and I need it to fetch the files out of the many directories and subdirectories from within all on E:\ and pass all files to a single folder at C:\cleanedup\ and looking for a solution to make this happen.

example:

E:\data1\somefile1.txt ( date stamp of say 4:30pm 7/27/2006 )
E:\data2\somefile1.txt ( date stamp of say 5:39pm 8/22/2006 )
E:\somefile1.txt ( date stamp of say 7:39pm 9/21/2007 )
E:\somefile2.txt


and it will copy only the newest somefile1.txt and somefile2.txt to C:\cleanedup\I think I may have found a way finally. Reworded google search and found this: http://stackoverflow.com/questions/15209559/copy-all-files-recursively-into-a-single-folder-without-recreating-folders

The person who started this at the site wanted .mp3 files and I am going to try it with wildcard of all *.* files and see if that works.

Code: [Select]e:
for /d %a in (*) do @copy %a\*.* c:\cleanedupGot an interesting error I have never seen before:

Quote

E:\>%a\*.* was unexpected at this time.

E:\>

Any suggestions on how to correct this? OS of system I am testing this on is Windows XP Home SP3Code from stackoverflow - it's a command line and you need to double all % to execute it in a batch file.

Code: [Select]for /d %a in (*) do @copy %a\*.mp3 e:\myfolder
It won't handle filename collisions.




The easiest way to handle this is to add a counter to each filename, so they are unique names.

filename 0001.ext
apple 0002.ext
apple 0003.ext
banana 0004.ext

etcHello Foxidrive... thanks for your help. I forgot about using the extra % as an escape character.

In regards to appending a number to the end of the files... I was looking into if DIR /S /B /O:d could be used to sort all files by oldest first and newest last so that the order that the batch executes, the newest like file name would overwrite the oldest.

But its not working as intended and not sure why?

Here is the latest one I created:

Code: [Select]for /f "delims==" %%k in ('dir e:\*.* /s /b /o:d') do copy "%%k" f:\test1\
pause
Looking at the /O switch for DIR, it looks as if I should be able to pass the output from DIR to the FOR loop and process oldest first and then newest last as the order in which the DIR output is oldest files first and newest files last, so it should be the same as an xcopy /d switch that overwrites the old same file name with the new same file name. ( But not working out as intended when testing )

Quote
C:\>dir /?
Displays a list of files and subdirectories in a directory.

DIR [drive:][path][filename] [/A[[:]attributes]] [/B] [/C] [/D] [/L] [/N]
[/O[[:]sortorder]] [/P] [/Q] [/R] [/S] [/T[[:]timefield]] [/W] [/X] [/4]

[drive:][path][filename]
Specifies drive, directory, and/or files to list.

/A Displays files with specified attributes.
attributes D Directories R Read-only files
H Hidden files A Files ready for archiving
S System files I Not content indexed files
L Reparse Points - Prefix meaning not
/B Uses bare format (no heading information or summary).
/C Display the thousand separator in file sizes. This is the
default. Use /-C to disable display of separator.
/D Same as wide but files are list sorted by column.
/L Uses lowercase.
/N New long list format where filenames are on the far right.
/O List by files in sorted order.
sortorder N By name (alphabetic) S By size (smallest first)
E By extension (alphabetic) D By date/time (oldest first)
G Group directories first - Prefix to reverse order
Press any key to continue . . .

My test was 3 folders such as folder1, folder2, and folder3 all containing a same test.txt file. In folder2 I have the newest date/time stamp version to test if its executing the process alpha numeric order or by the DIR O/:D method that should place them in an order in which it should be:

folder1\test.txt
folder3\test.txt
folder2\test.txt

so the latest/newest date time stamp file in folder2 should be the last file of like file name to write to the destination, but it seems to still be executing alpha numerically as folder1, folder2, and folder3 since the file written to the destination is the date/time stamp of that of the same date/time stamp that is on the test.txt file in folder3 .... so kind of stumped as to why its not working as I believe it should

The one way i can see this MAYBE malfunctioning is if its looking only at the roots of each folder to sort the files by date/time in which in will process the transfer alpha numerically but oldest to newest in each folder not taking into account the oldest from newest in neighboring folders.

Here is the output showing the process of transfer, which seems alpha numeric still when it shouldnt be with the DIR /O:D I would think.

Quote
Z:\>for /F "delims==" %k in ('dir e:\*.* /s /b /o:d') do copy "%k" f:\test1\

Z:\>copy "e:\folder1" f:\test1\
e:\folder1\test.txt
1 file(s) copied.

Z:\>copy "e:\folder2" f:\test1\
e:\folder2\test.txt
1 file(s) copied.

Z:\>copy "e:\folder3" f:\test1\
e:\folder3\test.txt
1 file(s) copied.

Z:\>copy "e:\test.txt" f:\test1\
1 file(s) copied.

Z:\>copy "e:\folder1\test.txt" f:\test1\
1 file(s) copied.

Z:\>copy "e:\folder2\insidefolder" f:\test1\
e:\folder2\insidefolder\test.txt
1 file(s) copied.

Z:\>copy "e:\folder2\test.txt" f:\test1\
1 file(s) copied.

Z:\>copy "e:\folder2\insidefolder\test.txt" f:\test1\
1 file(s) copied.

Z:\>copy "e:\folder3\test.txt" f:\test1\
1 file(s) copied.

Z:\>pause
Press any key to continue . . .

So the last:

Z:\>copy "e:\folder3\test.txt" f:\test1\
1 file(s) copied.

Is the last file to overwrite the destination when it should be the file located at folder2\insiderfolder\test.txt which has the newest date/time stamp among all like files at E:\>

<<< Update: >>>

Just tested the DIR /O:D and DIR /O:-D inverse and it appears that it only sorts the root directory files and not subdirectories Is there a way to make it look at subdirectory date/time stamps to sort the order oldest to newest?

Quote
E:\>dir
Volume in drive E is NEW VOLUME
Volume Serial Number is 089B-F770

Directory of E:\

09/01/2013 09:03 PM <DIR> folder1
09/01/2013 09:03 PM <DIR> folder2
09/01/2013 09:03 PM <DIR> folder3
09/01/2013 09:04 PM 14 test.txt
1 File(s) 14 bytes
3 Dir(s) 122,673,152 bytes free

E:\>dir /o:d
Volume in drive E is NEW VOLUME
Volume Serial Number is 089B-F770

Directory of E:\

09/01/2013 09:03 PM <DIR> folder1
09/01/2013 09:03 PM <DIR> folder2
09/01/2013 09:03 PM <DIR> folder3
09/01/2013 09:04 PM 14 test.txt
1 File(s) 14 bytes
3 Dir(s) 122,673,152 bytes free

E:\>dir /o:-d
Volume in drive E is NEW VOLUME
Volume Serial Number is 089B-F770

Directory of E:\

09/01/2013 09:04 PM 14 test.txt
09/01/2013 09:03 PM <DIR> folder3
09/01/2013 09:03 PM <DIR> folder2
09/01/2013 09:03 PM <DIR> folder1
1 File(s) 14 bytes
3 Dir(s) 122,673,152 bytes free

E:\>

Excuse me for not following all your text -

DIR will sort in a folder, but not in a folder tree.

You can use this code to get a list of timestamps and filenames.

Code: [Select]@echo off
cd \
del \list.txt 2>nul
for /r %%a in (*) do >>\list.txt echo %%~ta %%a
The resulting list.txt file can be changed to list in yymmdd hhmm format and then sorted to give you a list of oldest to newest file,
but to make this easier it will help to change your msdos date and time format to yyyy/mm/dd and 24 hour time hh:mm
and change it back to what you prefer afterward. Even just changing to 24 hour time will help to rewrite the file in a sortable format.

If you want to use this method (you can use this sorted file to easily copy all the files) then supply a few lines of the list.txt file
and someone can help massage it into a sortable format.

Thanks for your help foxidrive ... going to play around with suggestions and see if I can make it work somehow, but I have some doubts I will be able to get it to work because DOS has been around for many years and google is generally good at coming up with hits with other people that wanted to do something a certain way before yourself, and those who have and have instructions listed that can be placed into batches all dont seem to do the trick. The biggest issue is that DIR only works with the local folder that it is targeting and not with directory trees as you said and I have seen in testing. Have also come across a tool called xxcopy that I am going to check out. Never used it before and the google search for info on doing what I am trying to do has had info suggesting that others have used it for this purpose possibly.

Definately going to play around with the code you supplied though as for with all files in a list.txt in an order from oldest to newest etc, depending on what is written to this file, I could import that list to C++ or Perl and maybe process this list to transfer all files from start to finish to the single folder from oldest files first to newest files last so that only the newest version is located at the destination as for any processed prior of the same file name would be overwritten due to the order of processing the file copying.
Just wanted to follow up with the end result of what I ended up going with to ACHIEVE what I needed. After lots of digging, I found that XXCOPY with the /SGNO switch will copy all files from a source with directory tree all to 1 location collapsing the directory tree and it keeps the newest files of like file names in what it copies. I DOWNLOADED the freeware version and it works flawlessly on my Windows XP Home SP3 Netbook that I tested it on with a flash drive as the source drive with test files in a junk test tree.

Below is the first run of it with test data in a test tree with data hidden in subdirectories.

Quote
C:\>xxcopy f:\*.* c:\crush1\ /SGNO

XXCOPY == Freeware == Ver 3.21.0 (c)1995-2013 Pixelab, Inc.

-------------------------------------------------------------------------------
F:\testfile1.txt 80
F:\testfile2.txt 34
F:\testfile3.txt 34
F:\testfile4.txt 34
F:\1\fff\fffff\testthis\testfile5.txt 34
F:\2\testfile1.txt 80
-------------------------------------------------------------------------------
Directories Processed = 7
Total Data in Bytes = 296
Elapsed time in sec. = 0.078
Action speed (MB/min) = 0.2277
Files Examined = 8
Files Copied = 6
Exit code = 0 (No error, Successful operation)

C:\>

I then tested to see if xxcopy acted like xcopy in that it would update/copy over only the single file that changed, and it did as seen below.

Quote
C:\>xxcopy f:\*.* c:\crush1\ /SGNO

XXCOPY == Freeware == Ver 3.21.0 (c)1995-2013 Pixelab, Inc.

-------------------------------------------------------------------------------
F:\2\testfile1.txt 22
-------------------------------------------------------------------------------
Directories Processed = 7
Total Data in Bytes = 22
Elapsed time in sec. = 0.047
Action speed (MB/min) = 0.02809
Files Examined = 8
Files Copied = 1
Exit code = 0 (No error, Successful operation)


C:\>

So I'm all set and thanks for help. I wanted to post back with what I ended up doing so that anyone else in the future who finds this will save themselves hours of trial and error trying to get it to work.


Discussion

No Comment Found