1.

Solve : Moving chaotic data ... looking for suggestions?

Answer»

A friend of mine came to me with her laptop that her whole family uses and data is scattered everywhere. She was wondering if I could help her get the data organized into folders for types of files such as music, pictures, documents etc.

I was thinking that I could write up a batch file using the MOVE COMMAND, but am concerned over the possibility of data loss SINCE I dont see where the MOVE command verifies that the data was copied from point A to point B successfully as a checksum match prior to deleting the data's prior location.

I looked into xcopy as a possible solution since it will verify, but it also will replicate the data and if trying to write back to the source drive I will get a cyclic copy error.

So i am stumped trying to figure out the best way to get her chaos of data sorted into folders through multiple batch lines specifying each file extension to search and move if found with data integrity verification.

I remember years ago my computer science teacher trying to tell us not to use the MOVE command because it was dangerous, but I dont see any other way to make this happen.

Any suggestions with code snippets on how to safely achieve a data move with sorting and verification of data integrity when reading and writing to same drive?

Thanks

Check out TeraCopy, which is free, works in GUI or command line mode, and does CRC verification of copied & moved files.

http://www.codesector.com/teracopy.php
Yes, don't use the MOVE command. *

And there are way to do what you what. You can organize what seems to be a random distribution of stuff. Without moving stuff. Do you have a fair amount of drive space? I mean like at LEAST 20 per cent free?

Do you know for sure that file extensions are the only thing that matters?
What about the content of the files? That requires Human Reasoning.


* Using the MOVE command for lots of files you surely will get to a point where a file name is duplicated in another folder. Trying to move both files into the same directory will cause and error or you will delete one file which may have different content.
Find all your jpeg pictures with :


Code: [Select]C:\>dir /b /s *.jpg > jpeg.txt
use a For loop to copy all pictures in jpeg.txt to whereever you please.


C:\>type copyfiles.bat
Code: [Select]@echo off

setLocal EnableDelayedExpansion
cd \

dir /b /s *.jpg > jpeg.txt

for /f "delims==" %%a in (jpeg.txt) do (

copy "%%a" c:\jpeg\

)C:\>

Output:

04/14/2008 07:00 AM 25,759 wpakey.jpg
04/14/2008 07:00 AM 17,719 wpatop.jpg
11/19/2002 07:59 AM 9,161 wreath.jpg
05/02/2009 08:51 AM 1,301 wrenches_66x57.jpg
05/02/2009 08:51 AM 3,550 wsadvisor_logo_66x57.jpg
07/08/2009 07:33 PM 1,480 xp_logo_50x50.jpg
07/17/2009 05:31 PM 20,866 yellowrose.jpg
11/20/2008 10:16 PM 5,857 yellow_tiles.jpg
01/30/2009 12:50 AM 46,135 ZHCN_dell_bg_firstrun.jpg
01/30/2009 12:50 AM 46,126 ZHTW_dell_bg_firstrun.jpg
02/13/2003 03:35 AM 18,669 z_burst1.jpg
02/13/2003 03:35 AM 19,910 z_burst2.jpg
02/13/2003 03:35 AM 16,985 z_heart.jpg
02/13/2003 03:35 AM 17,611 z_kidney.jpg
02/13/2003 03:35 AM 17,004 z_oval.jpg
05/11/2009 09:33 PM 81,709 {46CAC505-936C-4891-891B-D1ABB86F4
use3.jpg
724 File(s) 55,246,166 bytes
2 Dir(s) 306,873,536,512 bytes free

C:\jpeg>yellowrose.jpg

C:\jpeg>

Rem the " " is necessary since the path to a picture might contain a space
copy "%%a" c:\jpeg\Thanks everyone for your help, and verifying that the instructor was correct in avoiding "MOVE" command years ago back in High school.

Yah Geek-9PM good point about reasoning... the other issue was to maintain some of the structure she has, where a replication of a tree would be established after Pictures folder category and Documents folder category as say in the path

C:\vacation\ Bermuda contained both a bunch of JPG photos and a poem she wrote while on vacation as a .DOC file

the tree would remain intact and so at

C:\Data\Pictures the sub directory tree would include

c:\Data\Pictures\Vacation\Bermuda and all the JPGs


and at

C:\Data\Documents\Vacation\Bermuda she would have her single document poem


*So if she is hunting for pictures all pictures are found in that PICTURES Folder Tree under C:\data\pictures\

and her poem as well as any other .DOC, .RTF, or any other document format that she needs will be in the C:\Data\Documents and maintain her prior group of Vacation\Bermuda

By maintaining her old structure there would be no LIKE filename overwrite problems.

So what do you think? *Only potential problem I could see in this setup is hitting the maximum path length and clipping (losing acces to ) data at the edge of maximum file/folder path length. I pray that she doesnt have data that burried under multiples of sub-directories. Suppose I shoudl check...lol Quote

replication of a tree would be established
You get the idea. Make an abstract structure.
Create a directory with a tree structure that makes sense. Populate the tree with shortcuts. Most application programs, in my experience, have no trouble using a shortcut to find the actual location of a file. The shortcuts take very little disk space ans allow you to use an alias name and an alternate icon.

The are some programs that even do this for you. Some MP3 player software will organize your songs into play lists without the need to actually move the data. Also, photo album programs can do the same thing a deal with the problem of duplicated names.

And you can even replace the Windows Explorer (Not Internte Explorer) with another program that does better job of organizing this.

The advantage of using shortcuts is that you can move them, make new ones, delete them, modify them, rename them all without doing harm to the original file.
And they are quick, very little system, overhead.

Check this out:
http://techblogs.wordpress.com/2009/01/31/15-windows-explorer-alternatives/




Hey thats cool ... I didnt even think about that. But the problem there would be when she wants to backup her data, it is still scattered.... so that I why I am thinking that the data has to physically move to a new path of C:\data\....

Maybe I should use more than batch, but a thumb drive as a cache to store the data, an intermediary bucket to move the data to, "but without the move command". An instruction that XCopys the matching data to thumb drive, then back to the C:\data\... then have a CLEANUP process after a verification that the files locates at C:\..."wherever they came from" matches the new location at C:\data\...

I was thinking that this would fix the Cyclic Error issue with searching drive C and writing back to drive C and avoiding a loop. I suppose I could also use an exclusion, trying to figure out of the exclusion of path C:\data would work or if the thumb drive temporary file cache is a better idea. Thumb drive would likely slow down the process vs disk to disk without the USB communications bottleneck.

What do you think?
DaveLembke,
You are smarter that you need to be. It does not have to be that difficult.
The standard backup tools have all kinds of options that you can use to backup data and make special data sets with exceptions. You can even have the backup program detect files changed and make additional backups of just the files that have recently changed.

Normally all the user data should be in the Documents and Settings area, if using Windows XP. Some application programs will put stuff in the Programs folder, which is annoying. But you can have the backup set selectively backup part of the
programs folder.

What I love about the backup program it can know where everything relay belongs. You only have to tell it to restore to original location and it will do just that. And if yo chose another location, it will duplicate the tree structure. Trying to do this in batch is a big chore.

If you have another hard drive bailable you can transfer all of My Documents onto the spare drive and not ruin the My Documents link.Please read this:
http://support.microsoft.com/kb/310147

My preference is to have My Documents of the main user on another partition. That way I can just backup all of the drive. Other users on not in the partition. Some files I do not use anymore I put into the documents of the other users. That is one case where I use the move. I do not need to do that very often.

The are so many ways to backup it is hard to give a simple answer. But it is not realty hard. Just requires some fore thought.
http://www.genie-soft.com/asp/community/KnowledgeArticle.asp?KBID=105

Hope this is of some help. And I am sure OTHERS here even more ideas about how they would take on this kind of project.



Discussion

No Comment Found