I'm using 'FOR /F' to change every occurrence of 'X' to 'Y' in every line of a text file.

So far so good, but problems occur if the text file contains exclamation marks [!]. Exclamation marks, and any text between them are lost.

My input file [Test-myfile.txt] contains...

Line 01 containing A but not X!
Line 02 containing A but not X!Watch this space!X
Line 03 containing X but not Y! and another xylophone.
Line 04 containing X and Y and another x and a word like axiom.
Line 05 containing Y but not X;
Line 06 containing Xxx but not Yyy:
Line 07 containing B and Z and Y~
Line 08 containing Y but not X#
Line 09 containing X and Y and another xylophone

My MS-DOS batch code [Test_Change_CharactersInStrings_110508.bat] is as follows...

@ECHO OFF
COLOR F5

REM====================================================
REMCode to change every occurrence of 'X' to 'Y' --
REMin every line of 'Test-myfile.txt'...
REM====================================================
REMNeed to check why exclamation marks [!], and any text between them are lost???
REM====================================================

@ECHO OFF > newfile.txt
SETLOCAL EnableDelayedExpansion
FOR /F "tokens=1,2,3,4,5,6,7-15* delims= " %%a in (Test-myfile.txt) DO (
REMPAUSE
ECHO a[%%a]
REMPAUSE
ECHO b[%%b]
ECHO c[%%c]
ECHO d[%%d]
ECHO e[%%e]
ECHO f[%%f]
ECHO g[%%g]
ECHO h[%%h]
ECHO i[%%i]
ECHO j[%%j]
ECHO k[%%k]
ECHO l[%%l]
ECHO m[%%m]
ECHO n[%%n]
ECHO o[%%o]
REMSET LINE_TEXT=%%a
SET LINE_TEXT=%%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o
ECHO Original line...!LINE_TEXT!
SET LINE_TEXT=!LINE_TEXT:X=Y!
ECHO Modified line...!LINE_TEXT!
ECHO !LINE_TEXT! >> newfile.txt
)
ECHO Done!
SETLOCAL DisableDelayedExpansion
ECHO Last modified line...%LINE_TEXT%
PAUSE
EXIT

After running the code, my output file [newfile.txt] contains...

Line 01 containing A but not Y
Line 02 containing A but not YY
Line 03 containing Y but not Y and another Yylophone.
Line 04 containing Y and Y and another Y and a word like aYiom.
Line 05 containing Y but not Y;
Line 06 containing YYY but not Yyy:
Line 07 containing B and Z and Y~
Line 08 containing Y but not Y#
Line 09 containing Y and Y and another Yylophone

Any ideas of how I can prevent exclamation marks, and any text between them, from being lost?
The string variable for each line is not being created correctly because with delayed expansion enabled, exclamation marks are variable delimiters and thus are ignored along with everything between them. Batch scripts get terribly fiddly when special characters ("poison characters" are embedded in strings to be processed. The worst is the ampersand but parentheses and percent signs are among the others.

What I did below is to take the set of tokens spat out by FOR /F and create a string enclosed in quotes so I could pass it as one single parameter to a CALLed subroutine where the tilde variable modifier strips off the quotes and then the X=Y substitution takes place, followed by the output to the file. No delayed expansion needed.

Consider "CALL :replace" to be equivalent to "GOSUB replace" in BASIC, and "GOTO :eof" to be equivalent to "RETURN" in BASIC. The colons are obligatory, not optional like they are when using GOTO with labels.

batch script
Code: [Select]@ECHO OFF
REM ====================================================
REM Code to change every occurrence of 'X' to 'Y' --
REM in every line of 'Test-myfile.txt'...
REM ====================================================
@ECHO OFF > newfile.txt
FOR /F "tokens=1,2,3,4,5,6,7-15* delims= " %%a in (Test-myfile.txt) DO (
call :replace "%%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o"
)
ECHO Done!
pause
exit

:replace
set string1=%~1
set string2=%string1:X=Y%
echo %string2% >> newfile.txt
goto :eof

newfile.txt
Code: [Select]Line 01 containing A but not Y!
Line 02 containing A but not Y!Watch this space!Y
Line 03 containing Y but not Y! and another Yylophone.
Line 04 containing Y and Y and another Y and a word like aYiom.
Line 05 containing Y but not Y;
Line 06 containing YYY but not Yyy:
Line 07 containing B and Z and Y~
Line 08 containing Y but not Y#
Line 09 containing Y and Y and another Yylophone
ST, thank you very much for providing a solution to my problem -- I had unsuccessfully searched several websites looking for information on it before you came up with the goods.

One further point about my example is that I notice that the character substitution changes 'X' or 'x' (upper and lower case) to 'Y' (the resulting character is the case as specified). Is there a way to preserve the original case of the target character? (i.e. 'X' becomes 'Y', and 'x' becomes 'y'). Thanks in anticipation.

Incidently, ST, I tried to thank you by clicking the 'Thank Salmon Trout' button but GOT...

"An Error Has Occurred! Sorry, you can't repeat a karma action without waiting 2 hours."

Don't know what that's all about!
Give this a try

Code: [Select]@ECHO OFF
REM ====================================================
REM Code to change every occurrence of 'X' to 'Y' --
REM in every line of 'Test-myfile.txt'...
REM ====================================================
@ECHO OFF > newfile.txt
FOR /F "tokens=1,2,3,4,5,6,7-15* delims= " %%a in (Test-myfile.txt) DO (
call :replace "%%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o"
)
ECHO Done!
pause
exit

:replace
set string1=%~1
set string2=
set j=0
:Loop
call set inchar=%%string1:~%j%,1%%
if "%inchar%"=="" goto ExitLoop
set outchar=%inchar%
IF "%inchar%"=="X" set outchar=Y
IF "%inchar%"=="x" set outchar=y
set "string2=%string2%%outchar%"
set /a j=%j%+1
goto Loop
:ExitLoop
echo %string2% >> newfile.txt
goto :eof
Alternative in which a one-line VBScript is created, used and destroyed; it will run slightly more slowly. Note that FOR loop metavariables are case sensitive, so %%A will not interfere with %%a. Note also that the first FOR loop can be one line.

Code: [Select]@ECHO OFF
REM ====================================================
REM Code to change every occurrence of 'X' to 'Y' --
REM in every line of 'Test-myfile.txt'...
REM ====================================================
echo wscript.echo Replace(wscript.arguments(0), wscript.arguments(1), wscript.arguments(2))>Srep.vbs
@ECHO OFF > newfile.txt
FOR /F "tokens=1,2,3,4,5,6,7-15* delims= " %%a in (Test-myfile.txt) DO call :replace "%%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o"
ECHO Done!
del Srep.vbs>nul
pause
exit

:replace
set string2=%~1
for /f "delims=" %%A in ( ' cscript //nologo Srep.vbs "%string2%" "X" "Y" ' ) do set string2=%%A
for /f "delims=" %%A in ( ' cscript //nologo Srep.vbs "%string2%" "x" "y" ' ) do set string2=%%A
echo %string2% >> newfile.txt
goto :eof
ST, thanks again for the 2 solutions which you provided -- both of which preserve case sensitivity.

Am I right in assuming that the use of double-quotes in the [SET "string2=%string2%%outchar%"] statement -- in the non-VBS version -- is a way of ensuring that spaces, currently at the end of the string being processed, are not lost?Quote from: TomTheCabinBoy on May 11, 2011, 04:59:14 AM

ST, thanks again for the 2 solutions which you provided -- both of which preserve case sensitivity.

Am I right in assuming that the use of double-quotes in the [SET "string2=%string2%%outchar%"] statement -- in the non-VBS version -- is a way of ensuring that spaces, currently at the end of the string being processed, are not lost?

REM For each line of the Input File, string processing requires 'EnableDelayedExpansion' --
REM which can only be executed a maximum of 16 times before reaching the --
REM 'Maximum setlocal recursion level' (despite also executing 'DisableDelayedExpansion').