1.

Solve : For /f Tokens?

Answer»

I was reading through one of the questions posted here, and geek's response to it, and it got me thinking. He said you can read at most 52 (he also said it could be 62 or more) variables from a file. Through my really quick and dirty research, I discovered that the number is actually 69 (both alphabet cases, numberic, and 7 mysterious variables). However, I found that the maximum number of tokens you can USE in one for command is 31 (trial and error, any more and the command refuses to run but doesn't give an error).
The code I tested with
Code: [Select]echo off
mode con cols=200
setlocal enabledelayedexpansion
for /l %%A in (10,10,620) do set var=!var!%%A
echo %var% > test.txt
for /f "tokens=1-31" %%0 in (test.txt) do echo %%0 %%1 %%2 %%3 %%4 %%5 %%6 %%7 %%8 %%9 %%A %%B %%C %%D %%E %%F %%G %%H %%I %%J %%K %%L %%M %%N %%O %%P %%Q %%R %%S %%T %%U %%V %%W %%X %%Y %%Z %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p %%q %%r %%s %%t %%u %%v %%w %%x %%y %%z
pause
As you can see if you run it, the variables 0-9 and A-N expand (10-100 and 180-310, respectively), but there is a 7-variable gap between 9 and A. Does anyone have an idea what would be in that 7-variable gap? It's not numbers >=10.Weird how I can't edit. Oh well. I just discovered something relevant while explaining this to another user on a different forum. There's also a 6-variable gap in the rollover between Z and a. So it works out like this:
0-9 >>> x 7 >>> A-Z >>> x 7 >>> a-z

There could also be something after z, but I have no way of testing this. So, instead of 69 variables, we're dealing with something greater or equal to 75. Quote from: Helpmeh on March 18, 2012, 12:06:26 AM

There could also be something after z, but I have no way of testing this.

You do have a way to check the "gaps". You have the prompt. You can do it manually or you could write something such as a vbscript that goes through the whole ascii table and CREATES, and shells out to, a command line like below, recording which ones work and which don't. (There are threads in the Google Groups archive of alt.msdos.batch.nt which touch on this topic)

C:\>for %{ in (*.txt) do echo %{
01.txt
1-sfc-problems.txt
1-sfcdetails.txt
1.txt
2.txt
3.txt
hello.txt

C:\>for %| in (*.txt) do echo %|
% was unexpected at this time.

C:\>for %~ in (*.txt) do echo %~
01.txt
1-sfc-problems.txt
1-sfcdetails.txt
1.txt
2.txt
3.txt
hello.txt


You have two issues here, I think:

First, what characters can be used as FOR metavariables (answer: practically every printable character which does not have a special meaning [e.g. % ! < > etc])

Quote
there is a 7-variable gap between 9 and A. Does anyone have an idea what would be in that 7-variable gap? It's not numbers >=10.

You will see the answer to this when you look at an ASCII chart.

Second the maximum number of tokens possible in a FOR construct; clearly this requires an unbroken consecutive run of valid variable characters and this has been well established to be from ASCII 63 '?' to ASCII 93 ']'



Quote from: Salmon Trout on March 18, 2012, 01:54:17 AM
what characters can be used as FOR metavariables (answer: practically every printable character which does not have a special meaning [e.g. % ! < > etc])

You will see the answer to this when you look at an ASCII chart.

I tested 1 to 255 and these failed, so, at a first glance, maybe you could use some non-printables, although you might have to use tricks to get them in a script.

ASCII   9
ASCII  10
ASCII  11
ASCII  12
ASCII  13
ASCII  32
ASCII  34 "
ASCII  37 %
ASCII  38 &
ASCII  44 ,
ASCII  59 ;
ASCII  60 <
ASCII  61 =
ASCII  62 >
ASCII  94 ^
ASCII 124 |
ASCII 160 á


You can do this

for /f "tokens=1-2" %} in ("a b") do echo %} %~

but not above ASCII 127 (the first token works, not the later ones)

but this works (☺ is ASCII 1 and ☻ is ASCII 2) (you type them by holding down ALT and typing the number on the keypad)

for /f "tokens=1-2" %☺ in ("a b") do echo %☺ %☻


This that I wrote above obviously only applies to one loop; by nesting you can get more simultaneously existing FOR variables.

Quote from: Me
the maximum number of tokens possible in a FOR construct; clearly this requires an unbroken consecutive run of valid variable characters and this has been well established to be from ASCII 63 '?' to ASCII 93 ']'

Quote from: Helpmeh
we're dealing with something greater or equal to 75.

I managed 81 so far, and I believe I could squeeze a few more out.

The essential point is that the command interpreter sees the variable character as an ASCII code, and it is quite happy, with the exceptions already noted, to deal with the codes that aren't letters of the alphabet or numbers, including ones that don't correspond to keys on the keyboard. I imagine that when they wrote the documentation they deliberately kept it simple and only mentioned A-Z and a-z and that's where the figure of 52 comes from.

echo off
set String1="v1 v2 V3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31"
set String2="v32 v33 v34 v35 v36 v37 v38 v39 v40 v41 v42 v43 v44 v45 v46 v47 v48 v49 v50 v51 v52 v53 v54 v55 v56 v57 v58 v59 v60"
Set String3="v61 v62 v63 v64 v65 v66 v67 v68 v69 v70 v71 v72 v73"
Set String4="v74 v75 v76 v77 v78 v79 v80 v81"
FOR /F "tokens=1-31" %%? in (%String1%) do (
        FOR /F "tokens=1-29" %%_ in (%String2%) do (
                FOR /F "tokens=1-14" %%- in (%String3%) do (
                        REM ASCII 1 to 8 [Won't display in CH]
                        FOR /F "tokens=1-8" %% in (%String4%) do (
                                Echo %%? %% %%A %%B %%C %%D %%E %%F %%G %%H %%I %%J %%K %%L %%M %%N %%O %%P %%Q %%R %%S %%T %%U %%V %%W %%X %%Y %%Z %%[ %%\ %%]
                                Echo %%_ %%` %%a %%b %%c %%d %%e %%f %%g %%h %%i %%j %%k %%l %%m %%n %%o %%p %%q %%r %%s %%t %%u %%v %%w %%x %%y %%z %%{
                                Echo %%- %%. %%/ %%0 %%1 %%2 %%3 %%4 %%5 %%6 %%7 %%8 %%9 %%:
                                Echo %% %% %% %% %% %% %% %%
                                )
               
                        )
                )
        )



Output:

v1 v2 v3 v4 v5 v6 v7 v8 v9 v10 v11 v12 v13 v14 v15 v16 v17 v18 v19 v20 v21 v22 v23 v24 v25 v26 v27 v28 v29 v30 v31
v32 v33 v34 v35 v36 v37 v38 v39 v40 v41 v42 v43 v44 v45 v46 v47 v48 v49 v50 v51 v52 v53 v54 v55 v56 v57 v58 v59 v60
v61 v62 v63 v64 v65 v66 v67 v68 v69 v70 v71 v72 v73
v74 v75 v76 v77 v78 v79 v80 v81
Quote from: Helpmeh on March 18, 2012, 12:06:26 AM
Weird how I can't edit.

You get 1 hour from the time of posting, I believe.
Quote
FOR /F "tokens=1-14" %%- in (%String3%) do (

The above works, but for the sake of accuracy this should be

FOR /F "tokens=1-13" %%- in (%String3%) do (


Discussion

No Comment Found

Related InterviewSolutions