3

Sub-string expansion works within the set of a for loop (that is the parenthesised part after in) when immediate expansion is used (write %%I instead of %I in a batch file):

set "X=123"
for %I in (%X:~1,1%) do @echo %I

However, it fails when delayed expansion is applied:

for %I in (!X:~1,1!) do @echo %I

I would expect the output to be:

2

But instead it is:

!X:~1
1!

Why is this and how can I prevent that?

I know I could work around that by quoting the set and using ~, but this is not what I want here:

for %I in ("!X:~1,1!") do @echo %~I

The following command line fails too:

for %I in (!X:*2=!) do @echo %I

The unexpected output is:

!

Also for command lines using the /R, /L and /R switches fail with such sub-string syntax.

It surely has got something to do with the fact that , and = are token separators for cmd, just like SPACE, TAB, ;, etc.

aschipfl
  • 33,626
  • 12
  • 54
  • 99
  • Are you testing that from a batch file or the cmd prompt? – Squashman May 16 '18 at 21:52
  • Both actually, and the output is exactly the same... – aschipfl May 16 '18 at 21:56
  • 3
    Yes. I believe it is because of delayed expansion. The parsing phase must see the comma as a command separator before delayed expansion happens. So the only work around is to quote surround it and then remove the quotes. – Squashman May 16 '18 at 22:02
  • 5
    If you escape the comma it works. `for %%I in (!X:~1^,1!) do @echo %%I` – Squashman May 16 '18 at 22:03
  • 1
    How did you enable delayed expansion on cmd line, via registry or `cmd /V:ON` ? Did you check this behaviour with [the phases from here](https://stackoverflow.com/questions/4094699/how-does-the-windows-command-interpreter-cmd-exe-parse-scripts) –  May 16 '18 at 22:06
  • Everything between the parentheses is evaluated in another cmd instance. @LotPings beat me to it on posting that link. – Squashman May 16 '18 at 22:08
  • @LotPings, I used `cmd /V`... – aschipfl May 16 '18 at 22:08
  • 1
    @Squashman, I don't believe it's another `cmd` instance; rather it is something specific to the parsing of `for`, I believe... – aschipfl May 16 '18 at 22:09
  • 4
    If you read that link that @LotPings posted you can see that phase 2 processes everything between the parentheses of the FOR command. Delayed Expansion does not happen until Phase 5. So you need to protect or escape the special characters within the parentheses of the IN block of the FOR command otherwise it sees the comma as a delimiter. – Squashman May 16 '18 at 22:17
  • 3
    Yes, @Squashman, you are absolutely right, it is described in LotPings' link -- thank you both! I am always wondering how logical and intuitive the good old Windows command prompt is... ;-) – aschipfl May 16 '18 at 22:26

1 Answers1

1

According to the thread How does the Windows Command Interpreter (CMD.EXE) parse scripts? and also numerous comments here, the answer lies in the special way for loops are parsed.

The key is the following excerpt of this answer (see the italic text in particular):

Phase 2) Process special characters, tokenize, and build a cached command block:
[...]

  • Three commands get special handling - IF, FOR, and REM
    [...]
    • FOR is split in two after the DO. A syntax error in the FOR construction will result in a fatal syntax error.
    • The portion through DO is the actual FOR iteration command that flows all the way through phase 7
      • All FOR options are fully parsed in phase 2.
      • The IN parenthesized clause treats <LF> as <space>. After the IN clause is parsed, all tokens are concatenated together to form a single token.
      • Consecutive token delimiters collapse into a single space throughout the FOR command through DO.

Due to the fact that delayed expansion happens after parsing of for and the described behaviour, token separators like SPACE, TAB, ,, ;, =, etc. become converted to a single SPACE, hence a sub-string expansion expression like !X:~1,1! is changed to !X:~1 1!, and a sub-string substitution expression like !X:*2=! is changed to !X:*2 !, which both are invalid syntax.

Therefore to solve the issue you need to escape the token separators by ^ like:

for %I in (!X:~1^,1!) do @echo %I

and:

for %I in (!X:*2^=!) do @echo %I

(By the way, there is a very similar problem with if statements.)

aschipfl
  • 33,626
  • 12
  • 54
  • 99