34

next code doesnt work because of spaces in file names, How to fix?

IFS = '\n'
for name in `ls `
do
    number=`echo "$name" | grep -o "[0-9]\{1,2\}"`
    if [[ ! -z "$number" ]]; then
        mv "$name" "./$number"
    fi
done
jww
  • 97,681
  • 90
  • 411
  • 885
Yola
  • 18,496
  • 11
  • 65
  • 106
  • 2
    See http://mywiki.wooledge.org/ParsingLs#line-75 – user123444555621 Dec 27 '11 at 14:52
  • GNU coreutils added `ls --zero` (i.e., end each output line with NUL and not newline "\n")(as of 2022-06-11). This avoid the issues with filenames containing spaces. See [ls source code](https://github.com/coreutils/coreutils/blob/master/src/ls.c) – user2514157 Jun 12 '22 at 01:37

4 Answers4

41

Just don't use command substitution: use for name in *.

fge
  • 119,121
  • 33
  • 254
  • 329
  • 1
    Note that this may fail in an empty directory. Thee loop will be executed once with `name="*"`. Not a problem in this particular case though. – user123444555621 Dec 27 '11 at 15:03
  • 1
    If it does turn out to be a problem, `shopt -s nullglob` will cause `*` to expand to zero words if there are no matches. – ephemient Dec 27 '11 at 19:31
  • if I wanted to get the files in a specific order, such as with `| sort` or `| shuf`, the "just use globbing" advice seems pretty useless. – Wyatt Ward Dec 20 '18 at 09:33
23

Replace

for name in `ls`

with:

ls | while read name

Notice: bash variable scoping is awful. If you change a variable inside the loop, it won't take effect outside the loop (in my version it won't, in your version it will). In this example, it doesn't matter.

Notice 2: This works for file names with spaces, but fails for some other strange but valid file names. See Charles Duffy's comment below.

ugoren
  • 16,023
  • 3
  • 35
  • 65
  • "If you change a variable inside the loop, it won't take effect outside the loopIf you change a variable inside the loop, it won't take effect outside the loop" -- if you use your construct, yes, because whatever is after the pipe is launched in a subshell. It is therefore expected that it has a different variable scope. – fge Dec 27 '11 at 14:28
  • I think bash's behavior here is erratic. If you understand it, and it seems like you (i.e. fge) do, then you know what to expect. But I learned it the hard way, it it still looks strange to me. – ugoren Dec 27 '11 at 14:36
  • 1
    `ls | while read name` is wrong. It doesn't correctly handle filenames with literal backslashes, or filenames with literal newlines, or filenames with leading or trailing whitespace. – Charles Duffy Nov 22 '16 at 19:27
  • @ugoren, ...it's not even really a question of deliberate scoping rules, but a question of what pipelines are and what they mean; the rules just fall out. `foo | bar` means "run a subprocess foo, and a subprocess bar, with the output of the first subprocess connected to the input of the second subprocess". Since things are happening in subprocesses, of *course* they can't impact the parent process's variables or state. (The above is actually not *quite* true -- the POSIX sh standard doesn't specify that all pipeline components *must* be subprocesses -- but it's a good first approximation). – Charles Duffy Dec 06 '16 at 16:03
  • (...to clarify the distance between approximation and truth: To have a reasonable implementation, it's effectively mandatory that either all components, or all components but one, run in a subprocess; bash takes the "all components" approach unless the `lastpipe` option introduced in 4.2 is enabled and certain other prerequisites are met; ksh runs the last component of a pipeline in the parent shell, and puts all the others in subprocesses; other implementations of the standard may differ further). – Charles Duffy Dec 06 '16 at 16:07
  • @CharlesDuffy, Thanks for the clarification. If you try to think of Bash as a programming language, with variables, functions and return values, you get awful scoping laws. If you think of it as a way to invoke programs, your explanation makes perfect sense. – ugoren Dec 07 '16 at 08:05
  • As Charles mentioned, a `ls | while read` won't cope if the script encounters filenames containing any manner of special characters. If a more general solution is required, using find -print0 might be a better option, as described in Gordon's answers to [capturing-output-of-find-print0-into-a-bash-array](https://stackoverflow.com/questions/1116992/capturing-output-of-find-print0-into-a-bash-array). Using a while read does cope nicely with the possibility that the number of arguments might exceed the maximum line length, it's my preference when handling unbounded numbers of arguments. – Michael Hamilton Mar 28 '21 at 21:48
5

Looks like two potential issues:

First, the IFS variable and it's assignment should not have space in them. Instead of

IFS = '\n' it should be IFS=$'\n'

Secondly, for name in ls will cause issues with filename having spaces and newlines. If you just wish to handle filename with spaces then do something like this

for name in *

I don't understand the significance of the line 

number=`echo "$name" | grep -o "[0-9]\{1,2\}"`

This will give you numbers found in filename with spaces in new lines. May be that's what you want.

jaypal singh
  • 74,723
  • 23
  • 102
  • 147
  • 2
    Actually, it should be `IFS=$'\n'` -- without the `$`, `\n` doesn't get interpreted as a newline, but as a literal backslash and the letter "n". – Gordon Davisson Dec 27 '11 at 15:48
0

For me, I had to move to use find.

find /foo/path/ -maxdepth 1 -type f -name "*.txt" | while read name
do
    #do your stuff with $name
done
Lucas
  • 1,514
  • 3
  • 16
  • 23