56

I'm trying to store the files listing into an array and then loop through the array again. Below is what I get when I run ls -ls command from the console.

total 40
36 -rwxrwxr-x 1 amit amit 36720 2012-03-31 12:19 1.txt
4 -rwxrwxr-x 1 amit amit  1318 2012-03-31 14:49 2.txt

The following bash script I've written to store the above data into a bash array. 

i=0
ls -ls | while read line
do
    array[ $i ]="$line"        
    (( i++ ))
done

But when I echo $array, I get nothing!

FYI, I run the script this way: ./bashscript.sh

Alex Raj Kaliamoorthy
  • 1,931
  • 2
  • 25
  • 40
codef0rmer
  • 10,079
  • 9
  • 48
  • 76
  • bash run pipeline in sub shell, so your assignment to array is only available in do .. done. – yuanjianpeng Apr 27 '19 at 14:43
  • I would suggest that the question here is really "How to iterate over a directory list"? **Arrays are NOT universally supported in shell scripts**. – BuvinJ Oct 16 '19 at 12:44
  • And even if you have a shell with arrays, you don't want or need to keep the file names in memory just to loop over them one by one. An array is useful if you want to compare every file to every other file, for example, but to just loop over files, use a regular `for file in *` or whatever, and don't squander memory on keeping a copy of the information the shell is perfectly capable of producing at any time. – tripleee Feb 06 '21 at 12:17

8 Answers8

126

I'd use

files=(*)

And then if you need data about the file, such as size, use the stat command on each file.

glenn jackman
  • 223,850
  • 36
  • 205
  • 328
  • 3
    Fantastic! Even works with directory names prepended, to get files from more than one directory (ie `files_in_dirs=(dir/* other_dir/*)`. Very useful, thanks. – Gus Shortz Sep 06 '13 at 21:15
  • 11
    And then to list all elements in this files array: `echo ${files[@]}` or use that in for loop – HankCa May 17 '15 at 22:54
  • 1
    On macs this seems to not work when files contain spaces (which is unfortunately frequent). There may be a work around. If anyone can find what this syntax is doing in the bash documentation, that may help us figure out a solution to this issue. – David Jun 30 '15 at 23:42
  • 2
    empty dirs will cause unpredictable errors without "shopt -s nullglob". – Asain Kujovic Apr 22 '16 at 19:31
  • 6
    @David, there is not problem with files with spaces: they will be stored in the array properly as a single element. It is extracting the file from the array as a single unit that requires care: `for filename in "${files[@]}"` -- where the quotes are crucial. See [this answer](http://stackoverflow.com/a/12316565/7552) for examples. – glenn jackman May 16 '17 at 17:47
  • @glennjackman hey `for filename in "${files[@]}"`, it treat all elements into one unit. at least in win7... – user2959760 Oct 13 '17 at 16:48
  • 1
    Where do you have bash on win7? – glenn jackman Oct 13 '17 at 17:18
  • Can this be used to get (only files) or (everything except symlinks). – RatDon Jun 11 '18 at 18:29
  • On Macs and elsewhere, this works fine if you then also quote your variable inside the loop. See [When to wrap quotes around a shell variable](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable) – tripleee Feb 06 '21 at 12:21
  • @RatDon Look into `find` options; it can do all that and lots more. – tripleee Feb 06 '21 at 12:21
39

Try with:

#! /bin/bash

i=0
while read line
do
    array[ $i ]="$line"        
    (( i++ ))
done < <(ls -ls)

echo ${array[1]}

In your version, the while runs in a subshell, the environment variables you modify in the loop are not visible outside it.

(Do keep in mind that parsing the output of ls is generally not a good idea at all.)

Mat
  • 195,986
  • 40
  • 382
  • 396
7

Here's a variant that lets you use a regex pattern for initial filtering, change the regex to be get the filtering you desire.

files=($(find -E . -type f -regex "^.*$"))
for item in ${files[*]}
do
  printf "   %s\n" $item
done
harschware
  • 12,277
  • 17
  • 52
  • 85
  • 1
    Word-splitting applies to the expansion of the command substitution, so each space creates a separate array element. If there is a file called `foo bar.txt`, `item` will be set to `foo`, then `bar.txt`. – chepner Apr 29 '13 at 23:37
  • ah yes, after reading your comment I only focused on the regex.. nice catch. – harschware May 01 '13 at 15:50
  • So how would you catch spaces? – Jonny May 24 '13 at 02:23
  • This is horribly broken. [When to wrap quotes around a shell variable](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable) – tripleee Feb 06 '21 at 12:20
5

This might work for you:

OIFS=$IFS; IFS=$'\n'; array=($(ls -ls)); IFS=$OIFS; echo "${array[1]}"
potong
  • 51,370
  • 6
  • 49
  • 80
  • 2
    Or simpler: `IFS=$'\n' array=($(ls -ls))` – Guss Aug 17 '15 at 19:00
  • 1
    @Guss Simpler, yes. Correct, no. The use of `OIFS` here involves restoring the internal field separator after you've change it temporarily. Failing to do that is likely to cause you a lot of subsequent problems! – BuvinJ Oct 16 '19 at 12:48
  • Ok, @BuvinJ, granted. Then how about `(IFS=$'\n' array=($(ls -ls)))` ? the `$IFS` change is now limited to a subshell and won't propagate further. Still simpler ;-) – Guss Oct 16 '19 at 13:42
  • Great idea. I'm not sure about the syntax though. I believe you mean `$(IFS='\n'; array=($(ls -ls)))` would make it run in a subshell, but then your `array` variable is also lost upon return to the outer script! Each way I think about solving this, ends up being at least as long / complicated as the original approach from @potong. And his way is, in the fact, the "canonical" approach to messing with IFS. – BuvinJ Oct 16 '19 at 17:49
3

Running any shell command inside $(...) will help to store the output in a variable. So using that we can convert the files to array with IFS.

IFS=' ' read -r -a array <<< $(ls /path/to/dir)
rashok
  • 11,758
  • 13
  • 85
  • 95
1

You may be tempted to use (*) but what if a directory contains the * character? It's very difficult to handle special characters in filenames correctly.

You can use ls -ls. However, it fails to handle newline characters.

# Store la -ls as an array
readarray -t files <<< $(ls -ls)
for (( i=1; i<${#files[@]}; i++ ))
{
    # Convert current line to an array
    line=(${files[$i]})
    # Get the filename, joining it together any spaces
    fileName=${line[@]:9}
    echo $fileName
}

If all you want is the file name, then just use ls:

for fileName in $(ls); do
    echo $fileName
done

See this article or this this post for more information about some of the difficulties of dealing with special characters in file names.

Dan Bray
  • 6,447
  • 3
  • 47
  • 62
  • No, it's not hard at all. Just [don't use `ls`](https://mywiki.wooledge.org/ParsingLs) and [quote your variables](https://stackoverflow.com/questions/10067266/when-to-wrap-quotes-around-a-shell-variable) you'll be fine. – tripleee Feb 06 '21 at 12:18
  • @tripleee if I quote properly using `*`, then it displays newline characters in the filename. Therefore, it offers no advantages over `ls`, but if used incorrectly `*` can be dangerous. Using `ls` might not be perfect, but what would you suggest as a better way? – Dan Bray Feb 06 '21 at 13:53
  • If a name contains a newline character, storing and displaying that newline character is exactly the right thing to do, isn't it? The first link in my comment contains extensive documentation for why you should basically never use `ls` in scripts, and what to use instead. – tripleee Feb 06 '21 at 16:35
  • @tripleee I already read that article before I posted and I disagree with it. I'm sure there are times when `ls` is definitely the wrong tool for the job, but to assume that it never is absurd, especially, when the article does not present good alternatives. It says not to use `ls` and to use `(*)` instead, but provides no good reason to. Using `(*)' can be dangerous if not quoted and still has the same problems `ls` does. – Dan Bray Feb 06 '21 at 20:57
  • I'm trying to understand your argument, but you are not helping. Can you provide an example where `ls` works and `(*)` does not, or ends up being much more complex? – tripleee Feb 07 '21 at 06:29
0

Isn't these 2 code lines, either using scandir or including the dir pull in the declaration line, supposed to work?

src_dir="/3T/data/MySQL";
# src_ray=scandir($src_dir);
declare -a src_ray ${src_dir/*.sql}
printf ( $src_ray );
OldManRiver
  • 132
  • 1
  • 6
0

simply you can use this below for loop

declare -a arr
for file in *.txt
do
    arr=(${arrPi[*]} "$file")
done
Rifwan Jaleel
  • 1
  • 1