11

I have found that many of my files have DOS line endings. In VI they look like this: "^M". I don't want to modify files that don't have these DOS line endings. How do I do this using a bash script? Thanks!

EV

exvance
  • 1,299
  • 4
  • 12
  • 30

8 Answers8

19
grep -URl ^M . | xargs fromdos

grep gets you a list of all files under the current directory that have DOS line endings.

-U makes grep consider line endings instead of stripping them away by default

-R makes it recursive

-l makes it list only the filenames and not the matching lines

then you're piping that list into the converter command (which is fromdos on ubuntu, dos2unix where i come from).

NOTE: don't actually type ^M. instead, you'll need to press <Ctrl-V> then <Ctrl-M> to insert the ^M character and make grep understand what you're going for. or, you could type in $'\r' in place of ^M (but i think that may only work for bash...).

Nicolas Raoul
  • 57,417
  • 55
  • 212
  • 360
nullrevolution
  • 3,737
  • 1
  • 16
  • 19
5

One way using GNU coreutils:

< file.txt tr -d '\r'
Steve
  • 45,652
  • 12
  • 89
  • 100
4

On ubuntu, you use the fromdos utility

fromdos test.txt

The above example would take a MS-DOS or Microsoft Windows file or other file with different line separators and format the file with new line separators to be read in Linux and Unix.

Fredrik Pihl
  • 42,950
  • 7
  • 81
  • 128
1

Many options are there..you can try with any of these.. http://www.theunixschool.com/2011/03/different-ways-to-delete-m-character-in.html

Guru
  • 15,464
  • 2
  • 31
  • 43
1
cat origin_file.txt | sed "s/^M//" > dest_file.txt

You have to do the same thing mentioned above, ctl-V then ctl-M to get that character. This is preferable for me because it is portable across many platforms and keeps it simple within bash.

on ubuntu I also find this works:

cat origin_file.txt | sed "s/\r//" > dest_file.txt

Jim McNeely
  • 738
  • 5
  • 7
0

you can use the command:

   dos2ux file.in>file.out or:

in perl:

perl -pi -e 's/\r//g' your_file

alternatively you can do:

  • open in vi
  • go to command mode
  • type :%s/[ctrl-V][CTRL-M]//g
Vijay
  • 62,703
  • 87
  • 215
  • 314
0

Note if you're converting multi-byte files you need to take extra care, and should probably try to use the correct iconv or recode from-encoding specifications.

If it's a plain ASCII file, both of the below methods would work.

The flip program, in Debian the package is also called flip, can handle line-endings. From the manual:

When asked to convert a file to the same format that  it already 
has, flip  causes  no change to the file. Thus to convert all
files to **IX format you can type

flip -u *

and all files will end up right, regardless of whether they were 
in MS-DOS or in **IX format to begin with. This also works in the
opposite direction.

Or you could use GNU recode:

< /etc/passwd recode ..pc | tee a b > /dev/null
file a b

Output:

a: ASCII text, with CRLF line terminators
b: ASCII text, with CRLF line terminators

Convert to unix line-endings:

recode pc.. a b
file a b

Output:

a: ASCII text
b: ASCII text

recode abbreviates dos line-endings as pc, so the logic with pc.. is: convert from pc format to the default, which is latin1 with unix line-endings.

Thor
  • 42,211
  • 10
  • 116
  • 125
0

A modification to the Winning answer if you need to filter by file ending

grep -URl ^M . | grep .php | xargs dos2unix

I used dos2unix instead of fromdos but the effect should be the same.

Jarrod Juleff
  • 75
  • 8