1

I'm trying to append, via bash, one text file to another using foo.txt >> bar.txt. The problem is, foo.txt is a windows text file (I think), because once appended in bar.txt 

every new^M
line^M
ends^M
like this.^M

Thus, they are surely two different file-types.

I've searched Google for answers but Google doesn't accept special characters like ">>" and "^" in search. The closest solution I can find is a reported solution on commandlinefu.com, but all it does is strip the r's from bar.txt which, really, is no solution at all.

So, I have two questions here:

  1. How do I discover, using bash, what file-type foo.txt really is?
  2. How do convert foo.txt properly to make it appendable to bar.txt?
Community
  • 1
  • 1
trench
  • 127
  • 1
  • 5

6 Answers6

5

Convert the Windows line endings with dos2unix:

dos2unix foo.txt
Michael Berkowski
  • 260,803
  • 45
  • 432
  • 377
5
sed 's/$'"/`echo \\\r`/" foo.txt >> bar.txt

or use dos2unix, if it's available.

Marc B
  • 348,685
  • 41
  • 398
  • 480
  • How do I determine it is a dos file before attempting to convert it? – trench Sep 12 '11 at 18:58
  • view it in vi? If there's ^M chars at the end of the line, it's in dos format. or if you're in Windows, load the file into notepad. If you have no line breaks at all, it's unix format. – Marc B Sep 12 '11 at 19:01
  • The `sed` command is safe to use even if there are no carriage returns (^M). – brightlancer Sep 12 '11 at 19:02
  • I'll come back and vote you up for offering the sed solution, Marc (when I can up-vote, I mean). I was definitely looking for a native solution first, but now that I know the answer to both questions I'll probably be using the dos2unix approach. Much appreciated. – trench Sep 12 '11 at 19:07
  • 2
    @trench you can use the `file` command to tell the format of the file before converting. However, using `dos2unix` on a file thats already in unix format is harmless. – tMC Sep 12 '11 at 19:33
  • Thanks, @tMC, but I was looking for a command that would perform a test on the file. `file` is definitely my solution there. dos2unix is working great though. @Marc B, could you edit your answer to include @Christopher Foy's `file foo.txt` answer as well? And maybe upvote him? Because that definitely answered my first question... you answered the second question. thanks. Again, much appreciated. – trench Sep 12 '11 at 20:03
  • Also, of note: When viewed in vim and vi, the ^M characters are not visible in the dos file. Another reason the `file foo.txt` answer needs some upvote love. – trench Sep 12 '11 at 20:10
  • @Marc: vi (at least `vim`) recognizes DOS-format text files and doesn't display the trailing `^M`s. But it does show `[dos]` on the status line. – Keith Thompson Sep 13 '11 at 00:56
4
  1. file foo.txt will output: "ASCII text, with CRLF line terminators" if the file has DOS-style line endings.
glenn jackman
  • 223,850
  • 36
  • 205
  • 328
Christopher Foy
  • 758
  • 5
  • 13
  • Thanks, that took care of the remaining answer I was looking for. – trench Sep 12 '11 at 19:02
  • If it's just plain text, yes. But for some recognized types of files, the `file` command doesn't show you how it's encoded. For example, `file tmp.html` prints `tmp.html: HTML document text`, regardless of the encoding (at least the Ubuntu version of the `file` command does this). – Keith Thompson Sep 13 '11 at 01:01
  • file --mime-encoding foo.html will output us-ascii, utf-8 &c. – Christopher Foy Sep 16 '11 at 19:48
1

Use fromdos and todos, or unix2dos and dos2unix.

Arnaud Le Blanc
  • 95,062
  • 22
  • 198
  • 192
0

Look up the commands dos2unix and unix2dos. You may wish to make a copy first (cp)

Ed Heal
  • 57,599
  • 16
  • 82
  • 120
0

A quick-and-dirty solution:

tr -d '\r' < foo.txt
Keith Thompson
  • 242,098
  • 41
  • 402
  • 602