16

I got some pig generated files with part-r-00000.deflate extension. I know this is a compressed file. How do I generate a normal file in a readable format. When I used hadoop fs -text, I cannot get plaintext output. The output is still binary. How can I fix this problem?

xgMz
  • 3,149
  • 2
  • 28
  • 21
Himateja Madala
  • 311
  • 1
  • 4
  • 15

2 Answers2

20

You might be using a quite old Hadoop version (e.g: 0.20.0) in which fs -text can't inflate the compressed file.

As a workaround you may try this one-liner (based on this answer):

hadoop fs -text file.deflate | perl -MCompress::Zlib -e 'undef $/; print uncompress(<>)'
Community
  • 1
  • 1
Lorand Bendig
  • 10,560
  • 1
  • 35
  • 45
16

you can decompress on the fly by using this command

hdfs dfs -text file.deflate | hdfs dfs -put - uncompressed_destination_file

guignol
  • 365
  • 3
  • 9