7

I've installed Hadoop 2.7.2 single node on Ubuntu and I want to run the java wordcount program. The compilation and the creation of the jar file are done succesfully, but when I run the jar file on Hadoop I receive this message:

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

I set environment variables by editing .bashrc file:

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib

When I type hadoop checknative -a I get this:

hadoop: true /usr/local/hadoop/lib/native/libhadoop.so.1.0.0
zlib: true /lib/x86_64-linux-gnu/libz.so.1
snappy: false 
lz4: true revision:99
bzip2: false
openssl: true /usr/lib/x86_64-linux-gnu/libcrypto.so

16/05/09 00:48:53 INFO util.ExitUtil: Exiting with status 1

Hadoop version: 2.7.2

Ubuntu version: 14.04

Could anyone give some clues about the issue?

El Mehdi Belgasmi
  • 71
  • 1
  • 1
  • 4
  • 3
    Possible duplicate of [Hadoop "Unable to load native-hadoop library for your platform" warning](http://stackoverflow.com/questions/19943766/hadoop-unable-to-load-native-hadoop-library-for-your-platform-warning) – 32cupo Jan 26 '17 at 11:22

4 Answers4

4

Move your compiled native library files to $HADOOP_HOME/lib folder.

Then set your environment variables by editing .bashrc file

export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_HOME/lib  
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib"

Make sure your compiled native library files are in $HADOOP_HOME/lib folder.

It should fix the issue.

Nishu Tayal
  • 19,244
  • 8
  • 46
  • 96
1

Add the commands lines below to hadoop-env.sh, it should suppress the errors encountered

export HADOOP_HOME_WARN_SUPPRESS=1
export HADOOP_ROOT_LOGGER="WARN,DRFA"
Praveen
  • 1,461
  • 16
  • 25
Mike Onuorah
  • 11
  • 1
  • 1
    I configured hadoop on raspberry where none of the library files are native and while it all works it's nice to suppress the warnings. Many Thanks –  Mar 16 '19 at 21:10
0

Try to load the hadoop jars-"hadoop-common.jar and hadoop-core.jar" to your class path. You can simply do it in eclipse ans while creating a jar file those jars will be referenced automatically.

Ishan Kumar
  • 1,770
  • 3
  • 18
  • 26
-3

edit the file hadoop-env.sh in /usr/local/etc/hadoop

Adding Hadoop library into LD_LIBRARY_PATH :

export LD_LIBRARY_PATH=/usr/local/hadoop/lib/native/:$LD_LIBRARY_PATH

try it, works for me

MatejMecka
  • 1,423
  • 2
  • 27
  • 35
strom
  • 7
  • 4