Yarn container status: exit_status = -1000

Very often, I saw the yarn-hadoop-nodemanager log file is full of the following log:
2013-04-05 19:14:09,345 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1365187402863, }, attemptId: 1, }, id: 57, }, state: C_RUNNING, diagnostics: "", exit_status: -1000, 2013-04-05 19:14:09,345 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1365187402863, }, attemptId: 1, }, id: 58, }, state: C_RUNNING, diagnostics: "", exit_status: -1000, 2013-04-05 19:14:09,345 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Sending out status for container: container_id {, app_attempt_id {, application_id {, id: 1, cluster_timestamp: 1365187402863, }, attemptId: 1, }, id: 60, }, state: C_RUNNING, diagnostics: "", exit_status: -1000,
Looked at the interface org.apache.hadoop.yarn.api.records.ContainerStatus and there was a comment about the getExitStatus() method:

   * Get the exit status for the container.
   *  
   * Note: This is valid only for completed containers i.e. 
   * containers with state {@link ContainerState#COMPLETE}. 
   * Otherwise, it returns an invalid exit code equal 
   * to {@literal -1000};

Seems the status is quit normal. Just wonder why Yarn does not change it to a more meaningful output or simply suppress this log.

cmake 2.6 breaks hadoop 2.0.2-alpha native lib build

Recently, my environment was automatically updated and the cmake was updated to 2.6. Then suddenly the hadoop-2.0.2-alpha built from my box threw errors for native libraries. For example, you would see the following error:

INFO org.apache.hadoop.util.NativeCodeLoader: 
   Failed to load native-hadoop with error: java.lang.UnsatisfiedLinkError: 
   no hadoop in java.library.path
INFO org.apache.hadoop.util.NativeCodeLoader: java.library.path=/home/hadoop/lib/native
WARN org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop library 
   for your platform... using builtin-java classes where applicable

The root cause was that the native lib path only has libhadoop.so.1.0.0, but no libhadoop.so, which was caused by cmake 2.6. The issue was reported at https://issues.apache.org/jira/browse/HADOOP-9215. Adding a link from libhadoop.so.1.0.0 to libhadoop.so should solve this problem.

No module named pygtk, Meld requires pygtk2.6.0 or higher.

I tried to install meld on a redhat 5 box. When I run meld, I got the following error: "No module named pygtk, Meld requires pygtk2.6.0 or higher.". But I indeed have pygtk2.10 installed. It turned out that I have two versions of python installed on the system. The pygtk installed using yum only integrated with the python managed by yum as well as a module. To solve this problem, I changed the first line of meld program from "#! /usr/bin/env python" to "#! /usr/bin/python", i.e., let meld point to the python that has the pygtk module installed. After that, meld works.

make hbase 0.94.3 work with hadoop 2.0

The tar ball hbase-0.94.3.tar.gz I downloaded from apache web site was built for hadoop 1.0 and it does not work with hadoop 2.0 by default. To make it work with hadoop 2.0, I need to rebuild it from source as follows.

$ tar zxvf hbase-0.94.3.tar.gz
$ cd hbase-0.94.3
$ mvn -Dhadoop.profile=2.0 -DskipTests install

The new built hbase-0.94.3.tar.gz is under the target/ directory. Be aware that you need to turn off the tests with the Maven option -DskipTests because some tests fail for hadoop 2.0.

use pem file for sftp

Sometimes, some Linux hosts require a pem file for ssh authentication. In such a case, I use the following command.

sftp -o IdentityFile=jfang.pem hadoop@box1.mycompany.com

undefined symbol: Perl_Gthr_key_ptr

Sometimes, I encountered the following error for perl:

symbol lookup error: /usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi/auto/YAML/Syck/Syck.so: undefined symbol: Perl_Gthr_key_ptr

Looked at the PERL5LIB environment variable, it is as follows.

PERL5LIB=/usr/lib64/perl5/site_perl/5.8.8/x86_64-linux-thread-multi:/usr/lib/perl5/site_perl/5.8.8:/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi:/usr/lib/perl5/vendor_perl/5.8.8

Seems multiple versions of PERL exist. One solution to this problem is to unset the PERL5LIB environment variable. For example, if you use zsh, you could add the following line to the .zshenv file.

unset PERL5LIB

Start a new shell and the problem should go away.

alien

alien is very useful tool to convert linux packages among rpm, tgz, deb, and so on.