由于Hadoop-2.2.0是在32位的基础上编译的,如下图所示,我们看到从官网下载的hadoop-2.2.0默认是32位的,这在我们64位的虚拟机上运行会有问题。因此我们需要对Hadoop源码进行编译,将它变成64位的。
[root@itcast05 ~]# ls
anaconda-ks.cfgapache-maven-3.0.5-bin.tar.gz Documents hadoop-2.2.0 install.log jdk-7u80-linux-x64.gz PicturesPublic Templates apache-ant-1.9.2-bin.tar.gz DesktopDownloads hadoop-2.2.0.tar.gz install.log.syslog Musicprotobuf-2.5.0.tar.gz target Videos
[root@itcast05 ~]# cd hadoop-2.2.0
[root@itcast05 hadoop-2.2.0]# ls
bin etc include lib libexec LICENSE.txt NOTICE.txt README.txt sbin share
[root@itcast05 hadoop-2.2.0]# cd lib
[root@itcast05 lib]# ls
native
[root@itcast05 lib]# cd native/
[root@itcast05 native]# ls
libhadoop.a libhadooppipes.a libhadoop.so libhadoop.so.1.0.0 libhadooputils.a libhdfs.a libhdfs.so libhdfs.so.0.0.0
[root@itcast05 native]# file libhadoop.so.1.0.0
libhadoop.so.1.0.0: ELF 32-bit LSB shared object,Intel 80386,version 1 (SYSV),dynamically linked,not stripped
[root@itcast05 native]#
接下来我们便一步一步图文解说如何编译。
第一步:确保连网
要确保我们的虚拟机能够连网,关于如何给虚拟机配置固定IP并且怎样连网大家如果不知道怎么操作的话,可以参考:http://blog.csdn.net/u012453843/article/details/52839105这篇博客进行学习。
第二步:安装gcc
依次执行下面四条命令,在CentOS6.8当中,gcc和make已经安装过了,不过为了保证后续不出错,最好都执行一遍,如果有的已经安装过了,无非会提醒你已经安装过了,这没什么关系。
[root@itcast05 ~]# yum install gcc
[root@itcast05 ~]#yum -y install gcc-c++
[root@itcast05 ~]#yum install make
[root@itcast05 ~]# yum install autoconf automake libtool cmake ncurses-devel openssl-devel gcc*
第三步:安装JDK
3.1 从官网下载JDK,我下载的是jdk-7u80-linux-x64.gz这个版本,大家也可以到我的百度云盘:http://pan.baidu.com/s/1sl3rpPb这个网址进行下载。
3.2 接着我们使用FileZilla工具把jdk-7u80-linux-x64.gz安装包上传到我们的服务器(也就是我们的虚拟机),这里我上传到root根目录下了,如下图所示。关于FileZilla的使用如果大家不知道怎么使用的话,可以参考:http://blog.csdn.net/u012453843/article/details/52422736这篇博客进行学习。
3.3 我们在/usr/目录下创建一个目录,用来安装我们的JDK。
[root@itcast05 ~]# mkdir /usr/java
3.4 解压jdk-7u80-linux-x64.gz到/usr/java目录下
[root@itcast05 ~]# tar -zxvf jdk-7u80-linux-x64.gz -C/usr/java/
3.5 配置环境变量,把export JAVA_HOME=/usr/java/jdk1.7.0_80和export PATH=$JAVA_HOME/bin:$PATH放到最后,如下所示,配置完之后先按ESC键退出输入模式然后按住Shift键不放再连续两次按Z键即可保存退出该文件。
*)
if [ "$2" = "after" ] ; then
PATH=$PATH:$1
else
PATH=$1:$PATH
fi
esac
}
if [ -x /usr/bin/id ]; then
if [ -z "$EUID" ]; then
# ksh workaround
EUID=`id -u`
UID=`id -ru`
fi
USER="`id -un`"
LOGNAME=$USER
MAIL="/var/spool/mail/$USER"
fi
if [ "$EUID" = "0" ]; then
pathmunge /sbin
pathmunge /usr/sbin
pathmunge /usr/local/sbin
else
pathmunge /usr/local/sbin after
pathmunge /usr/sbin after
pathmunge /sbin after
fi
HISTSIZE=1000
if [ "$HISTCONTROL" = "ignorespace" ] ; then
export HISTCONTROL=ignoreboth
else
export HISTCONTROL=ignoredups
fi
# Current threshold for system reserved uid/gids is 200
# You could check uidgid reservation validity in
# /usr/share/doc/setup-*/uidgid file
if [ $UID -gt 199 ] && [ "`id -gn`" = "`id -un`" ]; then
umask 002
else
umask 022
fi
if [ -r "$i" ]; then
if [ "${-#*i}" != "$-" ]; then
. "$i"
else
. "$i" >/dev/null 2>&1
fi
fi
done
unset -f pathmunge
export JAVA_HOME=/usr/java/jdk1.7.0_80
export PATH=$JAVA_HOME/bin:$PATH
Usage: java [-options] class [args...]
(to execute a class)
or java [-options] -jar jarfile [args...]
(to execute a jar file)
where options include:
-d32 use a 32-bit data model if available
-d64 use a 64-bit data model if available
-server to select the "server" VM
The default VM is server.
-classpath <class search path of directories and zip/jar files>
A : separated list of directories,JAR archives,
and ZIP archives to search for class files.
-D<name>=<value>
set a system property
-verbose:[class|gc|jni]
enable verbose output
-version print product version and exit
-version:<value>
require the specified version to run
-showversion print product version and continue
-jre-restrict-search | -no-jre-restrict-search
include/exclude user private JREs in the version search
-? -help print this help message
-X print help on non-standard options
-ea[:<packagename>...|:<classname>]
-enableassertions[:<packagename>...|:<classname>]
enable assertions with specified granularity
-da[:<packagename>...|:<classname>]
-disableassertions[:<packagename>...|:<classname>]
disable assertions with specified granularity
-esa | -enablesystemassertions
enable system assertions
-dsa | -disablesystemassertions
disable system assertions
-agentlib:<libname>[=<options>]
load native agent library <libname>,e.g. -agentlib:hprof
see also,-agentlib:jdwp=help and -agentlib:hprof=help
-agentpath:<pathname>[=<options>]
load native agent library by full pathname
-javaagent:<jarpath>[=<options>]
load Java programming language agent,see java.lang.instrument
-splash:<imagepath>
show splash screen with specified image
See http://www.oracle.com/technetwork/java/javase/documentation/index.html for more details.
[root@itcast05 ~]# java -version
java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11,mixed mode)
[root@itcast05 ~]#
export MAVEN_HOME=/usr/local/maven/apache-maven-3.0.5
export PATH=$JAVA_HOME/bin:$MAVEN_HOME/bin:$PATH
Apache Maven 3.0.5 (r01de14724cdef164cd33c7c8c2fe155faf9602da; 2013-02-19 05:51:28-0800)
Maven home: /usr/local/maven/apache-maven-3.0.5
Java version: 1.7.0_80,vendor: Oracle Corporation
Java home: /usr/java/jdk1.7.0_80/jre
Default locale: en_US,platform encoding: UTF-8
OS name: "linux",version: "2.6.32-642.el6.x86_64",arch: "amd64",family: "unix"
[root@itcast05 ~]#
export MAVEN_HOME=/usr/local/maven/apache-maven-3.0.5
export ANT_HOME=/usr/local/ant/apache-ant-1.9.2
export PATH=$JAVA_HOME/bin:$MAVEN_HOME/bin :$ANT_HOME/bin:$PATH
Apache Ant(TM) version 1.9.2 compiled on July 8 2013
[root@itcast05 ~]#
[root@itcast05 protobuf]# ls
protobuf-2.5.0
[root@itcast05 protobuf]# cd protobuf-2.5.0/
[root@itcast05 protobuf-2.5.0]# ls
aclocal.m4 config.guess config.log configure COPYING.txt examples install-sh libtool Makefile missing protobuf.pc README.txt vsprojects autogen.sh config.h config.status configure.ac depcomp generate_descriptor_proto.sh INSTALL.txt ltmain.sh Makefile.am protobuf-lite.pc protobuf.pc.in src CHANGES.txt config.h.in config.sub CONTRIBUTORS.txt editors gtest java m4 Makefile.in protobuf-lite.pc.in python stamp-h1
[root@itcast05 protobuf-2.5.0]# ./configure
libprotoc 2.5.0
[root@itcast05 protobuf-2.5.0]#
[root@itcast05 hadoop-2.2.0-src]# ls
BUILDING.txt hadoop-assemblies hadoop-common-project hadoop-hdfs-project hadoop-maven-plugins hadoop-project hadoop-tools LICENSE.txt pom.xml
dev-support hadoop-client hadoop-dist hadoop-mapreduce-project hadoop-minicluster hadoop-project-dist hadoop-yarn-project NOTICE.txt README.txt
[root@itcast05 hadoop-2.2.0-src]# cd hadoop-common-project/
[root@itcast05 hadoop-common-project]# ls
dev-support hadoop-annotations hadoop-auth hadoop-auth-examples hadoop-common hadoop-nfs pom.xml target
[root@itcast05 hadoop-common-project]# cd hadoop-auth
[root@itcast05 hadoop-auth]# ls
BUILDING.txt pom.xml README.txt src target
[root@itcast05 hadoop-auth]# vim pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed under the Apache License,Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,either express or implied.
See the License for the specific language governing permissions and
limitations under the License. See accompanying LICENSE file.
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0
http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-project</artifactId>
<version>2.2.0</version>
<relativePath>../../hadoop-project</relativePath>
</parent>
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-auth</artifactId>
<version>2.2.0</version>
<packaging>jar</packaging>
<description>Apache Hadoop Auth - Java HTTP SPNEGO</description>
<maven.build.timestamp.format>yyyyMMdd</maven.build.timestamp.format>
<kerberos.realm>LOCALHOST</kerberos.realm>
</properties>
<dependency>
<!-- Used,even though 'mvn dependency:analyze' doesn't find it -->
<groupId>org.apache.hadoop</groupId>
<artifactId>hadoop-annotations</artifactId>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-all</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
"pom.xml" 219L,7174C
[root@itcast05 apache-maven-3.0.5]# ls
bin boot conf lib LICENSE.txt NOTICE.txt README.txt
[root@itcast05 apache-maven-3.0.5]# cd conf/
[root@itcast05 conf]# ls
setting.xml
<settingsxmlnssettingsxmlns="http://maven.apache.org/SETTINGS/1.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0http://maven.apache.org/xsd/settings-1.0.0.xsd">
<pluginGroups></pluginGroups>
<proxies></proxies>
<servers></servers>
<mirrors>
<mirror>
<id>nexus-osc</id>
<mirrorOf>*</mirrorOf>
<name>Nexusosc</name>
<url>http://maven.oschina.net/content/groups/public/</url>
</mirror>
</mirrors>
<profiles>
<profile>
<id>jdk-1.7</id>
<activation>
<jdk>1.7</jdk>
</activation>
<repositories>
<repository>
<id>nexus</id>
<name>localprivatenexus</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</repository>
</repositories>
<pluginRepositories>
<pluginRepository>
<id>nexus</id>
<name>localprivatenexus</name>
<url>http://maven.oschina.net/content/groups/public/</url>
<releases>
<enabled>true</enabled>
</releases>
<snapshots>
<enabled>false</enabled>
</snapshots>
</pluginRepository>
</pluginRepositories>
</profile>
</profiles>
</settings>
[root@itcast05 hadoop-2.2.0-src]# ls
BUILDING.txt hadoop-assemblies hadoop-common-project hadoop-hdfs-project hadoop-maven-plugins hadoop-project hadoop-tools LICENSE.txt pom.xml
dev-support hadoop-client hadoop-dist hadoop-mapreduce-project hadoop-minicluster hadoop-project-dist hadoop-yarn-project NOTICE.txt README.txt
[root@itcast05 hadoop-2.2.0-src]#
[root@itcast05 hadoop-dist]# ls
pom.xml target
[root@itcast05 hadoop-dist]# cd target/
[root@itcast05 target]# ls
antrun dist-layout-stitching.sh dist-tar-stitching.sh hadoop-2.2.0 hadoop-2.2.0.tar.gz hadoop-dist-2.2.0.jar hadoop-dist-2.2.0-javadoc.jar javadoc-bundle-options maven-archiver test-dir
[root@itcast05 target]#
[root@itcast05 hadoop-2.2.0]# ls
bin etc include lib libexec sbin share
[root@itcast05 hadoop-2.2.0]# cd lib
[root@itcast05 lib]# ls
native
[root@itcast05 lib]# cd native/
[root@itcast05 native]# ls
libhadoop.a libhadooppipes.a libhadoop.so libhadoop.so.1.0.0 libhadooputils.a libhdfs.a libhdfs.so libhdfs.so.0.0.0
[root@itcast05 native]# file libhadoop.so.1.0.0
libhadoop.so.1.0.0: ELF 64-bit LSB shared object,x86-64,not stripped
[root@itcast05 native]#