源码编译-Build Source

1.环境准备

想要更进一步了解和使用Trafodion,难免不得不自己尝试一番编译源码,此为之前做的一个小任务,踩过一些坑,现分享出来并做个总结。[参考](https://cwiki.apache.org/confluence/display/TRAFOdioN/Build+Source)

1.1 虚拟机

源码编译准备搭建自己的一个虚拟机,系统CentOS6.X即可,7版本会编译错误不支持。
配置参考:
硬盘:30G 最好不要默认,否则多次编译会使磁盘空间不足
内存:2G
网卡:NAT

Tips:避免出错需要重新新建虚拟机,最好每做一步做好快照,方便恢复。
查看系统版本:
cat /etc/redhat-release
CentOS release 6.7 (Final)

1.2 安装编译需要的包

# Install first to avoid dependency problems for the steps below
sudo yum install epel-release

sudo yum install alsa-lib-devel ant ant-nodeps apr-devel apr-util-devel \          boost-devel cmake device-mapper-multipath dhcp doxygen flex \          gcc-c++ gd git glibc-devel glibc-devel.i686 graphviz-perl gzip \          java-1.7.0-openjdk-devel libX11-devel libXau-devel libaio-devel \          libcurl-devel libibcm.i686 libibumad-devel libibumad-devel.i686 \          libiodbc libiodbc-devel librdmacm-devel librdmacm-devel.i686 \          libxml2-devel lua-devel lzo-minilzo \          net-snmp-devel net-snmp-perl openldap-clients openldap-devel \          openldap-devel.i686 openmotif openssl-devel openssl-devel.i686 \          perl-Config-IniFiles perl-Config-Tiny \          perl-DBD-sqlite perl-Expect perl-IO-Tty perl-Math-Calc-Units \          perl-Params-Validate perl-Parse-RecDescent perl-TermReadKey \          perl-Time-HiRes protobuf-compiler protobuf-devel \          readline-devel rpm-build saslwrapper sqlite-devel \          unixODBC unixODBC-devel uuid-perl wget xerces-c-devel xinetd
Tips: 避免因为包安装失败造成编译错误,请确保每一个包均安装成功

1.3 配置

配置java的环境变量,通过java -version 验证成功
#新建编译目录trafodion
mkdir trafodion
cd trafodion
#下载源码
git clone git://git.apache.org/incubator-trafodion.git
# 新建trafodion 编译目录
mkdir ~/trafodion-download ~/trafodion-build-tools

cd ~/mysource/incubator-trafodion/install
# 利用脚本安装
./traf_tools_setup.sh -d ~/trafodion-download -i ~/trafodion-build-tools
# 安装成功信息基本如下
INFO: Starting tools build on Mon Jan 25 12:30:41 PST 2016
Tools install directory /home/trafdeveloper/trafodion-build-tools does not exist,do you want to to create it? y/n : y
INFO: Created directory /home/trafdeveloper/trafodion-build-tools
INFO: Tar download location: /home/trafdeveloper/download
INFO: Tool install directory location: /home/trafdeveloper/trafodion-build-tools
INFO: LogFile location: /home/trafdeveloper/mysource/incubator-trafodion/install/traf_tools_setup.log
 ***********************************************************

INFO: Installing MPI on Mon Jan 25 12:30:45 PST 2016
INFO:   downloaded tar file: mpich-3.0.4.tar.gz
.
.
.
INFO:   downloaded tar file:  apache-maven-3.3.3-bin.tar.gz
INFO: Maven installation complete
***********************************************************
INFO: Completed tools build on Mon Jan 25 13:18:31 PST 2016
INFO: List of tools directory:

apache-log4cxx-0.10.0
apache-maven-3.3.3
bison_3_linux
dest-llvm-3.2
dest-mpich-3.0.4
icu4c_4.4
thrift-0.9.0
udis86-1.7.2
zookeeper-3.4.5
Tips : 此过程需要从网络上下载包并解压安装,由于网络限制,速度较慢,一般在1-1.5h
安装完成导入环境变量,后续需要使用,否则因为找不到变量而发生错误
export TOOLSDIR=$HOME/trafodion-build-tools
echo -en "export TOOLSDIR=$HOME/trafodion-build-tools\n" >> $HOME/.bashrc

本步骤结束,请验证
$JAVA_HOME
$TOOLSDIR
是否有值

2.编译

源码编译一般来说有debug和release两种方式,但是debug多用于开发百衲衣测试,在正式的环境中编译请推荐使用release方式
Tips:debug方式我没有编译成功过,具体问题需要待研究
# 在你的trafodion 目录
d <Trafodion source directory>/core/sqf
source ./sqenvr.sh
cd ../..
执行 make all
或者make package-all(后续就不需要安装local_hadoop)
# 需要等待2-3h 没有任何编译错误,出现success 就算成功
[INFO] Building jar: /home/trafdeveloper/mysource/incubator-trafodion/dcs/target/dcs-1.3.0.jar  ##(DCS)
[INFO] Building tar: /home/trafdeveloper/mysource/incubator-trafodion/dcs/target/dcs-1.3.0.tar.gz  ##(DCS)
[INFO] BUILD SUCCESS  ##(DCS)
mkdir -p ../distribution  ##(DCS)
mv target/dcs-1.3.0.tar.gz ../distribution  ##(DCS)
rm -f trafodion-dcs.jar.versions  ##(DCS)
make[2]: Leaving directory `/home/trafdeveloper/mysource/incubator-trafodion/dcs' ##(DCS) make[1]: Leaving directory `/home/trafdeveloper/mysource/incubator-trafodion/core'
Tips:这部分完成需要验证编译情况
sqvers -u
sqcers -u
TRAF_HOME=/home/trafdeveloper/mysource/incubator-trafodion/core/sqf
who@host=trafdeveloper@Box01
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk.x86_64
SQ_MBTYPE=64d (64-debug)
linux=2.6.32-504.el6.x86_64
redhat=6.6
NO patches
Most common Apache_Trafodion Release 1.3.0 (Build debug [trafdeveloper],branch -,date 25Jan16)
UTT count is 1
[6]     Release 1.3.0 (Build debug [trafdeveloper],date 25Jan16)
          export/lib/hbase-trx-cdh5_3-1.3.0.jar
          export/lib/hbase-trx-hbase_98_4-1.3.0.jar
          export/lib/hbase-trx-hdp2_2-1.3.0.jar
          export/lib/sqmanvers.jar
          export/lib/trafodion-dtm-1.3.0.jar
          export/lib/trafodion-sql-1.3.0.jar

3.搭建测试环境

编译完成,我们需要配置SSH免登录和测试一下环境服务

3.1 ssh 免登录

ssh-keygen -t rsa -N "" -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_rsa
echo "NoHostAuthenticationForLocalhost=yes" >>~/.ssh/config
chmod go-w ~/.ssh/config
chmod 755 ~/.ssh; chmod 640 ~/.ssh/authorized_keys;

3.2 修改默认配置

edit /etc/security/limits.conf and reboot the server
# Changes added for the Trafodion Test Environment
* -       core            1000000
* -       nproc           267263
* -       nofile          32000
* -       sigpending      515196
* -       memlock         49595556
* -       stack           10240

3.3 安装local_hadoop

如果前面编译make package-all,这步可不需要
# 选择编译源
d <Trafodion source directory>/core/sqf
source ./sqenvr.sh
cd ../..
# 进入脚本目录
cds
install_local_hadoop
# 大约2.5-1h
Checking for existing Hadoop processes...
The testware tpcds_kit.zip does not exist and will not be installed
This testware is needed to run developer HIVE regression tests
.
.
.
Installed directory size and name = 2.7G        /home/trafdeveloper/mysource/incubator-trafodion/core/sqf/sql/local_hadoop

Setup is complete. You can use the convenience scripts starting with sw... located in /home/trafdeveloper/mysource/incubator-trafodion/core/sqf/sql/scripts.
$ ./install_traf_components

Installing and configuring DCS,REST,TRAFCI & Phoenix tests for Trafodion...

Environment used for core,DCS,REST and Phonenix ...
.
.
.
Configuration scripts for DCS,TRAFCI and Phoenix test are set up

   Open a new session and start Trafodion by executing sqgen and sqstart scripts
   # 编译成功执行
./install_traf_components

3.4 测试服务

测试服务状态
swstatus
# 无报错,打印两行字大致是所有服务以启动即可
# 一般HBase 没有正常启动
执行
swstarthbase 启动HMaster
# 
sqgen
# 重新打开终端,引入源,执行
sqstart
所有服务启动正常,没有down的服务即大功告成

4.总结

1.以上Tips都是常遇到的错误,请注意
2.Trafodion 基于hadoop的基础

最后欢迎一起讨论学习Trafodion,支持企业版Esgyn

相关文章

有时候CentOS工作在无互联网的环境下,需要在离线环境下安装一些组件,这次实现的是模拟在离线环境下安...
首先参照https://www.cnblogs.com/wdw984/p/13330074.html,来进行如何安装Centos和离线下载rpm包。 离...
有两个.NET CORE3.1网站部署在CentOS7上(内网IP是192.168.2.32),现在想实现访问http://192.168.2.32...
1、yum -y install vsftpd 安装vsftpd 2、配置vsftpd的配置文件(/etc/vsftpd/vsftpd.conf)主要修改以...
首先去mysql官网下载mysql的离线rpm安装包(https://downloads.mysql.com/archives/community/) Mysql...
第一步下载erlang环境并安装: wget https://packages.erlang-solutions.com/erlang/rpm/centos/7/x86_...