CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

时间:2023-03-09 02:49:46
CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

         CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

                                           作者:尹正杰

版权声明:原创作品,谢绝转载!否则将追究法律责任。 

 一.搭建CM私有仓库

  详情请参考我的笔记: https://www.cnblogs.com/yinzhengjie/p/10412455.html

1>.安装web服务器并启动 

[root@node105 ~]# yum -y install httpd
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
base | 3.6 kB ::
extras | 3.4 kB ::
updates | 3.4 kB ::
(/): extras//x86_64/primary_db | kB ::
(/): updates//x86_64/primary_db | 2.4 MB ::
Resolving Dependencies
--> Running transaction check
---> Package httpd.x86_64 :2.4.-.el7.centos will be installed
--> Processing Dependency: httpd-tools = 2.4.-.el7.centos for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: /etc/mime.types for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: libaprutil-.so.()(64bit) for package: httpd-2.4.-.el7.centos.x86_64
--> Processing Dependency: libapr-.so.()(64bit) for package: httpd-2.4.-.el7.centos.x86_64
--> Running transaction check
---> Package apr.x86_64 :1.4.-.el7_4. will be installed
---> Package apr-util.x86_64 :1.5.-.el7 will be installed
---> Package httpd-tools.x86_64 :2.4.-.el7.centos will be installed
---> Package mailcap.noarch :2.1.-.el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
httpd x86_64 2.4.-.el7.centos base 2.7 M
Installing for dependencies:
apr x86_64 1.4.-.el7_4. base k
apr-util x86_64 1.5.-.el7 base k
httpd-tools x86_64 2.4.-.el7.centos base k
mailcap noarch 2.1.-.el7 base k Transaction Summary
========================================================================================================================================================================
Install Package (+ Dependent packages) Total download size: 3.0 M
Installed size: M
Downloading packages:
(/): mailcap-2.1.-.el7.noarch.rpm | kB ::
(/): apr-1.4.-.el7_4..x86_64.rpm | kB ::
(/): apr-util-1.5.-.el7.x86_64.rpm | kB ::
(/): httpd-tools-2.4.-.el7.centos.x86_64.rpm | kB ::
(/): httpd-2.4.-.el7.centos.x86_64.rpm | 2.7 MB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | 3.0 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : apr-1.4.-.el7_4..x86_64 /
Installing : apr-util-1.5.-.el7.x86_64 /
Installing : httpd-tools-2.4.-.el7.centos.x86_64 /
Installing : mailcap-2.1.-.el7.noarch /
Installing : httpd-2.4.-.el7.centos.x86_64 /
Verifying : httpd-tools-2.4.-.el7.centos.x86_64 /
Verifying : apr-1.4.-.el7_4..x86_64 /
Verifying : mailcap-2.1.-.el7.noarch /
Verifying : httpd-2.4.-.el7.centos.x86_64 /
Verifying : apr-util-1.5.-.el7.x86_64 / Installed:
httpd.x86_64 :2.4.-.el7.centos Dependency Installed:
apr.x86_64 :1.4.-.el7_4. apr-util.x86_64 :1.5.-.el7 httpd-tools.x86_64 :2.4.-.el7.centos mailcap.noarch :2.1.-.el7 Complete!
[root@node105 ~]#

[root@node105 ~]# yum -y install httpd                              #安装httpd服务

[root@node105 ~]# cp /etc/httpd/conf/httpd.conf /etc/httpd/conf/httpd.conf.`date +%F`
[root@node105 ~]#
[root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #'
AddType application/x-gzip .gz .tgz
[root@node105 ~]#
[root@node105 ~]# sed -i s'#.tgz#.tgz .parcel#' /etc/httpd/conf/httpd.conf
[root@node105 ~]#
[root@node105 ~]# grep ".tgz" /etc/httpd/conf/httpd.conf | grep -v ' #'
AddType application/x-gzip .gz .tgz .parcel                        #其实我就是在.tgz后面加了一个“.parcel”的后缀,这是CM官方强调要做的一个操作!
[root@node105 ~]#
[root@node105 ~]#
[root@node105 ~]# systemctl enable httpd
Created symlink from /etc/systemd/system/multi-user.target.wants/httpd.service to /usr/lib/systemd/system/httpd.service.
[root@node105 ~]#
[root@node105 ~]# systemctl start httpd
[root@node105 ~]#
[root@node105 ~]#
[root@node105 ~]# systemctl status httpd
● httpd.service - The Apache HTTP Server
Loaded: loaded (/usr/lib/systemd/system/httpd.service; enabled; vendor preset: disabled)
Active: active (running) since Fri -- :: CST; 3s ago
Docs: man:httpd()
man:apachectl()
Main PID: (httpd)
Status: "Processing requests..."
CGroup: /system.slice/httpd.service
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
├─ /usr/sbin/httpd -DFOREGROUND
└─ /usr/sbin/httpd -DFOREGROUND Feb :: node105.yinzhengjie.org.cn systemd[]: Starting The Apache HTTP Server...
Feb :: node105.yinzhengjie.org.cn systemd[]: Started The Apache HTTP Server.
[root@node105 ~]#
[root@node105 ~]#

[root@node105 ~]# systemctl start httpd              #启动httpd服务

2>.下载CM对应到版本到对应到自建到web路径中(https://archive.cloudera.com/cm5/redhat/7/x86_64/cm/5.16.1/RPMS/x86_64/

[root@node105 ~]# mkdir /var/www/html/cm-5.16.                          #创建CM安装包存放目录
[root@node105 ~]#
[root@node105 ~]# ll /var/www/html/cm-5.16./                          #去官方网站把你需要的rpm包都下载到你到安装目录即可!
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm            #这个软件是我从Oracle官方上download下来的,CM默认的JDK是6或者7,而Oracle官方已经不支持使用JDK7啦!
-rw-r--r-- 1 root root      1690 Feb 21  2018 RPM-GPG-KEY-cloudera
[root@node105 ~]#
[root@node105 ~]#

3>.制作本地镜像

[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors.aliyun.com
* extras: mirrors.tuna.tsinghua.edu.cn
* updates: mirrors.tuna.tsinghua.edu.cn
Resolving Dependencies
--> Running transaction check
---> Package createrepo.noarch :0.9.-.el7 will be installed
--> Processing Dependency: python-deltarpm for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: libxml2-python for package: createrepo-0.9.-.el7.noarch
--> Processing Dependency: deltarpm for package: createrepo-0.9.-.el7.noarch
---> Package yum-plugin-priorities.noarch :1.1.-.el7 will be installed
---> Package yum-utils.noarch :1.1.-.el7 will be installed
--> Processing Dependency: python-kitchen for package: yum-utils-1.1.-.el7.noarch
--> Running transaction check
---> Package deltarpm.x86_64 :3.6-.el7 will be installed
---> Package libxml2-python.x86_64 :2.9.-.el7_2. will be installed
---> Package python-deltarpm.x86_64 :3.6-.el7 will be installed
---> Package python-kitchen.noarch :1.1.-.el7 will be installed
--> Processing Dependency: python-chardet for package: python-kitchen-1.1.-.el7.noarch
--> Running transaction check
---> Package python-chardet.noarch :2.2.-.el7_1 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
createrepo noarch 0.9.-.el7 base k
yum-plugin-priorities noarch 1.1.-.el7 base k
yum-utils noarch 1.1.-.el7 base k
Installing for dependencies:
deltarpm x86_64 3.6-.el7 base k
libxml2-python x86_64 2.9.-.el7_2. base k
python-chardet noarch 2.2.-.el7_1 base k
python-deltarpm x86_64 3.6-.el7 base k
python-kitchen noarch 1.1.-.el7 base k Transaction Summary
========================================================================================================================================================================
Install Packages (+ Dependent packages) Total download size: 1.1 M
Installed size: 4.8 M
Downloading packages:
(/): libxml2-python-2.9.-.el7_2..x86_64.rpm | kB ::
(/): createrepo-0.9.-.el7.noarch.rpm | kB ::
(/): python-deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): deltarpm-3.6-.el7.x86_64.rpm | kB ::
(/): yum-utils-1.1.-.el7.noarch.rpm | kB ::
(/): python-chardet-2.2.-.el7_1.noarch.rpm | kB ::
(/): yum-plugin-priorities-1.1.-.el7.noarch.rpm | kB ::
(/): python-kitchen-1.1.-.el7.noarch.rpm | kB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total kB/s | 1.1 MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : deltarpm-3.6-.el7.x86_64 /
Installing : libxml2-python-2.9.-.el7_2..x86_64 /
Installing : python-deltarpm-3.6-.el7.x86_64 /
Installing : python-chardet-2.2.-.el7_1.noarch /
Installing : python-kitchen-1.1.-.el7.noarch /
Installing : yum-utils-1.1.-.el7.noarch /
Installing : createrepo-0.9.-.el7.noarch /
Installing : yum-plugin-priorities-1.1.-.el7.noarch /
Verifying : yum-utils-1.1.-.el7.noarch /
Verifying : python-deltarpm-3.6-.el7.x86_64 /
Verifying : yum-plugin-priorities-1.1.-.el7.noarch /
Verifying : libxml2-python-2.9.-.el7_2..x86_64 /
Verifying : deltarpm-3.6-.el7.x86_64 /
Verifying : createrepo-0.9.-.el7.noarch /
Verifying : python-chardet-2.2.-.el7_1.noarch /
Verifying : python-kitchen-1.1.-.el7.noarch / Installed:
createrepo.noarch :0.9.-.el7 yum-plugin-priorities.noarch :1.1.-.el7 yum-utils.noarch :1.1.-.el7 Dependency Installed:
deltarpm.x86_64 :3.6-.el7 libxml2-python.x86_64 :2.9.-.el7_2. python-chardet.noarch :2.2.-.el7_1 python-deltarpm.x86_64 :3.6-.el7
python-kitchen.noarch :1.1.-.el7 Complete!
[root@node105 ~]#

[root@node105 ~]# yum -y install yum-utils createrepo yum-plugin-priorities        #安装本地源制作相关工具

[root@node105 ~]# cd /var/www/html/cm-5.16./
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]# ll
total
-rwxr-xr-x root root Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm
-rw-r--r-- root root Feb RPM-GPG-KEY-cloudera
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]# createrepo .                              #创建yum源的元数据文件!
Spawning worker with pkgs
Spawning worker with pkgs
Workers Finished
Saving Primary metadata
Saving file lists metadata
Saving other metadata
Generating sqlite DBs
Sqlite DBs complete
[root@node105 cm-5.16.]#
[root@node105 cm-5.16.]# ll
total
-rwxr-xr-x root root   Feb : cloudera-manager-agent-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : cloudera-manager-server-db--5.16.-.cm5161.p0..el7.x86_64.rpm
-rwxr-xr-x root root Feb : jdk-8u201-linux-x64.rpm
drwxr-xr-x root root Feb : repodata                  #这个目录就是我们执行上面的命令后生成的目录,该文件不可被删除!否则你配置的yum的使用者依旧无法使用yum命令进行安装
-rw-r--r-- root root Feb RPM-GPG-KEY-cloudera [root@node105 cm-5.16.]#
[root@node105 cm-5.16.]#

4>.下载CDH的版本存放到指定路径(https://archive.cloudera.com/cdh5/parcels/5.15.1/

[root@node105 ~]# mkdir /var/www/html/cdh-5.15.
[root@node105 ~]#
[root@node105 ~]# cd /var/www/html/cdh-5.15./
[root@node105 cdh-5.15.]#
[root@node105 cdh-5.15.]# ll
total
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r--r-- root root Feb : manifest.json
[root@node105 cdh-5.15.]#

 5>.各个主机配置CM自建的yum仓库

[root@node101 ~]# cat /etc/yum.repos.d/cloudera-manager.repo
[cloudera-manager]
# Packages for Cloudera Manager, Version , on RedHat or CentOS x86_64
name=Cloudera Manager
baseurl=http://node105.yinzhengjie.org.cn/cm-5.16.1/
gpgkey =http://node105.yinzhengjie.org.cn/cm-5.16.1//RPM-GPG-KEY-cloudera
enabled =
gpgcheck =
[root@node101 ~]#

二.安装和配置元数据库

  详情请参考我的笔记:安装和配置元数据库

三.Cloudera Manager 部署

  安装服务前请确保你磁盘的可用空间大于50G!

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

1>.安装Cloudera Manager Server端(由于我们使用自建的镜像地址安装,而且也配置好了yum源,我们这里直接安装即可!)

[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server
Loaded plugins: fastestmirror
Loading mirror speeds from cached hostfile
* base: mirrors..com
* extras: mirror01.idc.hinet.net
* updates: centos.cs.nctu.edu.tw
Resolving Dependencies
--> Running transaction check
---> Package cloudera-manager-daemons.x86_64 :5.16.-.cm5161.p0..el7 will be installed
---> Package cloudera-manager-server.x86_64 :5.16.-.cm5161.p0..el7 will be installed
--> Finished Dependency Resolution Dependencies Resolved ========================================================================================================================================================================
Package Arch Version Repository Size
========================================================================================================================================================================
Installing:
cloudera-manager-daemons x86_64 5.16.-.cm5161.p0..el7 cloudera-manager M
cloudera-manager-server x86_64 5.16.-.cm5161.p0..el7 cloudera-manager 8.5 k Transaction Summary
========================================================================================================================================================================
Install Packages Total download size: M
Installed size: M
Downloading packages:
(/): cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64.rpm | 8.5 kB ::
(/): cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64.rpm | MB ::
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total MB/s | MB ::
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Installing : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64 /
Installing : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64 /
Verifying : cloudera-manager-server-5.16.-.cm5161.p0..el7.x86_64 /
Verifying : cloudera-manager-daemons-5.16.-.cm5161.p0..el7.x86_64 / Installed:
cloudera-manager-daemons.x86_64 :5.16.-.cm5161.p0..el7 cloudera-manager-server.x86_64 :5.16.-.cm5161.p0..el7 Complete!
[root@node101 ~]#

[root@node101 ~]# yum -y install cloudera-manager-daemons cloudera-manager-server

2>.配置Cloudera Manager Server的默认堆内存大小

[root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server              #Cloudera Manager默认分配的JVM堆内存是2G,某些情况下会出现OOM的情况,我们可以适当将其调大。
export CMF_JAVA_OPTS="-Xmx2G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"
[root@node101 ~]#
[root@node101 ~]# sed -i s'#Xmx2G#Xmx4G#' /etc/default/cloudera-scm-server           #我们将Cloudera Manager的堆内存调大到4G。
[root@node101 ~]#
[root@node101 ~]# grep CMF_JAVA_OPTS /etc/default/cloudera-scm-server
export CMF_JAVA_OPTS="-Xmx4G -XX:MaxPermSize=256m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/tmp"
[root@node101 ~]#
[root@node101 ~]#

3>.初始化数据库配置

  Cloudera官网已经提供了一个sh脚本用户配置Cloudera Manager连接元数据库,官方文档参考:

    https://www.cloudera.com/documentation/enterprise/5-14-x/topics/cm_ig_installing_configuring_dbs.html#concept_i2r_m3m_hn

  我们的安装规划下,元数据库和Cloudera Manager运行于同一个节点,且已经预先创建了scm库和用户,因此,命令很简单,具体操作如下:

[root@node101 ~]# /usr/share/cmf/schema/scm_prepare_database.sh  mysql scm scm yinzhengjie
JAVA_HOME=/usr/java/jdk1.
Verifying that we can write to /etc/cloudera-scm-server
Creating SCM configuration file in /etc/cloudera-scm-server
Executing: /usr/java/jdk1./bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/usr/share/java/postgresql-connector-java.jar:/usr/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.
[ main] DbCommandExecutor INFO Successfully connected to database.
All done, your SCM database is configured correctly!                      #注意,如果你看到了这一行提示,说明配置完成,否则需要根据提示解决相应的错误!
[root@node101 ~]#

4>.启动Cloudera Manager Server服务

[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: inactive (dead)
Docs: man:systemd-sysv-generator() Feb :: node101.yinzhengjie.org.cn systemd[]: [/run/systemd/generator.late/cloudera-scm-server.service:] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument
Feb :: node101.yinzhengjie.org.cn systemd[]: [/run/systemd/generator.late/cloudera-scm-server.service:] Failed to add dependency on +cloudera-scm-server-db.service, ignoring: Invalid argument
[root@node101 ~]#
[root@node101 ~]# systemctl start cloudera-scm-server                       #启动Cloudera Manager,所需要的时间可能在5~10秒左右!
[root@node101 ~]#
[root@node101 ~]# systemctl status cloudera-scm-server
● cloudera-scm-server.service - LSB: Cloudera SCM Server
Loaded: loaded (/etc/rc.d/init.d/cloudera-scm-server; bad; vendor preset: disabled)
Active: active (exited) since Thu -- :: CST; 2min 38s ago
Docs: man:systemd-sysv-generator()
Process: ExecStart=/etc/rc.d/init.d/cloudera-scm-server start (code=exited, status=/SUCCESS) Feb :: node101.yinzhengjie.org.cn systemd[]: Starting LSB: Cloudera SCM Server...
Feb :: node101.yinzhengjie.org.cn su[]: PAM unable to dlopen(/usr/lib64/security/pam_fprintd.so): /usr/lib64/security/pam_fprintd.so: cannot open shared object file: No such file or directory
Feb :: node101.yinzhengjie.org.cn su[]: PAM adding faulty module: /usr/lib64/security/pam_fprintd.so
Feb :: node101.yinzhengjie.org.cn su[]: (to cloudera-scm) root on none
Feb :: node101.yinzhengjie.org.cn cloudera-scm-server[]: Starting cloudera-scm-server: [ OK ]
Feb :: node101.yinzhengjie.org.cn systemd[]: Started LSB: Cloudera SCM Server.
[root@node101 ~]#

[root@node101 ~]# systemctl start cloudera-scm-server                       #启动Cloudera Manager,所需要的时间可能在5~10秒左右!

5>.查看Cloudera Manager Server的日志并访问WebUI

[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log                      #查看Cloudera Manager的服务的日志
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Initializing SearchTemplateManager:--21T04::.540Z
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating entities:--21T04::.673Z
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num entities:
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Generating documents:--21T04::.721Z
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Num docs:
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Constructing repo:--21T04::.823Z
-- ::, INFO WebServerImpl:org.mortbay.log: jetty-6.1..cloudera.
-- ::, INFO WebServerImpl:org.mortbay.log: Started SelectChannelConnector@0.0.0.0:        #注意,如果出现这一行提示,说明你的服务启动成功啦!默认端口为7180,下面有WebServerImpl:Started Jetty server提示!
-- ::, INFO WebServerImpl:com.cloudera.server.cmf.WebServerImpl: Started Jetty server.
-- ::, INFO SearchRepositoryManager-:com.cloudera.server.web.cmf.search.components.SearchRepositoryManager: Finished constructing repo:--21T04::.827Z

[root@node101 ~]# tail -10f /var/log/cloudera-scm-server/cloudera-scm-server.log                     #查看Cloudera Manager的服务的日志

  打开浏览器,访问:http://node101.yinzhengjie.org.cn:7180/cmf/login,如下图所示:

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

四.安装CDH5.15.1

1>.登陆Cloudera Manager Server 的WebUI

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

2>.接受协议,点击继续

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

3>.选择企业版(该版本的CDH是免费的)

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

4>.选择CDH

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

5>.为 CDH 群集安装指定主机。

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

6>.点击更多选项

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

7>.选择自建的CDH Parcels包镜像地址

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

8>.配置Cloudera Manager Agent的地址

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

9>.不要选中安装oracle Java,因为我们已经安装过JDK8,选中的话会装会JDK7,直接点击继续即可!

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

10>.不启用单用户模式安装,直接点击继续即可 

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

11>.提供SSH登陆凭据

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

12>.等待Cloudera Manager Agent 安装完成

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

13>.查看安装过程的日志详情

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

14>.安装 Cloudera Manager Agent成功(根据集群性而定,我使用的是MAC,该过程大概5~10分钟左右!)

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

15>.下载CDH(从我们自建的镜像地址下载)

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

16>.CDH安装完成,点击继续

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

17>.等待主机检查

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

18>.点击完成

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

19>.完成上述步骤就已经完成了CDH的安装,还有一种方法安装CDH,就是下载CDH并存放到指定目录(了解即可!

  第一步:下载CDH 5.15.1的parcels包(https://archive.cloudera.com/cdh5/parcels/5.15.1/

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

  第二步:将下载的rpm包存放到CM默认到目录中(/opt/cloudera/parcel-repo)

[root@node101 ~]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1
-rw-r--r-- root root Feb : manifest.json
[root@node101 ~]#
[root@node101 ~]# mv /opt/cloudera/parcel-repo/CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha1 /opt/cloudera/parcel-repo/CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha     #我们这里需要做一个重命名的操作!
[root@node101 ~]#
[root@node101 ~]# ll /opt/cloudera/parcel-repo/
total
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel
-rw-r--r-- root root Feb : CDH-5.15.-.cdh5.15.1.p0.-el7.parcel.sha
-rw-r--r-- root root Feb : manifest.json
[root@node101 ~]#

五.安装Hadoop初始化的服务

1>.选择自定义服务

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

2>.选择需要安装的服务,点击继续(为了减少虚拟机的开销和安装时间,我这里只演示安装zookeeper集群和hdfs集群即可!其他服务的安装方法类似)

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

3>.自定义角色分配

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

4>.审核更改(这里咱们可以修改一些默认的配置,比如hdfs的块大小,数据存储的路径等等,实验环境我们默认即可,生产环境需要根据实际情况稍作修改即可!)

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

5>.安装初始化服务(安装过程时间可能需要3~5分钟)

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

6>.服务安装完成

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

7>.查看Cloudera Manager的监控界面

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager

8>.安装服务完成后,我们可以查看到剩余的可用空间

CDH构建大数据平台-使用自建的镜像地址安装Cloudera Manager