hbase部署
Hbase:
更细的操作和原理研究笔记和视频
cloudera Hbase:
https://sysit.cn/blog/post/sysit/cloudera%E5%AE%89%E8%A3%85HBASE
官网:
https://docs.cloudera.com/documentation/enterprise/6/6.2/topics/hbase.html
CDH启动hbase
https://sysit.cn/blog/post/sysit/cloudera%E5%AE%89%E8%A3%85HBASE
hbase和hive的区别:
hive是数据仓库,适用于海量数据的分析,不适用于实时的读写和更新.
hbase是数据库,适用于随机实时读写
和RDBMS(关系性数据库)的区别
Hbase是分布式数据库，其原型是 Google 的 BigTable，初始作为 Hadoop 的子项目来开发维护，用于支持结构化的数据存储。后期独立出来，成为Apache*项目。
Hbase是基于Hadoop的非关系型数据库，实时性，低延迟。分布式、可伸缩、大数据存储。适用于对大数据的随机实时读写操作。
官方对于Hbase单个表能够存储的容量定义为：十亿行x百万列x上千个版本
CDH上启用hbase:
集群--添加服务
部署到客户端并重启:
重启:
检查:
[root@zhep-opay-temp-1 ~]# /usr/bin/hbase shell
Java HotSpot(TM) 64-Bit Server VM warning: Using incremental CMS is deprecated and will likely be removed in a future release
HBase Shell
Use "help" to get list of supported commands.
Use "exit" to quit this interactive shell.
For Reference, please visit: http://hbase.apache.org/2.0/book.html#shell
Version 2.1.0-cdh6.2.0, rUnknown, Wed Mar 13 23:39:58 PDT 2019
Took 0.0024 seconds
hbase(main):002:0> help
habase手动hadoop集群安装:
wget http://mirrors.hust.edu.cn/apache/hbase/2.1.3/hbase-2.1.3-bin.tar.gz
tar -xzvf hbase-2.1.3-bin.tar.gz -C /opt/module/
vim /etc/profile
export HBASE_HOME=/opt/module/hbase-2.1.3
export PATH=$PATH:$HBASE_HOME/bin
cd /opt/module/hbase-2.1.3/conf/
mv hbase-env.sh hbase-env.sh.bak
vi hbase-env.sh
export JAVA_HOME=/usr/java/jdk1.8.0_181-cloudera
export HBASE_OPTS="-XX:+UseConcMarkSweepGC"
export HBASE_MANAGES_ZK=false
配置hbase-site.xml（清空原配置文件）
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
</property>
<property>
<name>hbase.rootdir</name>
<value>hdfs://mycluster/user/hbase</value>
</property>
<property>
<name>hbase.zookeeper.property.dataDir</name>
<value>/var/lib/zookeeper</value>
</property>
<property>
<name>hbase.zookeeper.quorum</name>
<value>zhep-opay-temp-2.novalocal:2181,zhep-opay-temp-3.novalocal:2181,zhep-opay-temp-4.novalocal:2181</value>
</property>
</configuration>
#解释
/opt/cloudera/parcels/CDH/lib/zookeeper
<name>hbase.zookeeper.property.dataDir</name>
<value>/zookeeper/data</value>
<description>
注意这里的zookeeper数据目录与hadoop ha的共用，也即要与 zoo.cfg 中配置的一致
Property from ZooKeeper config zoo.cfg.
The directory where the snapshot is stored.
</description>
<property>
<name>hbase.cluster.distributed</name>
<value>true</value>
<description>
分布式集群配置，这里要设置为true，如果是单节点的，则设置为false
The mode the cluster will be in. Possible values are
false: standalone and pseudo-distributed setups with managed ZooKeeper
true: fully-distributed with unmanaged ZooKeeper Quorum (see hbase-env.sh)
</description>
vim regionservers
zhep-opay-temp-1.novalocal
zhep-opay-temp-2.novalocal
zhep-opay-temp-3.novalocal
zhep-opay-temp-4.novalocal
将hdfs配置文件链接到hbase
为了使用上面的配置:
<name>hbase.rootdir</name>
<value>hdfs://mycluster/user/hbase</value>
------------------------
ln /etc/hadoop/conf/core-site.xml /opt/module/hbase-2.1.3/conf/core-site.xml
ln /etc/hadoop/conf/hdfs-site.xml /opt/module/hbase-2.1.3/conf/hdfs-site.xml
ansible-book把/etc/profile 和scp把hbase文件夹copy到从节点
启动:
[root@zhep-opay-temp-1 conf]# start-hbase.sh
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hbase-2.1.3/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
running master, logging to /opt/module/hbase-2.1.3/logs/hbase-root-master-zhep-opay-temp-1.novalocal.out
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hbase-2.1.3/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
zhep-opay-temp-1.novalocal: running regionserver, logging to /opt/module/hbase-2.1.3/bin/../logs/hbase-root-regionserver-zhep-opay-temp-1.novalocal.out
zhep-opay-temp-2.novalocal: running regionserver, logging to /opt/module/hbase-2.1.3/bin/../logs/hbase-root-regionserver-zhep-opay-temp-2.novalocal.out
zhep-opay-temp-3.novalocal: running regionserver, logging to /opt/module/hbase-2.1.3/bin/../logs/hbase-root-regionserver-zhep-opay-temp-3.novalocal.out
zhep-opay-temp-4.novalocal: running regionserver, logging to /opt/module/hbase-2.1.3/bin/../logs/hbase-root-regionserver-zhep-opay-temp-4.novalocal.out
zhep-opay-temp-1.novalocal: SLF4J: Class path contains multiple SLF4J bindings.
zhep-opay-temp-1.novalocal: SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-1.novalocal: SLF4J: Found binding in [jar:file:/opt/module/hbase-2.1.3/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-1.novalocal: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
zhep-opay-temp-1.novalocal: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
zhep-opay-temp-2.novalocal: SLF4J: Class path contains multiple SLF4J bindings.
zhep-opay-temp-2.novalocal: SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-2.novalocal: SLF4J: Found binding in [jar:file:/opt/module/hbase-2.1.3/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-2.novalocal: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
zhep-opay-temp-2.novalocal: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
zhep-opay-temp-3.novalocal: SLF4J: Class path contains multiple SLF4J bindings.
zhep-opay-temp-3.novalocal: SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-3.novalocal: SLF4J: Found binding in [jar:file:/opt/module/hbase-2.1.3/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-3.novalocal: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
zhep-opay-temp-3.novalocal: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
zhep-opay-temp-4.novalocal: SLF4J: Class path contains multiple SLF4J bindings.
zhep-opay-temp-4.novalocal: SLF4J: Found binding in [jar:file:/opt/cloudera/parcels/CDH-6.2.0-1.cdh6.2.0.p0.967373/jars/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-4.novalocal: SLF4J: Found binding in [jar:file:/opt/module/hbase-2.1.3/lib/client-facing-thirdparty/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
zhep-opay-temp-4.novalocal: SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
zhep-opay-temp-4.novalocal: SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
#检查,进入hbase shell
[root@zhep-opay-temp-1 conf]# hbase shell
查看节点信息
hbase(main):001:0> status
1 active master, 0 backup masters, 5 servers, 0 dead, 0.4000 average load
Took 0.3888 seconds
查看webUI，端口为16010
http://195.189.142.89:16010/master-status
jps:
