cdh版hbase构建Phoenix 遇到的坑

时间:2021-06-24 19:28:12

Phoenix 构建cdh版hbase遇到的坑

1. 安装phoenix

下载:在github上下载对应版本https://github.com/apache/phoenix

解压:略

编译

修改根目录及其子目录下的pom.xml文件,使cdh版本对应自己集群版本。如图所示

注意:编译中修改版本号遵循原始的写法。4.14.0-cdh5.12.1 版本不可以写成4.14-cdh5.12.1,因为在编译时会有正则校验,书写不规范会导致编译不通过。

cdh版hbase构建Phoenix 遇到的坑

命令:

 vim pom.xml
/cdh5. #搜索

修改所有的cdh版本号

cdh版hbase构建Phoenix 遇到的坑

cdh版hbase构建Phoenix 遇到的坑

编译:mvn clean package -DskipTests

拷贝jar :将下面的jar包拷贝至每一台RegionServer的lib下面

/opt/phoenix/APACHE_PHOENIX-4.14.-cdh5.12.1.p0./lib/phoenix/phoenix-4.14.-cdh5.12.1-server.jar

2. 配置hbase-site.xml

注意前方高能:phoenix4.8版本是个分水岭,配置有所不同

  • phoenix4.8以下版本的配置如下:

配置HMaster的hbase-site.xml

<!-- Phoenix订制的索引负载均衡器 -->
<property>
<name>hbase.master.loadbalancer.class</name> <value>org.apache.phoenix.hbase.index.balancer.IndexLoadBalancer</value>
</property>
<!-- Phoenix订制的索引观察者 -->
<property>
<name>hbase.coprocessor.master.classes</name> <value>org.apache.phoenix.hbase.index.master.IndexMasterObserver</value>
</property>

配置RegionServer的hbase-site.xml

<!-- Enables custom WAL edits to be written, ensuring proper writing/replay of the index updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression. -->
<property>
<name>hbase.regionserver.wal.codec</name>
<value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
</property>
<!-- Prevent deadlocks from occurring during index maintenance for global indexes (HBase 0.98.4+ and Phoenix 4.3.1+ only) by ensuring index updates are processed with a higher priority than data updates. It also prevents deadlocks by ensuring metadata rpc calls are processed with a higher priority than data rpc calls -->
<property>
<name>hbase.region.server.rpc.scheduler.factory.class</name>
<value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value>
<description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description>
</property>
<property>
<name>hbase.rpc.controllerfactory.class</name>
<value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value>
<description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description>
</property>
<!-- To support local index regions merge on data regions merge you will need to add the following parameter to hbase-site.xml in all the region servers and restart. (It’s applicable for Phoenix 4.3+ versions) -->
<property>
<name>hbase.coprocessor.regionserver.classes</name>
<value>org.apache.hadoop.hbase.regionserver.LocalIndexMerger</value>
</property>
  • phoenix4.8+版本只需添加以下配置
<!-- Enables custom WAL edits to be written, ensuring proper writing/replay of the index updates. This codec supports the usual host of WALEdit options, most notably WALEdit compression. -->
<property>
<name>hbase.regionserver.wal.codec</name>
<value>org.apache.hadoop.hbase.regionserver.wal.IndexedWALEditCodec</value>
</property>
<!-- Prevent deadlocks from occurring during index maintenance for global indexes (HBase 0.98.4+ and Phoenix 4.3.1+ only) by ensuring index updates are processed with a higher priority than data updates. It also prevents deadlocks by ensuring metadata rpc calls are processed with a higher priority than data rpc calls -->
<property>
<name>hbase.region.server.rpc.scheduler.factory.class</name>
<value>org.apache.hadoop.hbase.ipc.PhoenixRpcSchedulerFactory</value>
<description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description>
</property>
<property>
<name>hbase.rpc.controllerfactory.class</name>
<value>org.apache.hadoop.hbase.ipc.controller.ServerRpcControllerFactory</value>
<description>Factory to create the Phoenix RPC Scheduler that uses separate queues for index and metadata updates</description>
</property>
  • 解释

在4.8版本之后,安装Phoenix需要把hbase-site.xml文件中的这三个配置项给删掉,因为这三个配置项在4.7及之前的版本会导致在”security enabled”的情况下,每次建立和phoenix server的会话时都会建立一个新的连接实例,短时间内创建大量连接会触发Zookeeper内置的保护机制,拒绝连接。

简单来说,就是HBase在重启的时候,之前赋值到hbase lib目录下的phoenix-4.X.X-HBase-1.X-server.jar包会读取hbase-site.xml文件,结果发现了这三个过期的配置项,然后引起了异常,把这三个配置项删掉就好。。。。。。

(MMP,就这个搞了半天)

山重水复已无路,柳暗花明又一村。感谢这位熊蝶

https://blog.csdn.net/silentasriddle/article/details/78758950

如果是cdh版本的hbase,则在管理界面可直接修改,如下图(phoenxi 4.14的)

cdh版hbase构建Phoenix 遇到的坑

cdh版hbase构建Phoenix 遇到的坑

cdh版hbase构建Phoenix 遇到的坑

3. 修改和zookeeper相关的配置保证启动时通信不超时

修改zookeeper的配置maxSessionTimeout

默认是6000,改为1200000,单位是ms。这个值要大于hbase中的值。

cdh版hbase构建Phoenix 遇到的坑

修改hbase的配置

cdh版hbase构建Phoenix 遇到的坑

QQ:1710816711(欢迎交流,广告勿扰)