ubuntu下安装kafka集群connector

时间:2023-01-26 19:57:12


1.首先安装kafka集群,安装步骤参考链接如下:

2.创建安装connector安装目录
mkdir -p /kafka/kafka-1/kafka_2.12-2.2.1/connector-plugin
mkdir -p /kafka/kafka-2/kafka_2.12-2.2.1/connector-plugin
mkdir -p /kafka/kafka-3/kafka_2.12-2.2.1/connector-plugin
3.下载插件
下载链接: ​​​https://debezium.io/docs/install/stable/​

cd /kafka/kafka-1/kafka_2.12-2.2.1/connector-plugin
wget ​​​https://repo1.maven.org/maven2/io/debezium/debezium-connector-mysql/0.9.5.Final/debezium-connector-mysql-0.9.5.Final-plugin.tar.gz​​​ wget ​​https://repo1.maven.org/maven2/io/debezium/debezium-connector-postgres/0.9.5.Final/debezium-connector-postgres-0.9.5.Final-plugin.tar.gz​​ wget ​​https://repo1.maven.org/maven2/io/debezium/debezium-connector-mongodb/0.9.5.Final/debezium-connector-mongodb-0.9.5.Final-plugin.tar.gz​​ wget ​​https://repo1.maven.org/maven2/io/debezium/debezium-connector-sqlserver/0.9.5.Final/debezium-connector-sqlserver-0.9.5.Final-plugin.tar.gz​​ wget ​​https://repo1.maven.org/maven2/io/debezium/debezium-connector-oracle/0.9.5.Final/debezium-connector-oracle-0.9.5.Final-plugin.tar.gz​

下载完后解压
tar -zxvf debezium-connector-mysql-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-mongodb-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-oracle-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-postgres-0.9.5.Final-plugin.tar.gz
tar -zxvf debezium-connector-sqlserver-0.9.5.Final-plugin.tar.gz

4.下载后配置文件connect-distributed.properties
cd /kafka/kafka-1/kafka_2.12-2.2.1/config
vim connect-distributed.properties
修改1:bootstrap.servers=192.168.0.79:9092,192.168.0.79:9093,192.168.0.79:9094
修改2:rest.port=8083(端口依次增加,8084,8085)
修改3:plugin.path=/kafka/kafka-1/kafka_2.12-2.2.1/connector-plugin(修改目录编号)

cp -r debezium-connector-mysql /kafka/kafka-2/kafka_2.12-2.2.1/connector-plugin
cp /kafka/kafka-1/kafka_2.12-2.2.1/config/connect-distributed.properties /kafka/kafka-2/kafka_2.12-2.2.1/config/connect-distributed.properties

cp -r debezium-connector-mysql /kafka/kafka-3/kafka_2.12-2.2.1/connector-plugin
cp /kafka/kafka-1/kafka_2.12-2.2.1/config/connect-distributed.properties /kafka/kafka-3/kafka_2.12-2.2.1/config/connect-distributed.properties

cd /kafka/kafka-1/kafka_2.12-2.2.1/config
vim connect-distributed.properties

cd /kafka/kafka-2/kafka_2.12-2.2.1/config
vim connect-distributed.properties

cd /kafka/kafka-3/kafka_2.12-2.2.1/config
vim connect-distributed.properties

继续修改如上参数
5.启动
cd /kafka/kafka-1/kafka_2.12-2.2.1/bin
nohup sh ​​​connect-distributed.sh​​ …/config/connect-distributed.properties &

cd /kafka/kafka-2/kafka_2.12-2.2.1/bin
nohup sh ​​​connect-distributed.sh​​ …/config/connect-distributed.properties &

cd /kafka/kafka-3/kafka_2.12-2.2.1/bin
nohup sh ​​​connect-distributed.sh​​ …/config/connect-distributed.properties &