参考文章：https://blog.****.net/weixin_35852328/article/details/79327996
官网：https://azkaban.readthedocs.io/en/latest/
Azkaban是由Linkedin开源的一个批量工作流任务调度器。用于在一个工作流内以一个特定的顺序运行一组工作和流程。Azkaban定义了一种KV文件格式来建立任务之间的依赖关系，并提供一个易于使用的web用户界面维护和跟踪你的工作流。
它有如下功能特点：Web用户界面，方便上传工作流，方便设置任务之间的关系，调度工作流，认证/授权(权限的工作)，能够杀死并重新启动工作流，模块化和可插拔的插件机制，项目工作区，工作流和任务的日志记录和审计。azkaban工作流调度器及相关工具对比。
1.准备资源
注：因公司主机无法连外网，所以先用本地虚拟机执行下载编译后，将相关文件打包，再上传到公司服务器进行安装
1.1下载
如果没有git可以（yum -y git）,建议把yum源设置为阿里源，否则各种资源找不到
git clone -b 3.12. https://github.com/azkaban/azkaban.git
1.2编译
进入azkaban，执行编译,此过程异常痛苦，可以多执行几次（20次后才build success）注：使用gradle进行安装，若没有gradle会自行安装
cd azkaban
./gradlew clean
./gradlew distTar
编译完成后，结果在/root/azkaban/azkaban-exec-server/build/distributions下
[root@localhost distributions]# pwd
/root/azkaban/azkaban-exec-server/build/distributions
[root@localhost distributions]# ls
azkaban-exec-server-3.12..tar.gz
[root@localhost distributions]#
包说明
azkaban-common : 常用工具类。
azkaban-db : 对应的sql脚本
azkaban-hadoop-secutity-plugin : hadoop 有关kerberos插件
azkaban-solo-server: web和executor 一起的项目。
azkaban-web/executor-server:azkaban的 web和executor的server信息
azkaban-spi: azkaban存储接口以及exception类
1.3打包上传
打包所有文件
[root@localhost distributions]# cd /root
[root@localhost ~]# ls
anaconda-ks.cfg ap AutoConfig.sh azkaban pullK8sImages.sh
[root@localhost ~]# tar -cvf azkaban.tar azkaban
再将azkaban.tar上传至公司服务器
2.安装
root@hadoop01:/software>mkdir azkaban-3.12.
root@hadoop01:/software>cd azkaban-3.12./
root@hadoop01:/software/azkaban-3.12.>ls
root@hadoop01:/software/azkaban-3.12.>cp /software/azkaban/azkaban-exec-server/build/distributions/azkaban-exec-server-3.12..tar.gz .
root@hadoop01:/software/azkaban-3.12.>cp /software/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.12..tar.gz .
root@hadoop01:/software/azkaban-3.12.>cp /software/azkaban/azkaban-sql/build/distributions/azkaban-sql-3.12..tar.gz .
2.1解压
解压并重命名
root@hadoop01:/software/azkaban-3.12.>tar -xvf azkaban-exec-server-3.12..tar.gz
root@hadoop01:/software/azkaban-3.12.>tar -xvf azkaban-sql-3.12..tar.gz
root@hadoop01:/software/azkaban-3.12.>tar -xvf azkaban-web-server-3.12..tar.gz
root@hadoop01:/software/azkaban-3.12.>mv azkaban-exec-server-3.12. azkaban-exec-server
root@hadoop01:/software/azkaban-3.12.>mv azkaban-sql-3.12. azkaban-sql
root@hadoop01:/software/azkaban-3.12.>mv azkaban-web-server-3.12. azkaban-web-server
root@hadoop01:/software/azkaban-3.12.>ll
总用量
drwxr-xr-x root root 4月 : azkaban-exec-server
-rw------- root root 4月 : azkaban-exec-server-3.12..tar.gz
drwxr-xr-x root root 4月 : azkaban-sql
-rw------- root root 4月 : azkaban-sql-3.12..tar.gz
drwxr-xr-x root root 4月 : azkaban-web-server
-rw------- root root 4月 : azkaban-web-server-3.12..tar.gz
root@hadoop01:/software/azkaban-3.12.>
2.2 建表
>mysql -h10.10.10. -P3306 -uroot -p Copyright (c) , , Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| mysql |
| performance_schema |
| sys |
+--------------------+
rows in set (0.00 sec) mysql> create database azkaban;
Query OK, row affected (0.01 sec) mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| azkaban |
| mysql |
| performance_schema |
| sys |
+--------------------+
rows in set (0.00 sec) mysql> use azkaban;
Database changed
mysql> show tables;
Empty set (0.00 sec)
mysql> source /software/azkaban-3.12./azkaban-sql/create-all-sql-3.12..sql;
mysql> show tables;
+------------------------+
| Tables_in_azkaban |
+------------------------+
| active_executing_flows |
| active_sla |
| execution_flows |
| execution_jobs |
| execution_logs |
| executor_events |
| executors |
| project_events |
| project_files |
| project_flows |
| project_permissions |
| project_properties |
| project_versions |
| projects |
| properties |
| schedules |
| triggers |
+------------------------+
rows in set (0.00 sec)
2.3配置keystore
root@hadoop01:/software>find -name keytool
./old-bak/jdk1..0_162/bin/keytool
./old-bak/jdk1..0_162/jre/bin/keytool
root@hadoop01:/software>keytool -keystore keystore -alias jetty -genkey -keyalg RSA
输入密钥库口令:
密钥库口令太短 - 至少必须为 个字符
输入密钥库口令:
再次输入新口令:
您的名字与姓氏是什么?
[Unknown]: pupu
您的组织单位名称是什么?
[Unknown]: pupu
您的组织名称是什么?
[Unknown]: pupu
您所在的城市或区域名称是什么?
[Unknown]: beijing
您所在的省/市/自治区名称是什么?
[Unknown]: beijing
该单位的双字母国家/地区代码是什么?
[Unknown]: CN
CN=pupu, OU=pupu, O=pupu, L=beijing, ST=beijing, C=CN是否正确?
[否]: y 输入 <jetty> 的密钥口令
(如果和密钥库口令相同, 按回车): Warning:
JKS 密钥库使用专用格式。建议使用 "keytool -importkeystore -srckeystore keystore -destkeystore keystore -deststoretype pkcs12" 迁移到行业标准格式 PKCS12。
root@hadoop01:/software>cp keystore /software/azkaban-3.12.0/azkaban-web-server/bin
完成上述工作后,将在当前目录生成 keystore 证书文件,将keystore 考贝到 azkaban web服务器根目录中的bin目录下.如:cp keystore /software/azkaban-3.12.0/azkaban-web-server/bin
2.4修改配置
配置web-server
# 因为web-server 和 exec-server都没有默认配置文件我们需要复制solo的配置文件
修改azkaban.properties文件
root@hadoop01:/software/azkaban-3.12.>cp /software/azkaban/azkaban-solo-server/build/distributions/azkaban-solo-server-3.12..tar.gz .
root@hadoop01:/software/azkaban-3.12.>ls
azkaban-exec-server azkaban-exec-server-3.12..tar.gz azkaban-solo-server-3.12..tar.gz azkaban-sql azkaban-sql-3.12..tar.gz azkaban-web-server azkaban-web-server-3.12..tar.gz
root@hadoop01:/software/azkaban-3.12.>tar -xzvf azkaban-solo-server-3.12..tar.gz
root@hadoop01:/software/azkaban-3.12.>mv azkaban-solo-server-3.12. azkaban-solo-server
root@hadoop01:/software/azkaban-3.12.>cp -r /software/azkaban-3.12./azkaban-solo-server/conf azkaban-web-server
root@hadoop01:/software/azkaban-3.12.>cp /software/azkaban/azkaban-web-server/src/test/resources/log4j.properties azkaban-web-server/conf/
root@hadoop01:/software/azkaban-3.12.0>cp -r azkaban-solo-server/plugins/ azkaban-web-server
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>pwd
/software/azkaban-3.12./azkaban-web-server/conf
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>cat azkaban.properties
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
#default.timezone.id=America/Los_Angeles
default.timezone.id=Asia/Shanghai # Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=conf/azkaban-users.xml # Loader for projects
executor.global.properties=conf/global.properties
azkaban.project.dir=projects #database.type=h2
#h2.path=./h2
#h2.create.tables=true
database.type=mysql
mysql.port=
mysql.host=your host IP
mysql.database=azkaban
mysql.user=root
mysql.password=your password
mysql.numconnections= # Velocity dev mode
velocity.dev.mode=false # Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=
jetty.port= # Azkaban Executor settings
executor.port= # mail settings
mail.sender=
mail.host=
job.failure.email=
job.success.email= lockdown.create.projects=false cache.directory=cache # JMX stats
jetty.connector.stats=true
executor.connector.stats=true # Azkaban plugin settings
azkaban.jobtype.plugin.dir=plugins/jobtypes
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes>pwd
/software/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes>cat commonprivate.properties
# set execute-as-user
azkaban.native.lib=false
execute.as.user=false
2.5 web-server启动验证及问题修正
2.5.1启动web-serrver并验证
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh /software/azkaban-3.12./azkaban-solo-server/bin/azkaban-solo-shutdown.sh
cat: /software/azkaban-3.12./azkaban-solo-server/bin/../currentpid: 没有那个文件或目录
killing AzkabanSingleServer
kill: 用法:kill [-s 信号声明 | -n 信号编号 | -信号声明] 进程号 | 任务声明 ... 或 kill -l [信号声明]
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh start-web.sh
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>start-web.sh:行5: logs/webServerLog_2019--+::.out: 没有那个文件或目录
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/bin/mkdir logs
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/bin>ls
azkaban-web-shutdown.sh azkaban-web-start.sh keystore logs start-web.sh
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/bin>cat start-web.sh
#!/bin/bash
base_dir=$(dirname $0)/..
#bin/azkaban-web-start.sh $base_dir >logs/webServerLog_`date +%F+%T`.out 2>&1 &
nohup sh azkaban-web-start.sh $base_dir >logs/webServerLog_`date +%F+%T`.out 2>&1 &
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/bin>cd logs
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/bin/logs>ll
总用量 12
-rw------- 1 root root 73 4月 25 17:15 webServerLog_2019-04-25+17:15:15.out
-rw------- 1 root root 5040 4月 25 17:18 webServerLog_2019-04-25+17:18:53.out
root@hadoop01:/software/azkaban-3.12.0/azkaban-web-server/bin/logs>cat webServerLog_2019-04-25+17:18:53.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7.2.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-web-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7.0.jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/commons-validator-1.3.1.jar:./../lib/data-1.15.7.jar:./../lib/data-transform-1.15.7.jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2.2.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/javax.inject-1.jar:./../lib/javax.servlet-api-3.0.1.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-20070829.jar:./../lib/li-jersey-uri-1.15.7.jar:./../lib/log4j-1.2.16.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mina-core-1.1.7.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/netty-3.2.3.Final.jar:./../lib/oro-2.0.8.jar:./../lib/parseq-1.3.6.jar:./../lib/pegasus-common-1.15.7.jar:./../lib/quartz-2.2.1.jar:./../lib/r2-1.15.7.jar:./../lib/restli-common-1.15.7.jar:./../lib/restli-server-1.15.7.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.7.jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-0.jar:./../lib/struts-core-1.3.8.jar:./../lib/struts-taglib-1.3.8.jar:./../lib/struts-tiles-1.3.8.jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
2019/04/25 17:18:53.533 +0800 INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
2019/04/25 17:18:53.547 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/25 17:18:53.548 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/25 17:18:53.549 +0800 INFO [AzkabanWebServer] Setting up connector with stats on: true
2019/04/25 17:18:53.569 +0800 ERROR [StdOutErrRedirect] SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
2019/04/25 17:18:53.569 +0800 ERROR [StdOutErrRedirect] SLF4J: Defaulting to no-operation (NOP) logger implementation
2019/04/25 17:18:53.569 +0800 ERROR [StdOutErrRedirect] SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2019/04/25 17:18:53.653 +0800 INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
2019/04/25 17:18:53.653 +0800 ERROR [AzkabanWebServer] Could not instantiate UserManager azkaban.user.XmlUserManager
2019/04/25 17:18:53.653 +0800 ERROR [StdOutErrRedirect] Exception in thread "main"
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:264)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.<init>(AzkabanWebServer.java:198)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:762)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] Caused by: java.lang.reflect.InvocationTargetException
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
2019/04/25 17:18:53.654 +0800 ERROR [StdOutErrRedirect] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:260)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] ... 2 more
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] Caused by: java.lang.IllegalArgumentException: User xml file conf/azkaban-users.xml doesn't exist.
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at azkaban.user.XmlUserManager.parseXMLFile(XmlUserManager.java:90)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] at azkaban.user.XmlUserManager.<init>(XmlUserManager.java:84)
2019/04/25 17:18:53.655 +0800 ERROR [StdOutErrRedirect] ... 7 more
2.5.2 报错：Failed to load class "org.slf4j.impl.StaticLoggerBinder"
解决上面问题，看起来是缺少slf4j包
比较azkaban-web-server/lib 和azkaban-solo-server/lib下slf4j相关的包，发现少的包应该是slf4j-log4j12-1.7.7.jar
root@hadoop01:/software/azkaban-3.12./azkaban-web-server>ls lib
activation-1.1.jar commons-codec-1.6.jar commons-jexl-2.1..jar dom4j-1.1.jar javax.servlet-api-3.0..jar mail-1.4..jar pegasus-common-1.15..jar
antlr-2.7..jar commons-collections-3.2..jar commons-lang-2.6.jar guava-13.0..jar jetty-6.1..jar metrics-core-3.1..jar quartz-2.2..jar snappy-0.3.jar
azkaban-common-3.12..jar commons-compress-1.2.jar commons-logging-1.1..jar httpclient-4.3..jar jetty-util-6.1..jar metrics-jvm-3.1..jar r2-1.15..jar sslext-1.2-.jar
azkaban-web-server-3.12..jar commons-dbcp-1.4.jar commons-math3-3.0.jar httpcore-4.3.jar joda-time-2.0.jar mina-core-1.1..jar restli-common-1.15..jar struts-core-1.3..jar
c3p0-0.9.1.1.jar commons-dbutils-1.5.jar commons-pool-1.5..jar jackson-core-2.2..jar jopt-simple-4.3.jar mysql-connector-java-5.1..jar restli-server-1.15..jar struts-taglib-1.3..jar
cglib-nodep-2.2.jar commons-digester-1.8.jar commons-validator-1.3..jar jackson-core-asl-1.9..jar json-.jar netty-3.2..Final.jar servlet-api-2.5-.jar struts-tiles-1.3..jar
commons-beanutils-1.7..jar commons-fileupload-1.2..jar data-1.15..jar jackson-mapper-asl-1.9..jar li-jersey-uri-1.15..jar oro-2.0..jar servlet-api-2.5.jar velocity-1.7.jar
commons-chain-1.1.jar commons-io-2.4.jar data-transform-1.15..jar javax.inject-.jar log4j-1.2.16.jar parseq-1.3..jar slf4j-api-1.7.7.jar velocity-tools-2.0.jar
root@hadoop01:/software/azkaban-3.12./azkaban-web-server>ls ../azkaban-solo-server/lib
activation-1.1.jar commons-collections-3.2..jar commons-pool-1.5..jar jackson-core-asl-1.9..jar kafka-clients-0.10.0.0.jar oro-2.0..jar snappy-0.3.jar
antlr-2.7..jar commons-compress-1.2.jar commons-validator-1.3..jar jackson-mapper-asl-1.9..jar kafka-log4j-appender-0.10.0.0.jar parseq-1.3..jar snappy-java-1.1.2.4.jar
azkaban-common-3.12..jar commons-dbcp-1.4.jar data-1.15..jar javax.inject-.jar li-jersey-uri-1.15..jar pegasus-common-1.15..jar sslext-1.2-.jar
azkaban-exec-server-3.12..jar commons-dbutils-1.5.jar data-transform-1.15..jar javax.servlet-api-3.0..jar log4j-1.2.17.jar quartz-2.2..jar struts-core-1.3..jar
azkaban-solo-server-3.12..jar commons-digester-1.8.jar dom4j-1.1.jar jetty-6.1..jar lz4-1.3..jar r2-1.15..jar struts-taglib-1.3..jar
azkaban-web-server-3.12..jar commons-fileupload-1.2..jar guava-13.0..jar jetty-util-6.1..jar mail-1.4..jar restli-common-1.15..jar struts-tiles-1.3..jar
c3p0-0.9.1.1.jar commons-io-2.4.jar h2-1.4..jar joda-time-2.0.jar metrics-core-3.1..jar restli-server-1.15..jar velocity-1.7.jar
cglib-nodep-2.2.jar commons-jexl-2.1..jar hamcrest-core-1.1.jar jopt-simple-4.3.jar metrics-jvm-3.1..jar servlet-api-2.5-.jar velocity-tools-2.0.jar
commons-beanutils-1.7..jar commons-lang-2.6.jar httpclient-4.3..jar json-.jar mina-core-1.1..jar servlet-api-2.5.jar
commons-chain-1.1.jar commons-logging-1.1..jar httpcore-4.3.jar json-simple-1.1..jar mysql-connector-java-5.1..jar slf4j-api-1.7.21.jar
commons-codec-1.6.jar commons-math3-3.0.jar jackson-core-2.2..jar junit-4.10.jar netty-3.2..Final.jar slf4j-log4j12-1.7.21.jar
root@hadoop01:/software/azkaban-3.12./azkaban-web-server>
然后到https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12/1.7.7，上将下面代码copy到idea中任一项目的pom.xml，后会自动下载到本地仓库，找到后将其传入上面azkaban-web-server/lib下，此问题解决
<!-- https://mvnrepository.com/artifact/org.slf4j/slf4j-log4j12 -->
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.7</version>
<scope>test</scope>
</dependency>
后再次重启azkaban-web-server/bin下sh start-web.sh
2.5.3 报错：Could not instantiate UserManager azkaban.user.XmlUserManager
检查/software/azkaban-3.12.0/azkaban-web-server/conf/下有此文件azkaban-users.xml ，可将azkaban.properties中相关配置设置为绝对路径
oot@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>cat webServerLog_2019--+::.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
// ::00.678 + INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
// ::00.692 + INFO [AzkabanServer] Loading azkaban settings file from ./../conf
// ::00.692 + INFO [AzkabanServer] Loading azkaban properties file
// ::00.694 + INFO [AzkabanWebServer] Setting up connector with stats on: true
// ::00.715 + INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
// ::00.762 + INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
// ::00.762 + ERROR [AzkabanWebServer] Could not instantiate UserManager azkaban.user.XmlUserManager
// ::00.763 + ERROR [StdOutErrRedirect] Exception in thread "main"
// ::00.763 + ERROR [StdOutErrRedirect] java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
// ::00.763 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:)
// ::00.763 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.<init>(AzkabanWebServer.java:)
// ::00.763 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:)
// ::00.764 + ERROR [StdOutErrRedirect] Caused by: java.lang.reflect.InvocationTargetException
// ::00.764 + ERROR [StdOutErrRedirect] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
// ::00.764 + ERROR [StdOutErrRedirect] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:)
// ::00.764 + ERROR [StdOutErrRedirect] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:)
// ::00.764 + ERROR [StdOutErrRedirect] at java.lang.reflect.Constructor.newInstance(Constructor.java:)
// ::00.764 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadUserManager(AzkabanWebServer.java:)
// ::00.764 + ERROR [StdOutErrRedirect] ... more
// ::00.764 + ERROR [StdOutErrRedirect] Caused by: java.lang.IllegalArgumentException: User xml file conf/azkaban-users.xml doesn't exist.
// ::00.764 + ERROR [StdOutErrRedirect] at azkaban.user.XmlUserManager.parseXMLFile(XmlUserManager.java:)
// ::00.764 + ERROR [StdOutErrRedirect] at azkaban.user.XmlUserManager.<init>(XmlUserManager.java:)
// ::00.764 + ERROR [StdOutErrRedirect] ... more
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>
更改过后的配置文件
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>ls
azkaban.properties azkaban-users.xml global.properties log4j.properties
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>pwd
/software/azkaban-3.12./azkaban-web-server/conf
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>cat azkaban.properties
# Azkaban Personalization Settings
azkaban.name=Test
azkaban.label=My Local Azkaban
azkaban.color=#FF3601
azkaban.default.servlet.path=/index
web.resource.dir=web/
#default.timezone.id=America/Los_Angeles
default.timezone.id=Asia/Shanghai # Azkaban UserManager class
user.manager.class=azkaban.user.XmlUserManager
user.manager.xml.file=/software/azkaban-3.12./azkaban-web-server/conf/azkaban-users.xml # Loader for projects
executor.global.properties=/software/azkaban-3.12./azkaban-web-server/conf/global.properties
azkaban.project.dir=projects #database.type=h2
#h2.path=./h2
#h2.create.tables=true
database.type=mysql
mysql.port=
mysql.host=youknow
mysql.database=azkaban
mysql.user=root
mysql.password=pwd
mysql.numconnections= # Velocity dev mode
velocity.dev.mode=false # Azkaban Jetty server properties.
jetty.use.ssl=false
jetty.maxThreads=
jetty.port= # Azkaban Executor settings
executor.port= # mail settings
mail.sender=
mail.host=
job.failure.email=
job.success.email= lockdown.create.projects=false cache.directory=cache # JMX stats
jetty.connector.stats=true
executor.connector.stats=true # Azkaban plugin settings
azkaban.jobtype.plugin.dir=/software/azkaban-3.12./azkaban-web-server/plugins/jobtypes
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf>
再次重启，进程已经启动起来了
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh start-web.sh
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>ps -ef|grep az
root : pts/ :: java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root : pts/ :: grep --color=auto az
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>
2.5.4 报错：java.sql.SQLException: Unknown column 'ex.executor_id'
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>cd logs
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>ls
webServerLog_2019--+::.out webServerLog_2019--+::.out webServerLog_2019--+::.out webServerLog_2019--+::.out webServerLog_2019--+::.out webServerLog_2019--+::.out
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>ll
总用量
-rw------- root root 4月 : webServerLog_2019--+::.out
-rw------- root root 4月 : webServerLog_2019--+::.out
-rw------- root root 4月 : webServerLog_2019--+::.out
-rw------- root root 4月 : webServerLog_2019--+::.out
-rw------- root root 4月 : webServerLog_2019--+::.out
-rw------- root root 4月 : webServerLog_2019--+::.out
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>cat webServerLog_2019--+::.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
// ::00.230 + INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
// ::00.247 + INFO [AzkabanServer] Loading azkaban settings file from ./../conf
// ::00.247 + INFO [AzkabanServer] Loading azkaban properties file
// ::00.249 + INFO [AzkabanWebServer] Setting up connector with stats on: true
// ::00.270 + INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
// ::00.319 + INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
// ::00.456 + INFO [XmlUserManager] Loading user azkaban
// ::00.456 + INFO [XmlUserManager] Loading user metrics
// ::00.486 + INFO [ExecutorManager] Initializing local executor localhost:
// ::00.785 + ERROR [StdOutErrRedirect] Exception in thread "main"
// ::00.785 + ERROR [StdOutErrRedirect] azkaban.executor.ExecutorManagerException: Error fetching active flows
// ::00.785 + ERROR [StdOutErrRedirect] at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:)
// ::00.785 + ERROR [StdOutErrRedirect] at azkaban.executor.ExecutorManager.loadRunningFlows(ExecutorManager.java:)
// ::00.785 + ERROR [StdOutErrRedirect] at azkaban.executor.ExecutorManager.<init>(ExecutorManager.java:)
// ::00.786 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.loadExecutorManager(AzkabanWebServer.java:)
// ::00.786 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.<init>(AzkabanWebServer.java:)
// ::00.786 + ERROR [StdOutErrRedirect] at azkaban.webapp.AzkabanWebServer.main(AzkabanWebServer.java:)
// ::00.786 + ERROR [StdOutErrRedirect] Caused by: java.sql.SQLException: Unknown column 'ex.executor_id' in 'on clause' Query: SELECT ex.exec_id exec_id, ex.enc_type enc_type, ex.flow_data flow_data, et.host host, et.port port, ax.update_time axUpdateTime, et.id executorId, et.active executorStatus FROM execution_flows ex INNER JOIN active_executing_flows ax ON ex.exec_id = ax.exec_id INNER JOIN executors et ON ex.executor_id = et.id Parameters: []
2019/04/25 18:23:00.786 +0800 ERROR [StdOutErrRedirect] at org.apache.commons.dbutils.AbstractQueryRunner.rethrow(AbstractQueryRunner.java:363)
// ::00.786 + ERROR [StdOutErrRedirect] at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:)
// ::00.786 + ERROR [StdOutErrRedirect] at org.apache.commons.dbutils.QueryRunner.query(QueryRunner.java:)
// ::00.786 + ERROR [StdOutErrRedirect] at azkaban.executor.JdbcExecutorLoader.fetchActiveFlows(JdbcExecutorLoader.java:)
// ::00.786 + ERROR [StdOutErrRedirect] ... more
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>
浏览器http://yourip:8443/，依然打不开,推测可能是sql语句没有执行完
root@hadoop01:/software/azkaban-3.12./azkaban-sql>cat update.active_executing_flows.3.0.sql
ALTER TABLE active_executing_flows DROP COLUMN host;
ALTER TABLE active_executing_flows DROP COLUMN port;root@hadoop01:/software/azkaban-3.12./azkaban-sql>
root@hadoop01:/software/azkaban-3.12./azkaban-sql>cat update.execution_flows.3.0.sql
ALTER TABLE execution_flows ADD COLUMN executor_id INT DEFAULT NULL;
CREATE INDEX executor_id ON execution_flows(executor_id);root@hadoop01:/software/azkaban-3.12./azkaban-sql>
root@hadoop01:/software/azkaban-3.12./azkaban-sql>mysql -h10.10.10.10 -P3306 -uroot -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is
Server version: 5.7.-log Source distribution Copyright (c) , , Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> soure /software/azkaban-3.12./azkaban-sql/update.active_executing_flows.3.0.sql;
ERROR (): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'soure /software/azkaban-3.12.0/azkaban-sql/update.active_executing_flows.3.0.sql' at line
mysql> ALTER TABLE active_executing_flows DROP COLUMN host;
ERROR (3D000): No database selected
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| azkaban |
| mysql |
| performance_schema |
| sys |
| ums |
+--------------------+
rows in set (0.00 sec) mysql> use azkaban
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A Database changed
mysql> soure /software/azkaban-3.12./azkaban-sql/update.active_executing_flows.3.0.sql;
ERROR (): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'soure /software/azkaban-3.12.0/azkaban-sql/update.active_executing_flows.3.0.sql' at line
mysql> ALTER TABLE active_executing_flows DROP COLUMN host;
Query OK, rows affected (0.05 sec)
Records: Duplicates: Warnings: mysql> ALTER TABLE active_executing_flows DROP COLUMN port;
Query OK, rows affected (0.13 sec)
Records: Duplicates: Warnings: mysql> ALTER TABLE execution_flows ADD COLUMN executor_id INT DEFAULT NULL;
Query OK, rows affected (0.06 sec)
Records: Duplicates: Warnings: mysql> CREATE INDEX executor_id ON execution_flows(executor_id);
Query OK, rows affected (0.03 sec)
Records: Duplicates: Warnings: mysql> desc execution_flows
-> ;
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| exec_id | int() | NO | PRI | NULL | auto_increment |
| project_id | int() | NO | MUL | NULL | |
| version | int() | NO | | NULL | |
| flow_id | varchar() | NO | | NULL | |
| status | tinyint() | YES | | NULL | |
| submit_user | varchar() | YES | | NULL | |
| submit_time | bigint() | YES | | NULL | |
| update_time | bigint() | YES | | NULL | |
| start_time | bigint() | YES | MUL | NULL | |
| end_time | bigint() | YES | MUL | NULL | |
| enc_type | tinyint() | YES | | NULL | |
| flow_data | longblob | YES | | NULL | |
| executor_id | int() | YES | MUL | NULL | |
+-------------+--------------+------+-----+---------+----------------+
rows in set (0.00 sec) mysql> desc active_executing_flows
-> ;
+-------------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------+------+-----+---------+-------+
| exec_id | int() | NO | PRI | NULL | |
| update_time | bigint() | YES | | NULL | |
+-------------+------------+------+-----+---------+-------+
rows in set (0.00 sec) mysql>
停止进程后重启
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh azkaban-web-shutdown.sh
Killing Web Server. [pid: ]
done.
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>ps -ef|grep az
root : pts/ :: grep --color=auto az
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh start-web.sh
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>ps -ef|grep az
root : pts/ :: java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root : pts/ :: grep --color=auto az
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>
2.5.5报错：plugin path plugins/triggers doesn't exist!
查看日志，报错plugin path plugins/triggers doesn't exist!，并提示服务运行端口8081
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>cat webServerLog_2019--+::.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar
// ::36.006 + INFO [AzkabanWebServer] Starting Jetty Azkaban Web Server...
// ::36.020 + INFO [AzkabanServer] Loading azkaban settings file from ./../conf
// ::36.020 + INFO [AzkabanServer] Loading azkaban properties file
// ::36.022 + INFO [AzkabanWebServer] Setting up connector with stats on: true
// ::36.043 + INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
// ::36.091 + INFO [AzkabanWebServer] Loading user manager class azkaban.user.XmlUserManager
// ::36.119 + INFO [XmlUserManager] Loading user azkaban
// ::36.120 + INFO [XmlUserManager] Loading user metrics
// ::36.151 + INFO [ExecutorManager] Initializing local executor localhost:
// ::36.464 + INFO [AzkabanWebServer] Loading JDBC for project management
// ::36.465 + INFO [ExecutorManager] Cleaning old logs from execution_logs
// ::36.468 + INFO [ProjectManager] Project version retention is set to
// ::36.470 + WARN [XmlValidatorManager] Validator directory validators does not exist or is not a directory.
// ::36.471 + WARN [XmlValidatorManager] Azkaban properties file does not contain the key project.validators.xml.file
// ::36.515 + INFO [ExecutorManager] Cleaning old log files before --31T18::36.471+:
// ::36.523 + INFO [ExecutorManager] Cleaned up log entries.
// ::36.523 + INFO [ExecutorManager] log clean up time: seconds.
// ::36.550 + INFO [TriggerManager] TriggerManager loaded.
// ::36.550 + INFO [AzkabanWebServer] Loading built-in checker and action types
// ::36.554 + INFO [CheckerTypeLoader] Registering checker BasicTimeChecker
// ::36.554 + INFO [CheckerTypeLoader] Registering checker SlaChecker
// ::36.554 + INFO [CheckerTypeLoader] Registering checker ExecutionChecker
// ::36.554 + INFO [ActionTypeLoader] Registering action ExecuteFlowAction
// ::36.554 + INFO [ActionTypeLoader] Registering action KillExecutionAction
// ::36.554 + INFO [ActionTypeLoader] Registering action AlertAction
// ::36.554 + INFO [ActionTypeLoader] Registering action CreateTriggerAction
// ::36.554 + INFO [AzkabanWebServer] Loading trigger based scheduler
// ::36.556 + INFO [AzkabanWebServer] Loading plug-in checker and action types
// ::36.556 + ERROR [AzkabanWebServer] plugin path plugins/triggers doesn't exist!
// ::36.557 + INFO [AzkabanWebServer] Setting timezone to Asia/Shanghai
// ::36.557 + INFO [AzkabanWebServer] Registering MBeans...
// ::36.567 + INFO [AzkabanWebServer] Bean azkaban.jmx.JmxJettyServer registered.
// ::36.569 + INFO [AzkabanWebServer] Bean azkaban.jmx.JmxTriggerManager registered.
// ::36.571 + INFO [AzkabanWebServer] Bean azkaban.jmx.JmxExecutorManager registered.
// ::36.582 + INFO [AzkabanWebServer] Bean org.apache.log4j.jmx.HierarchyDynamicMBean registered.
// ::36.582 + INFO [StdOutErrRedirect] ************* loginLoggerObjName is null, make sure there is a logger with name azkaban.webapp.servlet.LoginAbstractAzkabanServlet
// ::36.584 + INFO [AzkabanWebServer] Setting up web resource dir web/
// ::36.636 + INFO [JdbcTriggerLoader] Loading all triggers from db.
// ::36.654 + INFO [JdbcTriggerLoader] Loaded triggers.
// ::36.654 + INFO [log] jetty-6.1.
// ::36.680 + INFO [ProjectManagerServlet] downloadBufferSize:
// ::36.768 + INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.ProjectManagerResource'
// ::36.788 + INFO [RestLiAnnotationReader] Processed actions resource 'azkaban.restli.UserManagerResource'
// ::36.804 + INFO [log] Started SocketConnector@0.0.0.0:
// ::36.804 + INFO [AzkabanWebServer] Server running on port .
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin/logs>
2.6浏览器登陆
浏览器http://ip:8081,
看到的页面奇丑无比，哈哈，解决方法如下
修改azkaban.properties文件中的web.resource.dir
同时修改以下项，在最后多加了个/
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/software/azkaban-3.12.0/azkaban-web-server/plugins/jobtypes/
重启服务，再重新在浏览器输入http://ip:8081, 登陆信息azkaban/azkaban
启动日志中仍然有ERROR [AzkabanWebServer] plugin path plugins/triggers doesn't exist!
2.7启动exec
root@hadoop01:/software/azkaban-3.12.>cd azkaban-exec-server
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server>ls
bin lib
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server>cp -r ../azkaban-web-server/conf .
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server>ls
bin conf lib
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server>cp -r ../azkaban-web-server/plugins .
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server>ls
bin conf lib plugins
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server>
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>mkdir logs
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>cat start-exec.sh
#!/bin/bash # pass along command line arguments to azkaban-executor-start.sh script
nohup sh azkaban-executor-start.sh "$@" >logs/executorServerLog__`date +%F+%T`.out 2>&1 & root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>sh start-exec.sh
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>ps -ef|grep az
root : ? :: java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root : pts/ :: java -Xmx3G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-exec-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-exec-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/guava-13.0..jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1..jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2..jar:./../lib/lz4-1.3..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/quartz-2.2..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.execapp.AzkabanExecutorServer -conf ./../conf
root : pts/ :: grep --color=auto az
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>cd logs
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin/logs>ls
executorServerLog__2019--+::.out executorServerLog__2019--+::.out
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin/logs>ll
总用量
-rw------- root root 4月 : executorServerLog__2019--+::.out
-rw------- root root 4月 : executorServerLog__2019--+::.out
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin/logs>cat executorServerLog__2019--+::.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-exec-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/guava-13.0..jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1..jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2..jar:./../lib/lz4-1.3..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/quartz-2.2..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar
Starting AzkabanExecutorServer on port ...
// ::42.828 + INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
// ::42.845 + INFO [AzkabanServer] Loading azkaban settings file from ./../conf
// ::42.845 + INFO [AzkabanServer] Loading azkaban properties file
// ::42.862 + INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
// ::42.885 + INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
// ::42.887 + INFO [AzkabanExecutorServer] Setting up connector with stats on: true
// ::42.889 + INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:, default header buffer size:
// ::42.889 + INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:, (if) new header buffer size:
// ::42.943 + INFO [FlowRunnerManager] Execution dir retention set to ms
// ::42.944 + INFO [FlowRunnerManager] useNewThreadPool: false
// ::42.946 + INFO [FlowRunnerManager] Cleaning recently finished
// ::42.946 + INFO [FlowRunnerManager] Cleaning old projects
// ::42.946 + INFO [FlowRunnerManager] Cleaning old execution dirs
// ::42.948 + INFO [JobTypeManager] Loading plugin default job types
// ::42.951 + INFO [JobTypeManager] Job type plugin directory set. Loading extra job types from /software/azkaban-3.12./azkaban-web-server/plugins/jobtypes
// ::42.951 + INFO [JobTypeManager] Common plugin job props file /software/azkaban-3.12./azkaban-web-server/plugins/jobtypes/common.properties not found. Using empty props.
// ::42.951 + INFO [JobTypeManager] Common plugin load props file /software/azkaban-3.12./azkaban-web-server/plugins/jobtypes/commonprivate.properties found. Attempt to load.
// ::42.952 + INFO [JmxJobMBeanManager] Initializing azkaban.execapp.jmx.JmxJobMBeanManager
// ::42.952 + INFO [AzkabanExecutorServer] Job callback enabled? true
// ::42.954 + INFO [JobCallbackManager] Initializing
// ::42.957 + INFO [JobCallbackRequestMaker] responseWaitTimeoutMS:
// ::42.958 + INFO [JobCallbackRequestMaker] Global request configuration , expectContinueEnabled=false, proxy=null, localAddress=null, staleConnectionCheckEnabled=true, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=, circularRedirectsAllowed=false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=, connectTimeout=, socketTimeout=]
// ::43.199 + INFO [JobCallbackRequestMaker] Jobcall thread pool size:
// ::43.201 + INFO [JobCallbackRequestMaker] Initialization for azkaban.execapp.event.JobCallbackRequestMaker is completed
// ::43.202 + INFO [JobCallbackManager] Initialization completed azkaban.execapp.event.JobCallbackManager
// ::43.202 + INFO [JobCallbackManager] azkabanHostName hadoop01
// ::43.202 + INFO [AzkabanExecutorServer] Registering MBeans...
// ::43.211 + INFO [AzkabanExecutorServer] Bean azkaban.jmx.JmxJettyServer registered.
// ::43.212 + INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
// ::43.214 + INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
// ::43.214 + INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobCallback registered.
// ::43.216 + INFO [SystemMemoryInfo] Current system free memory is kb (MemFree , Buffers , Cached , SwapCached )
// ::43.216 + INFO [SystemMemoryInfo] Scheduled thread to read /proc/meminfo every seconds
// ::43.218 + INFO [AzkabanExecutorServer] No value for property: jmx.attribute.processor.class was found
// ::43.218 + INFO [log] jetty-6.1.
// ::43.219 + INFO [SystemMemoryInfo] Current system free memory is kb (MemFree , Buffers , Cached , SwapCached )
// ::43.240 + INFO [log] Started SocketConnector@0.0.0.0:
// ::43.263 + INFO [AzkabanExecutorServer] Started Executor Server on hadoop01:
// ::13.218 + INFO [SystemMemoryInfo] Current system free memory is kb (MemFree , Buffers , Cached , SwapCached )
3.简单使用
详细请看这里：azkaban的简单使用
查看flow log
4.分布式部署（multiple-executor）
以上是two-server 方式运行，即web-service和exec-server运行在同一台机器上，且使用数据库。
分布式，即multiple-executor，即将exec-server运行在更多的节点上。
我们先关闭本机上的exec
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>sh azkaban-executor-s
azkaban-executor-shutdown.sh azkaban-executor-start.sh
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>sh azkaban-executor-shutdown.sh
cat: ./../executor.port: 没有那个文件或目录
Killing Executor. [pid: , port: ]
rm: 无法删除"./../executor.port": 没有那个文件或目录
done.
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>ps -ef|grep az
root : pts/ :: grep --color=auto az
root : ? :: java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root@hadoop01:/software/azkaban-3.12./azkaban-exec-server/bin>
在azkaban服务器(本机hadoop01)上配置好exec节点ip，这里cdh已经配置好了每台的hosts都配置成一样的，也可以
softwareapp@hadoop01:/software>cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
:: localhost localhost.localdomain localhost6 localhost6.localdomain6
10.10.10.70 hadoop01
10.10.10.71 hadoop02
10.10.10.72 hadoop03
10.10.10.84 hadoop04
10.10.10.85 hadoop05
拷贝exec到hadoop02和hadoop03节点
这里hadoop02，hadoop03无法直接sftp root,可以在hadoop01将目录压缩成tar包，用普通用户sftp user后再chown root后再tar -xvf即可，保留azkaban-exec,删除其它文件
hadoop02，hadoop03目录结构
root@hadoop03:/software/azkaban-3.12.>ls
azkaban-exec-server
root@hadoop03:/software/azkaban-3.12.>
hadoop01关闭web-server修改配置文件
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh azkaban-web-shutdown.sh
Killing Web Server. [pid: ]
done.
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>ps -ef|grep az
root : pts/ :: grep --color=auto az
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/conf> vi azkaban.properties ##新增下面内容
#启用multiple-executor模式
azkaban.use.multiple.executors=true
#在每次分发job时，先过滤出满足条件的executor，然后再做比较筛选
#如最小剩余内存,MinimumFreeMemory,过滤器会检查executor空余内存是否会大于6G，如果不足6G，则web-server不会将任务交由该executor执行。可参考Azkaban Github源码
#如CpuStatus，过滤器会检查executor的cpu占用率是否达到95%，若达到95%，web-server也不会将任务交给该executor执行。可参考Azkaban Github源码。
#参数含义参考官网说明http://azkaban.github.io/azkaban/docs/latest/#configuration
#azkaban.executorselector.filters=StaticRemainingFlowSize,MinimumFreeMemory,CpuStatus
# 由于是虚拟机，不需要过滤，只需要比较即可
# 某个任务是否指定了executor id
azkaban.executorselector.comparator.NumberOfAssignedFlowComparator=1
# 是否比较内存
azkaban.executorselector.comparator.Memory=1
# 是否最后一次被分发
azkaban.executorselector.comparator.LastDispatched=1
# 是否比较CPU
azkaban.executorselector.comparator.CpuUsage=1
在mysql中增加executors表中添加executor
主机上再次验证
mysql> use azkaban
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A Database changed
mysql> show tables;
+------------------------+
| Tables_in_azkaban |
+------------------------+
| active_executing_flows |
| active_sla |
| execution_flows |
| execution_jobs |
| execution_logs |
| executor_events |
| executors |
| project_events |
| project_files |
| project_flows |
| project_permissions |
| project_properties |
| project_versions |
| projects |
| properties |
| schedules |
| triggers |
+------------------------+
rows in set (0.00 sec) mysql> select * from executors
-> ;
+----+-----------+-------+--------+
| id | host | port | active |
+----+-----------+-------+--------+
| | localhost | | |
| | hadoop01 | | |
| | hadoop02 | | |
| | hadoop03 | | |
+----+-----------+-------+--------+
rows in set (0.00 sec) mysql>
然后可以启动程序了,顺序为web-server -> hadoop02 exec-server -> hadoop03 exec-server
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>sh start-web.sh
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>ps -ef|grep az
root : pts/ :: java -Xmx4G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-web-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/antlr-2.7..jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-web-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/cglib-nodep-2.2.jar:./../lib/commons-beanutils-1.7..jar:./../lib/commons-chain-1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-compress-1.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-digester-1.8.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/commons-validator-1.3..jar:./../lib/data-1.15..jar:./../lib/data-transform-1.15..jar:./../lib/dom4j-1.1.jar:./../lib/guava-13.0..jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-2.2..jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/javax.inject-.jar:./../lib/javax.servlet-api-3.0..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-.jar:./../lib/li-jersey-uri-1.15..jar:./../lib/log4j-1.2..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mina-core-1.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/netty-3.2..Final.jar:./../lib/oro-2.0..jar:./../lib/parseq-1.3..jar:./../lib/pegasus-common-1.15..jar:./../lib/quartz-2.2..jar:./../lib/r2-1.15..jar:./../lib/restli-common-1.15..jar:./../lib/restli-server-1.15..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-0.3.jar:./../lib/sslext-1.2-.jar:./../lib/struts-core-1.3..jar:./../lib/struts-taglib-1.3..jar:./../lib/struts-tiles-1.3..jar:./../lib/velocity-1.7.jar:./../lib/velocity-tools-2.0.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.webapp.AzkabanWebServer -conf ./../conf ./..
root : pts/ :: grep --color=auto az
root@hadoop01:/software/azkaban-3.12./azkaban-web-server/bin>
root@hadoop02:/software/azkaban-3.12./azkaban-exec-server/bin>sh start-exec.sh
root@hadoop02:/software/azkaban-3.12./azkaban-exec-server/bin>ps -ef|grep az
root : pts/ :: java -Xmx3G -Dlog4j.configuration=file:./../conf/log4j.properties -server -Dcom.sun.management.jmxremote -Djava.io.tmpdir=/tmp -Dexecutorport= -Dserverpath=/software/azkaban-3.12./azkaban-exec-server/bin -Dlog4j.log.dir=./../logs -cp .:/usr/local/jdk/jdk1..0_181/lib:/usr/local/jdk/jdk1..0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12..jar:./../lib/azkaban-exec-server-3.12..jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2..jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2..jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1..jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1..jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5..jar:./../lib/guava-13.0..jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3..jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9..jar:./../lib/jackson-mapper-asl-1.9..jar:./../lib/jetty-6.1..jar:./../lib/jetty-util-6.1..jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1..jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2..jar:./../lib/lz4-1.3..jar:./../lib/mail-1.4..jar:./../lib/metrics-core-3.1..jar:./../lib/metrics-jvm-3.1..jar:./../lib/mysql-connector-java-5.1..jar:./../lib/quartz-2.2..jar:./../lib/servlet-api-2.5-.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7..jar:./../lib/slf4j-log4j12-1.7..jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar azkaban.execapp.AzkabanExecutorServer -conf ./../conf
root : pts/ :: grep --color=auto az
root@hadoop02:/software/azkaban-3.12./azkaban-exec-server/bin>
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin/logs>ll
总用量 144
-rw------- 1 root root 79 4月 26 10:16 executorServerLog__2019-04-26+10:16:33.out
-rw------- 1 root root 135628 4月 26 15:12 executorServerLog__2019-04-26+10:18:42.out
-rw------- 1 root root 4031 4月 26 19:06 executorServerLog__2019-04-26+19:06:54.out
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin/logs>cat executorServerLog__2019-04-26+19:06:54.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar
Starting AzkabanExecutorServer on port 12321 ...
2019/04/26 19:06:54.827 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/04/26 19:06:54.841 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/26 19:06:54.842 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/26 19:06:54.862 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/04/26 19:06:54.883 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/26 19:06:54.885 +0800 INFO [AzkabanExecutorServer] Setting up connector with stats on: true
2019/04/26 19:06:54.887 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/04/26 19:06:54.887 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/04/26 19:06:54.944 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/04/26 19:06:54.945 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/04/26 19:06:54.949 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/04/26 19:06:54.949 +0800 ERROR [StdOutErrRedirect] Exception in thread "main"
2019/04/26 19:06:54.949 +0800 INFO [FlowRunnerManager] Cleaning old projects
2019/04/26 19:06:54.949 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/04/26 19:06:54.949 +0800 ERROR [StdOutErrRedirect] java.io.FileNotFoundException: /software/azkaban-3.12.0/azkaban-web-server/conf/global.properties (没有那个文件或目录)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at java.io.FileInputStream.open0(Native Method)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at java.io.FileInputStream.open(FileInputStream.java:195)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at java.io.FileInputStream.<init>(FileInputStream.java:138)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.utils.Props.<init>(Props.java:89)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.utils.Props.<init>(Props.java:75)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.execapp.FlowRunnerManager.<init>(FlowRunnerManager.java:203)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.execapp.AzkabanExecutorServer.<init>(AzkabanExecutorServer.java:119)
2019/04/26 19:06:54.950 +0800 ERROR [StdOutErrRedirect] at azkaban.execapp.AzkabanExecutorServer.main(AzkabanExecutorServer.java:382)
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin/logs>ls
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/conf>vi azkaban.properties
...
# Loader for projects
executor.global.properties=/software/azkaban-3.12.0/azkaban-exec-server/conf/global.properties
azkaban.project.dir=projects
....
# Azkaban plugin settings
azkaban.jobtype.plugin.dir=/software/azkaban-3.12.0/azkaban-exec-server/plugins/jobtypes/
...
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin>sh azkaban-executor-shutdown.sh
cat: ./../executor.port: 没有那个文件或目录
Killing Executor. [pid: 69178, port: ]
rm: 无法删除"./../executor.port": 没有那个文件或目录
done.
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin>sh start-exec.sh
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin>cd ../logs
-bash: cd: ../logs: 没有那个文件或目录
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin>ls
azkaban-executor-shutdown.sh azkaban-executor-start.sh executions executor.port logs projects start-exec.sh temp
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin>cd logs
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin/logs>ls
executorServerLog__2019-04-26+10:16:33.out executorServerLog__2019-04-26+10:18:42.out executorServerLog__2019-04-26+19:06:54.out executorServerLog__2019-04-26+19:11:25.out
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin/logs>ll
总用量 156
-rw------- 1 root root 79 4月 26 10:16 executorServerLog__2019-04-26+10:16:33.out
-rw------- 1 root root 135628 4月 26 15:12 executorServerLog__2019-04-26+10:18:42.out
-rw------- 1 root root 4195 4月 26 19:10 executorServerLog__2019-04-26+19:06:54.out
-rw------- 1 root root 5479 4月 26 19:11 executorServerLog__2019-04-26+19:11:25.out
root@hadoop03:/software/azkaban-3.12.0/azkaban-exec-server/bin/logs>cat executorServerLog__2019-04-26+19:11:25.out
Error: HADOOP_HOME is not set. Hadoop job types will not run properly.
./..
.:/usr/local/jdk/jdk1.8.0_181/lib:/usr/local/jdk/jdk1.8.0_181/jre/lib::./../lib/activation-1.1.jar:./../lib/azkaban-common-3.12.0.jar:./../lib/azkaban-exec-server-3.12.0.jar:./../lib/c3p0-0.9.1.1.jar:./../lib/commons-codec-1.6.jar:./../lib/commons-collections-3.2.2.jar:./../lib/commons-dbcp-1.4.jar:./../lib/commons-dbutils-1.5.jar:./../lib/commons-fileupload-1.2.1.jar:./../lib/commons-io-2.4.jar:./../lib/commons-jexl-2.1.1.jar:./../lib/commons-lang-2.6.jar:./../lib/commons-logging-1.1.3.jar:./../lib/commons-math3-3.0.jar:./../lib/commons-pool-1.5.4.jar:./../lib/guava-13.0.1.jar:./../lib/hamcrest-core-1.1.jar:./../lib/httpclient-4.3.1.jar:./../lib/httpcore-4.3.jar:./../lib/jackson-core-asl-1.9.5.jar:./../lib/jackson-mapper-asl-1.9.5.jar:./../lib/jetty-6.1.26.jar:./../lib/jetty-util-6.1.26.jar:./../lib/joda-time-2.0.jar:./../lib/jopt-simple-4.3.jar:./../lib/json-simple-1.1.1.jar:./../lib/junit-4.10.jar:./../lib/kafka-clients-0.10.0.0.jar:./../lib/kafka-log4j-appender-0.10.0.0.jar:./../lib/log4j-1.2.17.jar:./../lib/lz4-1.3.0.jar:./../lib/mail-1.4.5.jar:./../lib/metrics-core-3.1.0.jar:./../lib/metrics-jvm-3.1.0.jar:./../lib/mysql-connector-java-5.1.28.jar:./../lib/quartz-2.2.1.jar:./../lib/servlet-api-2.5-20081211.jar:./../lib/servlet-api-2.5.jar:./../lib/slf4j-api-1.7.21.jar:./../lib/slf4j-log4j12-1.7.21.jar:./../lib/snappy-java-1.1.2.4.jar:./../lib/velocity-1.7.jar:./../extlib/*.jar:./../plugins/*/*.jar
Starting AzkabanExecutorServer on port 12321 ...
2019/04/26 19:11:25.928 +0800 INFO [AzkabanExecutorServer] Starting Jetty Azkaban Executor...
2019/04/26 19:11:25.943 +0800 INFO [AzkabanServer] Loading azkaban settings file from ./../conf
2019/04/26 19:11:25.943 +0800 INFO [AzkabanServer] Loading azkaban properties file
2019/04/26 19:11:25.961 +0800 INFO [AzkabanExecutorServer] Setting timezone to Asia/Shanghai
2019/04/26 19:11:25.984 +0800 INFO [log] Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog
2019/04/26 19:11:25.985 +0800 INFO [AzkabanExecutorServer] Setting up connector with stats on: true
2019/04/26 19:11:25.987 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, default header buffer size: 4096
2019/04/26 19:11:25.987 +0800 INFO [AzkabanExecutorServer] Jetty connector name: 0.0.0.0:12321, (if) new header buffer size: 4096
2019/04/26 19:11:26.039 +0800 INFO [FlowRunnerManager] Execution dir retention set to 86400000 ms
2019/04/26 19:11:26.040 +0800 INFO [FlowRunnerManager] useNewThreadPool: false
2019/04/26 19:11:26.042 +0800 INFO [FlowRunnerManager] Cleaning recently finished
2019/04/26 19:11:26.043 +0800 INFO [FlowRunnerManager] Cleaning old projects
2019/04/26 19:11:26.043 +0800 INFO [FlowRunnerManager] Cleaning old execution dirs
2019/04/26 19:11:26.044 +0800 INFO [JobTypeManager] Loading plugin default job types
2019/04/26 19:11:26.047 +0800 INFO [JmxJobMBeanManager] Initializing azkaban.execapp.jmx.JmxJobMBeanManager
2019/04/26 19:11:26.047 +0800 INFO [AzkabanExecutorServer] Job callback enabled? true
2019/04/26 19:11:26.049 +0800 INFO [JobCallbackManager] Initializing
2019/04/26 19:11:26.052 +0800 INFO [JobCallbackRequestMaker] responseWaitTimeoutMS: 5000
2019/04/26 19:11:26.053 +0800 INFO [JobCallbackRequestMaker] Global request configuration , expectContinueEnabled=false, proxy=null, localAddress=null, staleConnectionCheckEnabled=true, cookieSpec=null, redirectsEnabled=true, relativeRedirectsAllowed=true, maxRedirects=50, circularRedirectsAllowed=false, authenticationEnabled=true, targetPreferredAuthSchemes=null, proxyPreferredAuthSchemes=null, connectionRequestTimeout=3000, connectTimeout=3000, socketTimeout=3000]
2019/04/26 19:11:26.261 +0800 INFO [JobCallbackRequestMaker] Jobcall thread pool size: 10
2019/04/26 19:11:26.262 +0800 INFO [JobCallbackRequestMaker] Initialization for azkaban.execapp.event.JobCallbackRequestMaker is completed
2019/04/26 19:11:26.262 +0800 INFO [JobCallbackManager] Initialization completed azkaban.execapp.event.JobCallbackManager
2019/04/26 19:11:26.263 +0800 INFO [JobCallbackManager] azkabanHostName hadoop03
2019/04/26 19:11:26.263 +0800 INFO [AzkabanExecutorServer] Registering MBeans...
2019/04/26 19:11:26.271 +0800 INFO [AzkabanExecutorServer] Bean azkaban.jmx.JmxJettyServer registered.
2019/04/26 19:11:26.272 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxFlowRunnerManager registered.
2019/04/26 19:11:26.273 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobMBeanManager registered.
2019/04/26 19:11:26.274 +0800 INFO [AzkabanExecutorServer] Bean azkaban.execapp.jmx.JmxJobCallback registered.
2019/04/26 19:11:26.275 +0800 INFO [SystemMemoryInfo] Current system free memory is 29289720 kb (MemFree 417156, Buffers 178268, Cached 28694296, SwapCached 0)
2019/04/26 19:11:26.276 +0800 INFO [SystemMemoryInfo] Scheduled thread to read /proc/meminfo every 30 seconds
2019/04/26 19:11:26.277 +0800 INFO [AzkabanExecutorServer] No value for property: jmx.attribute.processor.class was found
2019/04/26 19:11:26.277 +0800 INFO [log] jetty-6.1.26
2019/04/26 19:11:26.278 +0800 INFO [SystemMemoryInfo] Current system free memory is 29289720 kb (MemFree 417156, Buffers 178268, Cached 28694296, SwapCached 0)
2019/04/26 19:11:26.305 +0800 INFO [log] Started SocketConnector@0.0.0.0:12321
2019/04/26 19:11:26.332 +0800 INFO [AzkabanExecutorServer] Started Executor Server on hadoop03:12321
报错：/software/azkaban-3.12.0/azkaban-web-server/conf/global.properties (没有那个文件或目录)，将配置文件中该项改为/software/azkaban-3.12.0/azkaban-exec-server/conf/global.properties
重启即可
单击execution id为23，发现在hadoop03上执行了
至此，分布式azkaban安装完毕。