《OD大数据实战》Sqoop入门实例

时间：2023-03-09 07:34:08
《OD大数据实战》Sqoop入门实例

官网地址：

http://archive.cloudera.com/cdh5/cdh/5/sqoop-1.4.5-cdh5.3.6/SqoopUserGuide.html

一、环境搭建

1. 下载

sqoop-1.4.5-cdh5.3.6.tar.gz

2. 解压

tar -zxvf sqoop-1.4.5-cdh5.3.6.tar.gz -C /opt/modules/cdh/

3. 修改配置

1）修改sqoop-env.sh

#Set path to where bin/hadoop is available

export HADOOP_COMMON_HOME=/opt/modules/cdh/hadoop-2.5.-cdh5.3.6

#Set path to where hadoop-*-core.jar is available

export HADOOP_MAPRED_HOME=/opt/modules/cdh/hadoop-2.5.-cdh5.3.6

#set the path to where bin/hbase is available

export HBASE_HOME=/opt/modules/cdh/hbase-0.98.-cdh5.3.6

#Set the path to where bin/hive is available

export HIVE_HOME=/opt/modules/cdh/hive-0.13.-cdh5.3.6

2）修改sqoop-site.xml

4. copy 依赖的jar包

cd /opt/modules/cdh/sqoop-1.4.-cdh5.3.6/lib/

1）copy mysql jar包

cp /opt/software/mysql-connector-java-5.1.-bin.jar /opt/modules/cdh/sqoop-1.4.-cdh5.3.6/lib ./

2）copy hadoop jar包

cp /opt/modules/cdh/hadoop-2.5.-cdh5.3.6/share/hadoop/common/hadoop-common-2.5.-cdh5.3.6.jar ./

cp /opt/modules/cdh/hadoop-2.5.-cdh5.3.6/share/hadoop/hdfs/hadoop-hdfs-2.5.-cdh5.3.6.jar ./

cp /opt/modules/cdh/hadoop-2.5.-cdh5.3.6/share/hadoop/mapreduce/hadoop-mapreduce-client-core-2.5.-cdh5.3.6.jar ./

5. 检验是否安装成功

sqoop version

6. 去除警告信息

vim bin/vim configure-sqoop

## Moved to be a runtime check in sqoop.

#if [ ! -d "${HCAT_HOME}" ]; then

  #echo "Warning: $HCAT_HOME does not exist! HCatalog jobs will fail."

  #echo 'Please set $HCAT_HOME to the root of your HCatalog installation.'

#fi

#if [ ! -d "${ACCUMULO_HOME}" ]; then

  #echo "Warning: $ACCUMULO_HOME does not exist! Accumulo imports will fail."

  #echo 'Please set $ACCUMULO_HOME to the root of your Accumulo installation.'

#fi

#if [ ! -d "${ZOOKEEPER_HOME}" ]; then

  #echo "Warning: $ZOOKEEPER_HOME does not exist! Accumulo imports will fail."

  #echo 'Please set $ZOOKEEPER_HOME to the root of your Zookeeper installation.'

#fi

二、Sqoop命令介绍

1、 Sqoop总共有14个命令

相关文章