解决sqoop 导入oracle表时 --split-by参数为日期类型时的报错:ORA-01861: literal does not match format string

时间:2022-10-15 04:09:05

报错栈:

-- ::, INFO [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Executing query: select "JFRQ","ZYH","FYKS","KSSE","YBJE","YPJE","ZJJE" from BSHIS."DEPCS_T_DEPCS_BRMXCOST_DAY" tbl where ( JFRQ >= '2009-01-01 00:00:00.0' ) AND ( JFRQ < '2009-10-13 04:00:00.0' )
-- ::, ERROR [main] org.apache.sqoop.mapreduce.db.DBRecordReader: Top level exception:
java.sql.SQLDataException: ORA-: literal does not match format string
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:)
at org.apache.hadoop.mapred.YarnChild$.run(YarnChild.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:)
-- ::, INFO [Thread-] org.apache.sqoop.mapreduce.AutoProgressMapper: Auto-progress thread is finished. keepGoing=false
-- ::, WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.io.IOException: SQLException in nextKeyValue
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:)
at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:)
at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:)
at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:)
at org.apache.sqoop.mapreduce.AutoProgressMapper.run(AutoProgressMapper.java:)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:)
at org.apache.hadoop.mapred.YarnChild$.run(YarnChild.java:)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:)
Caused by: java.sql.SQLDataException: ORA-: literal does not match format string
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:)
at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:)
at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:)
at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.executeQuery(DBRecordReader.java:)
at org.apache.sqoop.mapreduce.db.DBRecordReader.nextKeyValue(DBRecordReader.java:)
... more

报错为sql抛出,sql改写后执行成功,下面两句都可以:

解决sqoop 导入oracle表时 --split-by参数为日期类型时的报错:ORA-01861: literal does not match format string

sqoop的一个bug: https://issues.apache.org/jira/browse/SQOOP-1946, 解决方案大致为转换成string类型即可。发现这个列的类型为date, 因此解决方案为: --split-by "to_char(我的日期列,'yyyy-mm-dd hh24:mi:ss')"。
参考:https://my.oschina.net/pomo/blog/725060