hive 中出现struct 结构化的问题

时间:2020-11-27 05:33:37

如果你使用udf,udaf,udtf中的某一个并且查询日志中出现如下之类的struct错误

java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:104)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:74)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:370)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
at org.apache.hadoop.mapred.Child.main(Child.java:165)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:99)
... 6 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:104)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:74)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 11 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:99)
... 15 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:130)
... 20 more
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Error: name expected at the position 67 of 'struct,mapl:map>,int>>,mapr:map>,int>>,sum:int>' but '>' is found.
at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initializeOp(ReduceSinkOperator.java:129)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:427)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:383)
at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:368)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:427)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:383)
at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:427)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:383)
at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:368)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:358)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:94)

那么请注意一下 编写的函数中是否如果是复合型数据结构,那么一定不要出现Character