spark-sql跑数据Failed with exception java.io.IOException:org.apache.parquet.io.ParquetDecodingExceptio

时间:2024-04-06 15:22:07

错误信息:

Failed with exception java.io.IOException:org.apache.parquet.io.ParquetDecodingException: Can not read value at 0 in block -1 in file oss:/xxxxxxxxxx.snappy.parquet

spark-sql跑数据Failed with exception java.io.IOException:org.apache.parquet.io.ParquetDecodingExceptio

修改方式:

在运行spark-sql前 添加这样一句话:

spark.conf.set("spark.sql.parquet.writeLegacyFormat","true")

这样就能跑成了。

spark-sql跑数据Failed with exception java.io.IOException:org.apache.parquet.io.ParquetDecodingExceptio