一、进入IDE界面
cd ~/Downloads/idea/bin
idea.sh
二、建立scala项目
Step 1：导入Spark-hadoop对应的包，次选择“File”–> “Project Structure” –> “Libraries”，选择“+”，将spark-hadoop 对应的包导入:
点击“OK”确认：
点击“OK”:
IDEA工作完成后会发现Spark的jar包导入到了我们的工程中：
第二步、写scala代码实现wordcount
import SparkContext
import SparkContext._
object Wordcount {
def main(args: Array[String]) {
val sc = new SparkContext("local", "Simple App", "home\\yuanqin\\Downloads\\spark-0.9.0-incubating-bin-hadoop1",null)
//List("lib/spark-assembly_2.10-0.9.0-incubating-hadoop1.0.4.jar")
val logFile = sc.textFile("hdfs://Master:9000/user/root/input/file.txt") // Should be some file on your system
val count=logFile.flatMap(line => line.split(" ")).map(word => (word,1)).reduceByKey(_+_)
println(count)
count.saveAsTextFile("home\\yuanqin\\output\\2014")
}
}
第三步、右击run as....
