apache-spark Migrating from Spark 1.6 to Spark 2.0 Update build.sbt file


Example

Update build.sbt with :

scalaVersion := "2.11.8" // Make sure to have installed Scala 11
sparkVersion := "2.0.0"  // Make sure to have installed Spark 2.0

Note that when compiling with sbt package, the .jar will now be created in target/scala-2.11/, and the .jar name will also be changed, so the spark-submit command need to be updated as well.