apache-spark Migrating from Spark 1.6 to Spark 2.0 Update ML Vector libraries


ML Transformers now generates org.apache.spark.ml.linalg.VectorUDT instead of org.apache.spark.mllib.linalg.VectorUDT.

They are also mapped locally to subclasses of org.apache.spark.ml.linalg.Vector. These are not compatible with old MLLib API which is moving towards deprecation in Spark 2.0.0.

//import org.apache.spark.mllib.linalg.{Vector, Vectors} // Depreciated in Spark 2.0 
import org.apache.spark.ml.linalg.Vector // Use instead