Scala Language User Defined Functions for Hive A simple Hive UDF within Apache Spark

Help us to keep this website almost Ad Free! It takes only 10 seconds of your time:
> Step 1: Go view our video on YouTube: EF Core Bulk Insert
> Step 2: And Like the video. BONUS: You can also share it!

Example

import org.apache.spark.sql.functions._

// Create a function that uses the content of the column inside the dataframe
val code = (param: String) => if (param == "myCode") 1 else 0
// With that function, create the udf function
val myUDF = udf(code)
// Apply the udf to a column inside the existing dataframe, creating a dataframe with the additional new column
val newDataframe = aDataframe.withColumn("new_column_name", myUDF(col(inputColumn)))


Got any Scala Language Question?