apache-spark Shared Variables Broadcast variables

Help us to keep this website almost Ad Free! It takes only 10 seconds of your time:
> Step 1: Go view our video on YouTube: EF Core Bulk Insert
> Step 2: And Like the video. BONUS: You can also share it!

Example

Broadcast variables are read only shared objects which can be created with SparkContext.broadcast method:

val broadcastVariable = sc.broadcast(Array(1, 2, 3))

and read using value method:

val someRDD = sc.parallelize(Array(1, 2, 3, 4))

someRDD.map(
    i => broadcastVariable.value.apply(i % broadcastVariable.value.size)
)


Got any apache-spark Question?