apache-spark Text files and operations in Scala Example usage


Example

Read text file from path:

val sc: org.apache.spark.SparkContext = ???
sc.textFile(path="/path/to/input/file") 

Read files using wildcards:

sc.textFile(path="/path/to/*/*") 

Read files specifying minimum number of partitions:

sc.textFile(path="/path/to/input/file", minPartitions=3)