Reading and writing Amazon S3 files from Apache Spark

The S3 Native Filesystem client present in Apache Spark running over Apache Hadoop allows access to the Amazon S3 service from a Apache Spark application. So it is enough to define the S3 Access Key and the S3 Secret Access Key in the Spark Context as shown below:

Continue reading “Reading and writing Amazon S3 files from Apache Spark”