Connect to the Spark shell
Spark provides a shell, which allows you to run Spark interactively. You can access the Spark shell with the following command:
After some seconds, you will see the prompt:
The Bitnami Hadoop Stack includes Spark, a fast and general-purpose cluster computing system. Spark includes several example programs.
In order to run Spark examples, you must use the run-example program. In order to estimate a value for Pi, you can run the following test:
$ run-example SparkPi 10
After some seconds, Spark will output the result:
2018-02-15 12:37:29,410 INFO scheduler.DAGScheduler: Job 0 finished: reduce at SparkPi.scala:38, took 1.697757 s Pi is roughly 3.1426351426351427