Connect to the Spark shell

Spark provides a shell, which allows you to run Spark interactively. You can access the Spark shell with the following command:

$ spark-shell

After some seconds, you will see the prompt:

scala>

The Bitnami Hadoop Stack includes Spark, a fast and general-purpose cluster computing system. Spark includes several example programs.

In order to run Spark examples, you must use the run-example program. In order to estimate a value for Pi, you can run the following test:

$ run-example SparkPi 10

After some seconds, Spark will output the result:

2018-02-15 12:37:29,410 INFO scheduler.DAGScheduler: Job 0 finished: reduce at SparkPi.scala:38, took 1.697757 s
Pi is roughly 3.1426351426351427