Jupyter Notebook Setup

In this guide you will see how to setup jupyter notebook as a default editor for PySpark

After completing the Spark installation in order to set jupyter notebook as the default editor we need to the following steps:

Step 1 (install jupyter notebook)

Open CMD and use the following command to install jupyter notebook

pip install jupyter

Step 2 (set environment variables)

create the following environment variables:

PYSPARK_PYTHON = {path of your python.exe}

PYSPARK_DRIVER_PYTHON = jupyter

PYSPARK_DRIVER_PYTHON_OPTS = 'notebook'

That's it!

You can verify if your setup was successful by typing the following command on cmd

pyspark

This should open jupyter notebook in your default browser.