There are 2 deploy modes in Spark. They are:

• Client Mode: The deploy mode is said to be in client mode when the spark driver component runs on the machine node from where the spark job is submitted.
  • The main disadvantage of this mode is if the machine node fails, then the entire job fails.
  • This mode supports both interactive shells or the job submission commands.
  • The performance of this mode is worst and is not preferred in production environments.
• CLUSTER Mode: If the spark job driver component does not run on the machine from which the spark job has been submitted, then the deploy mode is said to be in cluster mode.
  • The spark job launches the driver component within the cluster as a part of the sub-process of ApplicationMaster.
  • This mode supports deployment only using the spark-submit command (interactive shell mode is not SUPPORTED).
  • Here, since the driver programs are run in ApplicationMaster, in case the program fails, the driver program is re-instantiated.
  • In this mode, there is a dedicated cluster manager (such as stand-alone, YARN, Apache Mesos, Kubernetes, etc) for allocating the resources required for the job to run as shown in the below architecture.

Apart from the above two modes, if we have to run the application on our local machines for unit TESTING and development, the deployment mode is called “Local Mode”. Here, the JOBS run on a single JVM in a single machine which makes it highly inefficient as at some point or the other there would be a shortage of resources which results in the failure of jobs. It is also not possible to scale up resources in this mode DUE to the restricted memory and space.