InterviewSolution
Saved Bookmarks
| 1. |
What is Spark Driver and what are the roles and responsibility perform after the job submission? |
|
Answer» Spark driver prepares the context and declares the operations on the DATA using RDD transformations and actions. Driver submits the serialized RDD graph to the master, where master creates tasks out of it and submits them to the workers for execution. Executor is a distributed agent responsible for the execution of tasks. Below is the key point for the reference:
Spark driver coordinates the different job stages, where the tasks are actually executed. They should have the resources and network connectivity required to execute the operations requested on the RDDs. |
|