What is an executor in spark and how its support to perform the operat

1.	What is an executor in spark and how its support to perform the operation on volume of data?
Answer» Executors are WORKER nodes' processes in charge of running individual tasks when Spark job get submitted. They are launched at the beginning of a Spark application and typically run for the entire LIFETIME of an application. Once they have run the task, they send the results to the DRIVER. They also provide in-memory storage for RDDs that are cached by USER programs through Block Manager. Below are the key points on executors: Every spark application has its own executor process. Executor performs all the data processing. Reads from and Writes data to external sources. Executor stores the computation results data in-memory, cache or on HARD disk drives. Executor also work as a distributed agent responsible for the execution of tasks. When the job getting launched, spark trigger the executor, which act as a worker node which responsible for running individual task, which is assigned by spark driver.

What is an executor in spark and how its support to perform the operation on volume of data?

Answer»

Executors are WORKER nodes' processes in charge of running individual tasks when Spark job get submitted. They are launched at the beginning of a Spark application and typically run for the entire LIFETIME of an application. Once they have run the task, they send the results to the DRIVER. They also provide in-memory storage for RDDs that are cached by USER programs through Block Manager.

Below are the key points on executors:

Every spark application has its own executor process.
Executor performs all the data processing.
Reads from and Writes data to external sources.
Executor stores the computation results data in-memory, cache or on HARD disk drives.

Executor also work as a distributed agent responsible for the execution of tasks. When the job getting launched, spark trigger the executor, which act as a worker node which responsible for running individual task, which is assigned by spark driver.

What is an executor in spark and how its support to perform the operation on volume of data?

Discussion

No Comment Found

Related InterviewSolutions

Reply to Comment