File format support by Spark? How the lazy evaluation works in Spark?

1.	File format support by Spark? How the lazy evaluation works in Spark?
Answer» Parquet file. JSON Hive Cassandra, Mongo-DB Text file CSV file My-SQL Lazy evaluation in spark works to instantiate the variable when its really required for INSTANCE like when spark do the transformation, till this time it’s not computed, however when we APPLIED the action than its compute the DATA. Spark delays its evaluation till it is NECESSARY. Eg: lazy val lazydata = 10

Answer»

Parquet file.
JSON
Hive
Cassandra, Mongo-DB
Text file
CSV file
My-SQL

Lazy evaluation in spark works to instantiate the variable when its really required for INSTANCE like when spark do the transformation, till this time it’s not computed, however when we APPLIED the action than its compute the DATA. Spark delays its evaluation till it is NECESSARY.

Eg: lazy val lazydata = 10

Discussion

No Comment Found

Related InterviewSolutions

When submitting a Spark job, I notice that a few tasks are relatively taking longer time to get completed. What could be the cause of this and how to resolve this issue?
What is a broadcast variable in Spark? What purpose does it serve? How is it different from an accumulator?
Describe the Spark Memory model. What is the difference between Off-heap and On-heap memory?
What is checkpointing in Spark? How does it help Spark achieve exactly-once semantics? How does Checkpointing differ from Persistence?
What is Structured Streaming in Spark? What are the different output modes in Spark Structured Streaming?
List some of the use cases for Graph Analytics?
What is Graph Analytics in Spark?
List some of the use cases of Unsupervised Learning.
List some of the use cases of Supervised/Classification Algorithm.
What are the different types of Machine Learning?

File format support by Spark? How the lazy evaluation works in Spark?

Discussion

No Comment Found

Related InterviewSolutions

Reply to Comment