Saved Bookmarks
| 1. |
Why Is There A Need For Broadcast Variables When Working With Apache Spark? |
|
Answer» These are read only VARIABLES, present in-memory cache on every machine. When WORKING with Spark, usage of broadcast variables eliminates the necessity to ship copies of a variable for every TASK, so DATA can be processed faster. Broadcast variables help in storing a lookup table inside the memory which enhances the RETRIEVAL efficiency when compared to an RDD lookup (). These are read only variables, present in-memory cache on every machine. When working with Spark, usage of broadcast variables eliminates the necessity to ship copies of a variable for every task, so data can be processed faster. Broadcast variables help in storing a lookup table inside the memory which enhances the retrieval efficiency when compared to an RDD lookup (). |
|