1.

What is Lambda Architecture?

Answer»

LAMBDA architecture is a Big Data processing architecture. To handle the enormous quantities of data, the lambda architecture MAKES use of batch as well as stream processing methods. It is a fault-tolerant architecture and achieves a balance between LATENCY and throughput. Lambda architecture makes use of the model of data that has an append-only, immutable data source which serves as a system of record.

In this architecture, new events are appended to the existing events. The new events do not overwrite existing events. The lambda architecture is designed for ingesting and the processing of timestamp-based events. The STATE can be determined from the 'natural', 'time-based' ordering of the data.

In Lambda architecture, we have a system that consists of three layers:

  1. Batch processing
  2. Real-time processing
  3. Serving layer

The third layer is to respond to queries. The data is ingested to the processing layers from a master copy of the entire data set. This master copy is immutable.  The real-time processing layer processes the data streams in real-time. It does not require COMPLETENESS or any fix-ups.

This layer provides real-time views on the most recent data. So the latency is minimized but the throughput is sacrificed.   The real-time processing is also termed as speed processing.

As there is a lag by the batch layer in providing the views on the most recent data, we can say that the speed layer does the work of filling this gap. The benefit which we get from the speed layer is that the view is immediately available once we receive the data. This view may not be complete or we can say when compared with the view generated by the batch layer. However, there is always a choice with you to replace the view produced by the speed layer with the batch layer's view when that data made available to the batch layer. The output obtained from the batch layer and the speed layer is stored in the serving layer.  In response to the 'ad-hoc queries', this serving layer returns the views that are pre-computed or building the views from processed data.



Discussion

No Comment Found