Explore topic-wise InterviewSolutions in Current Affairs.

By Doug CUTTING, the father of HADOOP, APACHE AVRO was developed.

By Doug Cutting, the father of Hadoop, Apache AVRO was developed.

The only DISADVANTAGE of Hadoop Serialization is that the Writable and Sequence FILES have only a Java API. Hence to solve this issue, Avro COMES in picture.

The only disadvantage of Hadoop Serialization is that the Writable and Sequence Files have only a Java API. Hence to solve this issue, Avro comes in picture.

As with the HELP of the WRITABLE objects, Hadoop Writable-based serialization is able to reduce object-creation overhead, which is not POSSIBLE with the Java native serialization framework that’s why using Hadoop one is an advantage.

As with the help of the Writable objects, Hadoop Writable-based serialization is able to reduce object-creation overhead, which is not possible with the Java native serialization framework that’s why using Hadoop one is an advantage.

There is a standard sort order for DATA in AVRO which allows data WRITTEN by one system to be EFFICIENTLY sorted by another system. As sort order comparisons are sometimes the most frequent per-object operation, it can be an important optimization.

There is a standard sort order for data in Avro which allows data written by one system to be efficiently sorted by another system. As sort order comparisons are sometimes the most frequent per-object operation, it can be an important optimization.

Mainly, for two purposes, we use Avro, like:

• DATA serialization
• RPC (Remote procedure call) PROTOCOL

Although, some key points are:

• We are able to read the data from DISK with applications, by USING Avro even which are written in other languages besides java or the JVM.
• Also, Avro allows us to transfer data across a remote system WITHOUT any overhead of java serialization.
• We use Avro when we need to store the large set of data on disk, as it conserves space.
• Further, by using Avro for RPC, we get a better remote data transfer.

Mainly, for two purposes, we use Avro, like:

Although, some key points are:

The classes and methods which we use in the serialization, as well as deserialization of Avro SCHEMAS, are:

• Specific Datum Writer Class
• Specific Datum READER Class
• Data File Writer
• Data File Reader
• Class Schema. PARSER
• INTERFACE GENERIC Record
• Class Generic Data. Record

The classes and methods which we use in the serialization, as well as deserialization of Avro schemas, are:

In the FORMAT “lightweight text-based DATA interchange”, JavaScript OBJECT Notation (JSON), the Avro schema GETS CREATED.

We can make it in various ways:−

• A JSON string
• JSON object
• A JSON array

In the format “lightweight text-based data interchange”, JavaScript Object Notation (JSON), the Avro schema gets created.

We can make it in various ways:−

Basically, SASL OFFERS a FRAMEWORK for AUTHENTICATION and security of NETWORK PROTOCOLS. In Avro also we use SASL Profile for authentication and security purpose.

Basically, SASL offers a framework for authentication and security of network protocols. In Avro also we use SASL Profile for authentication and security purpose.

Whether from an RPC or a FILE, a reader of AVRO data, can always parse that data since its schema is offered. Yet it is possible that schema may not be exactly the schema what we EXPECT so for that purpose we use Schema Resolution.

Whether from an RPC or a file, a reader of Avro data, can always parse that data since its schema is offered. Yet it is possible that schema may not be exactly the schema what we expect so for that purpose we use Schema Resolution.

To transport the data over the network or to store on some persistent storage, the PROCESS of translating data STRUCTURES or objects state into binary or textual form is what we call Serialization. In other WORDS, serialization is also known as as MARSHALING and deserialization is known as UNMARSHALLING.

To transport the data over the network or to store on some persistent storage, the process of translating data structures or objects state into binary or textual form is what we call Serialization. In other words, serialization is also known as as marshaling and deserialization is known as unmarshalling.

In JSON, a SCHEMA is represented by ONE of:

• A JSON STRING
• A JSON object

{“TYPE”: “typename” …attributes…}

• A JSON array

In JSON, a Schema is represented by one of:

{“type”: “typename” …attributes…}

Cons of AVRO are:

• It is must to use .NET 4.5, in the case of C# Avro, to make the BEST use of it.
• POTENTIALLY slower serialization.
• In ORDER to read/write data, need a SCHEMA.

Cons of Avro are:

Pros of Avro are:

• The Smallest Size.
• It COMPRESSES BLOCK at a TIME; split table.
• Maintained Object structure.
• ALSO, supports reading old data W/ new schema.

Pros of Avro are:

Some of the best features of Avro are:

• SCHEMA evolution
• Untagged DATA
• Language SUPPORT
• Transparent compression
• DYNAMIC TYPING
• Native support in MapReduce
• Rich data structures

Some of the best features of Avro are:

There are six KINDS of complex types which AVRO supports:

• RECORDS
• ENUMS
• Arrays
• MAPS
• Unions
• Fixed

There are six kinds of complex types which Avro supports:

AVRO SUPPORTS a WIDE range of Primitive datatypes:

• Null: no VALUE
• Boolean: a binary value
• Int: 32-bit signed integer
• Long: 64-bit signed integer
• Float: single PRECISION (32-bit) IEEE 754 floating-point number
• Double: double precision (64-bit) IEEE 754 floating-point number
• Bytes: the sequence of 8-bit unsigned bytes
• String: Unicode character sequence

Avro supports a wide range of Primitive datatypes:

The workflow to USE Avro is:−

We need to create schemas at first to READ the schemas into our program that is possible in TWO ways.

• Generating a Class CORRESPONDING to Schema
• Using Parsers LIBRARY

Then perform the serialization by using serialization API provided for Avro. And then perform deserialization by using deserialization API provided for Avro.

The workflow to use Avro is:−

We need to create schemas at first to read the schemas into our program that is possible in two ways.

Then perform the serialization by using serialization API provided for Avro. And then perform deserialization by using deserialization API provided for Avro.

Some features where AVRO DIFFERS from other systems are:

• Dynamic TYPING.
• UNTAGGED data.
• No manually-assigned field IDs.

Some features where Avro differs from other systems are:

The most competent libraries with Avro are Thrift and Protocol Buffers. 

The difference between them is: −

• As PER the need, Avro SUPPORTS both dynamic and static types. Basically, to specify SCHEMAS and their types, Protocol Buffers and Thrift uses Interface DEFINITION Languages (IDLs).
• As Avro is built in the Hadoop ECOSYSTEM but Thrift and Protocol Buffers are not.

The most competent libraries with Avro are Thrift and Protocol Buffers. 

The difference between them is: −

Mainly, AVRO HEAVILY depends on its schema. Basically, it permits every data to be written with no PRIOR knowledge of the schema. We can say Avro serialized fast and the data resulting after SERIALIZATION is least in size with schemas.

Mainly, Avro heavily depends on its schema. Basically, it permits every data to be written with no prior knowledge of the schema. We can say Avro serialized fast and the data resulting after serialization is least in size with schemas.

Those who want to LEARN Avro must KNOW HADOOP’s architecture and APIs, before learning Avro. Also must know JAVA with experience in writing BASIC applications before going for Avro.

Those who want to learn Avro must know Hadoop’s architecture and APIs, before learning Avro. Also must know Java with experience in writing basic applications before going for Avro.

Those people who want to learn the basics of Big Data Analytics by using Hadoop Framework and also those who aspire to BECOME a SUCCESSFUL Hadoop developer can go for Avro. Further, those ASPIRANTS who want to use Avro for data serialization and DESERIALIZATION can also learn Avro.

Those people who want to learn the basics of Big Data Analytics by using Hadoop Framework and also those who aspire to become a successful Hadoop developer can go for Avro. Further, those aspirants who want to use Avro for data serialization and deserialization can also learn Avro.

AVRO offers:

• Avro offers Rich DATA STRUCTURES.
• And, a compact, fast, BINARY data format.
• Further, it offers a container file, to store persistent data.
• Remote procedure calls (RPC).

Avro offers:

Some key POINTS are:

• Avro is a Data serialization system
• It USES JSON based schemas
• Moreover, to send data, it uses RPC CALLS.
• And, during data EXCHANGE, Schema’s sent.

Some key points are:

An open SOURCE project which offers data serialization as well as data exchange SERVICES for Apache Hadoop is what we call Apache Avro. It is possible to use these services TOGETHER or independently both. However, programs can efficiently SERIALIZE data into files or into messages, with the serialization service. In addition, data storage is very compact and efficient in Avo because here data definition is in JSON, so, data itself is stored in the binary format MAKING it compact and efficient.

An open source project which offers data serialization as well as data exchange services for Apache Hadoop is what we call Apache Avro. It is possible to use these services together or independently both. However, programs can efficiently serialize data into files or into messages, with the serialization service. In addition, data storage is very compact and efficient in Avo because here data definition is in JSON, so, data itself is stored in the binary format making it compact and efficient.