Tag: hadoop

When ever a request is made to a data node to read a block, the data node will read the block from disk. If you know that you

We have seen here, that Avro supports three kinds of API (Generic,Specific,reflect). In this blog we will see how to use Reflect API. Reflect API is use full

Kafka is a distributed pub/sub broker which can scale horizontally. Kafka broker is designed to work in clusters , where you will run multiple Kafka brokers on different

Hadoop has many standard compression codecs available, namely DEFLATE (.deflate) ,gzip (.gz), bzip2 (.bz2) ,LZO (.lzo),LZ4 (.lz4),Snappy (.snappy) . Only bzip2 is splittable , it very important for

In the previous blog we have seen how to convert a text file to Avro data file.In this blog we will see how to run a map reduce

If the input avro data file has many fields and we are interested in only few fields, then we can define a new schema with the fewer fields. this

Avro Supports both primitive and complex data types Primitive data types null, boolean, int, long, float, double, string, bytes Complex data types array – ordered collection of objects

Most common action type you will find in oozie workflow is <map-reduce> action type. In this blog we will see how to define a map-reduce action type. the

We have already seen how to convert a text file to Avro data file using a simple java program. In this blog we will see how to process

Your reading schema doesn’t has to be same as that of the writing schema. You can add new fields or remove the existing fields(projection). If a new field