What is Hadoop?

Hadoop in Short :
Hadoop is an Apache Open Source Framework, designed for Distributed framework. It designed with the concept in mind to address "Hardware Failure" & the process to recover that through software framework..

Founder History : 
Hadoop was created by Doug Cutting and Mike Cafarella in 2005. Cutting, who was working at Yahoo! at the time, named it after his son's toy elephant. It was originally developed to support distribution for the Nutch search engine project.

Hadoop 2 EcoSystem :

Source : Google

Hadoop Current Version : 

Apache Hadoop Released Version 2.7.1(while this blog created).. You have to understand how to read version number first.. so that you can be clear, what kind of patches/upgrades are being carried out & how to react with it.

Version number comes in 0.0.0(3 digit formats)..

Version numbers usually consist of three numbers separated by dots. For example: 2.7.1 These numbers have names. The leftmost number (2) is called the major version. The middle number (7) is called the minor version. The rightmost number (1) is called the revision but it may also be referred to as a "point release" or "sub-minor version".

Most of the organizations are using Hadoop 1.2.1, but they can roll forward & upgrade to newer version..

Hadoop Functionality :

1) Storage , Analysis , Statistics, not all the tools that address big data problem have this beauty.. That's the reason, Hadoop stands unique.

2) It has two core components(HDFS/MapReduce).

HDFS stands for Hadoop distributed file system..

Why Hadoop Came into existence?

When we are dealing with data, we need to look into few V's..

Volume : Amount of data that generates / sec/min/hour/day...

Variety : Types of data that are getting generated.

Velocity : Speed in which data generates or we access it. 

Relational database systems cannot handle if volumes are higher than GigaBytes, so Organization tend to archive the data, when it reaches the threshold. So that's the end of those archived data. It will not be taken into account for analysis. 

At the same time, it is also important that we deals with variety of data that are getting generated through social media, mobile phones, auto mobiles etc.. 

So it's hard to take all those data for computation & generate results. Here Hadoop(DFS) plays a key role in moving computation to the data where it resides.  It follows Distributed File System architecture, where all the data will be stored into multiple notes(cluster) & allows the program/computation part to move over & perform the analysis. 

Clusters? 

Consider you have a bunch of computers/laptop that are connected in network LAN. Which we can take it as one Rack. Here, computers are referred as NODES. Nodes that are connected in same network switch will be considered as one RACK.

Hence multiple racks forms a cluster. Each racks holds minimum of 2 nodes to n Nodes...  for instance (see the picture below)

Source : Google

Who uses Hadoop?

It's a big list, many organizations have deployed Hadoop to analyze their data. Can review the list here in http://wiki.apache.org/hadoop/PoweredBy 

It's open source framework, so all companies are tend to move towards to it.

We will see each components in details in next post.

Thanks for reading, please don't forget to hit like/comment.

Disclaimer : All content provided on this blog is for informational purposes only. Content that are shared here, is purely what I learnt & my practical knowledge.  The Apache Software Foundation has no affiliation with and does not endorse or review the materials provided at this website.