Hadoop Hands-On exercise with Hortonworks: HiveQL basics

This is second post in the Hadoop hands-on series. In last post we have,

1. Hadoop instance running
2. Data uploaded in HDFS and HCatalog

In this post we query the data using Hadoop. HiveQL is the query language similar to SQL. We will use it to query the data stored in Hadoop clusters.    

Go to Hue > Beeswax (User interface for HiveQL)

Hadoop Hands-On exercise with Hortonworks: Setting up Hadoop sandbox and loading the data

Requirements:

1. You have a software like VitualBox, VMware, Hyper -V installed which can deal with the virtual environment.

2. You have downloaded appropriate sandbox image from here.

3. You have minimum 4 GB RAM

4. Optional: To get familiar with few basic concepts referred in this post (like HDFS, node) you might like to read previous post here. 

Lets get started!

Load the downloaded file, which is "Hortonworks+Sandbox+1.3+VirtualBox+RC6.ova" in my case  and start the virtual machine.

What is Hadoop?

Need: As we are generating more and more data everyday we need tools to deal with this huge scale. The tools can be programming languages, software or infrastructure. Obviously the infrastructure is the foundation for everything else. 

Short history: Google, one of the internet giants faced large scale computation problem pretty early. So Google researchers like Sanjay Ghemawat and Jeff Dean did some work to solve this issue. They published two research papers on Google File System (2003) and MapReduce (2004). These papers were basically trying to solve large scale computations in distributed manner.  In 2005 two researchers Doug Cutting (from Yahoo) and Mike Cafarella build something based on GFS and MapReduce called Hadoop.