How to install MapR M3 on Ubuntu through Ubuntu Partner Archive.

In a recent post of mine I had mentioned about the partnership between MapR and Canonical towards an initiative to make Hadoop available with Ubuntu natively through Ubuntu Partner Archive. Since, the package has been released now, I thought of showing how to get it done. Trust me it's really cool to install Hadoop by just one apt-get install :)

First things first. Open your sources.list file and add the MapR repositories into it.

deb http://package.mapr.com/releases/v2.1.2/ubuntu/ mapr optional
deb http://package.mapr.com/releases/ecosystem/ubuntu binary/

Now, update your repository.
sudo apt-get update

Note : If it throws any error regarding MapR repositories, just uncomment the lines which allow us to add software from Canonical's partner repository.

## Uncomment the following two lines to add software from #Canonical's
## 'partner' repository.
## This software is not part of Ubuntu, but is offered by #Canonical and the
## respective vendors as a service to Ubuntu users.
deb http://archive.canonical.com/ubuntu precise partner 
deb-src http://archive.canonical.com/ubuntu precise partner


Install hadoop.
sudo apt-get install mapr-single-node

1, 2, 3..and you are done. Isn't that cool?Just three easy steps and you have your brand new single node hadoop cluster in your lap. But, there are some pre-requisites and it's very important to satisfy them.

CPU : 64-bit

OS : Red Hat, CentOS, SUSE, or Ubuntu

Memory : 4 GB minimum, more in production

Disk : Raw, unformatted drives and partitions

DNS : Hostname, reaches all other nodes

Users : Common users across all nodes; Keyless ssh

Java : Must run Java

Other : NTP, Syslog, PAM

The above procedure will install following services on your machine :

CLDB : mapr-cldb

JobTracker : mapr-jobtracker

MapR Control Server : mapr-webserver

MapR Data Platform : mapr-fileserver

Metrics : mapr-metrics

NFS : mapr-nfs

TaskTracker : mapr-tasktracker

ZooKeeper : mapr-zookeeper

In order to install other hadoop projects and for further documentation you can visit the official documentation here.

I hope you found this post helpful, and as always comments and suggestions are welcome.

Hadoop+Ubuntu : The Big Fat Wedding.

Now, here is a treat for all you Hadoop and Ubuntu lovers. Last month, Canonical, the organization behind the Ubuntu operating system, partnered with MapR, one of the Hadoop heavyweights, in an effort to make Hadoop available as an integrated part of Ubuntu through its repositories. The partnership announced that MapR's M3 Edition for Apache Hadoop will be packaged and made available for download as an integrated part of the Ubuntu operating system. Canonical and MapR are also working to develop a Juju Charm that can be used by OpenStack and other customers to easily deploy MapR into their environments.

The free MapR M3 Edition includes HBase, Pig, Hive, Mahout, Cascading, Sqoop, Flume and other Hadoop-related components for unlimited production use. MapR M3 will be bundled with Ubuntu 12.04 LTS and 12.10 via the Ubuntu Partner Archive. MapR also announced that the source code for the component packages of the MapR Distribution for Apache Hadoop is now publicly available on GitHub.

MapR is the only distribution that enables Linux applications and commands to access data directly in the cluster via the NFS interface that is available with all MapR Editions. The MapR M5 and M7 Editions for Apache Hadoop, which provide enterprise-grade features for HBase and Hadoop such as mirroring, snapshots, NFS HA and data placement control, will also be certified for Ubuntu.

Now, as you get Hadoop integrated natively with Ubuntu, it's a lot easier to install it and go. No more unnecessary downloads and wacky configuration steps. And the best part is the NFS interface available with MapR's distribution that enables other Linux commands and application to access the cluster data directly. The Ubuntu/MapR package will be available through the Ubuntu Partner Archive for 12.04 LTS and 12.10 releases of Ubuntu on the official website starting from April 25, 2013.

For more info you can get the Ubuntu and Hadoop: the perfect match white paper from here.

How to install Java 6 on Ubuntu

In one of the previous post of mine I have shown you how to install Sun(Oracle) Java on Ubuntu through its repository. This will, by default, install Java-7 on your machine as Ubuntu 12.04(and onwards) have Java-7 in their repository. But, sometimes you may come across a situation wherein you need some specific version of Java. For example, it is advisable to use Java-6 while trying to configure or use Apache Hadoop. In such a scenario you need to download the appropriate version of Java and install it manually. It is again a straightforward process. Just follow the steps below :

Note : Java-6 has been taken here, as an example, on a machine running Ubuntu 12.10

Step 1 : Download the required version of Java from the official download page. It will download  jdk-6u38-linux-x64.bin inside your Downloads directory.

Step 2 : Go to the directory where jdk was downloaded(Download here) and make it an executable file using this command :
apache@hadoop:~/Downloads$ sudo chmod +x jdk-6u38-linux-x64.bin

Step 3 : Run this executable .bin file using :
apache@hadoop:~/Downloads$ ./jdk-6u38-linux-x64.bin

Step 4 : Now move the file to the /usr/lib/jvm directory with the help of mv command :
apache@hadoop:~/Downloads$ sudo mv jdk1.6.0_38/ /usr/lib/jvm/

Step 5 : This is the step where you actually do the installation. Use the following commands to do that :
apache@hadoop:~/Downloads$ sudo update-alternatives  --install /usr/bin/javac javac /usr/lib/jvm/jdk1.6.0_38/bin/javac 1
apache@hadoop:~/Downloads$ sudo update-alternatives --install /usr/bin/java java /usr/lib/jvm/jdk1.6.0_38/bin/java 1
apache@hadoop:~/Downloads$ sudo update-alternatives --install /usr/bin/javaws javaws /usr/lib/jvm/jdk1.6.0_38/bin/javaws 1

Once you are done with these steps you are good to go. One step is actually still required though. The above procedure will just install Java-6 on your machine. In case you have multiple versions of Java installed on your machine, you need to set which version you want to use as your default Java. To do so issue the command shown below :
apache@hadoop:~/Downloads$ sudo update-alternatives --config java

This will show you all the Java versions installed on your machine. Choose the number corresponding to the version of your choice and you are all set.

How to add minimize and maximize buttons in Gnome 3

In a recent post I had shown the steps to add the Power off option in the menu, which is absent by default, in Gnome 3. In this post we'll see another feature which most of us would love to have in our Gnome environment, especially guys coming from PC world. The minimize and maximize buttons.

In order to do that follow the specified below :

1. Open run box by pressing ALT+F2. Type dconf-editor in it and hit enter(Alternatively you can open   your terminal and type there).
   * If you don't have dconf install on your machine, you can install it using this command -
      apache@hadoop:~$ sudo apt-get install dconf-tools

2. This will open the dconf-editor and you will see something like this on your screen -

Now, go to org->gnome->shell->overrides and click there. It will open the editor in the right pane. Here click on the value against button-layout. By default it'll show only close there.

Add minimize and maximize option here and you are done.

Let me know if you face any issue. Thank you.

How to add shutdown option in the Gnome 3 menu

While I am just in love with Gnome 3, one thing which I don't like about it is the absence of shutdown or power off option. But, the good news it that it can be added very easily. All you have to do is follow these three simple steps :

1. Add the repository -
    apache@hadoop:~$ sudo add-apt-repository ppa:ferramroberto/gnome3

2. Update it -
    apache@hadoop:~$ sudo apt-get update

3. Install the Gnome extensions -
    apache@hadoop:~$ sudo apt-get install gnome-shell-extensions

Now restart your machine and you are good to go.

NOTE : This is procedure if you are working on Ubuntu. You might have to use a different procedure to add the repository and do the installation depending on your OS.

Thank you.

UBUNTU FOR ANDROID

Now, here is something that would really catch each open-source lover's attention. Team Ubuntu has come up with the idea of running Ubuntu on your Android device. Ubuntu for Android, as they call it, is an upcoming free and open source variant of Ubuntu designed to run on Android phones. It is expected to come pre-loaded on several phones. Android was shown at Mobile World Congress 2012. The best thing about Ubuntu for Android is that you don't have to reboot the device as both the operating systems use the same kernel.

Other salient features include :

1. Both Ubuntu and Android run at the same time on the device, without emulation or virtualization.

2. When the device is connected to a desktop monitor, it features a standard Ubuntu Desktop interface.

3. When the device is connected to a TV, the interface featured is the Ubuntu TV experience.

4. Ability to run standard Ubuntu Desktop applications, like Firefox, Thunderbird, VLC, etc.

5. Ability to run Android applications on the Ubuntu Desktop.

6. Make and receive calls and SMSs directly from the Desktop

In order to use Ubuntu for Android, the device should have :

1. Dual-core 1 GHz CPU

2. Video acceleration: shared kernel driver with associated X driver; OpenGL, ES/EGL

3. Storage: 2 GB for OS disk image

4. HDMI: video-out with secondary framebuffer device

5. USB host mode

6. 512 MB RAM

Here is an image showing Ubuntu running on an Android device, docked to a desktop monitor.

For more info you can visit the official page.

HOW TO INSTALL NAGIOS ON UBUNTU 12.04

Monitoring plays an important role in running our systems smoothly. It is always better to diagnose the problems and take some measures as early as possible, rather than waiting for things to go worse.

Nagios is a powerful monitoring system that enables organizations to identify and resolve IT infrastructure problems before they affect critical business processes. For a detailed information on Nagios you can visit the official documentation page here. I'll just cover the steps to install and get Nagios working on your Ubuntu box.

First of all install Nagios on your Ubuntu box using the following command :
$ sudo apt-get install -y nagios3

It will go through, and ask you about what mail server you want to use. You'll see something like this on your screen.



Pick one as per your requirements.



It will then ask you about the domain name you want to have email sent from. Again, fill that out based upon your needs.

It will ask you what password you want to use - put in a secure password. This is for the admin account nagiosadmin.

NOTE : Keep the account name and password name in mind as you'll need it to log in to Nagios.



It'll ask you to verify your password.

Once the install is all done, point your web browser to localhost/nagios3 or whatever the IP address/domain name of the server you installed Nagios on is.

At this point you'll be asked to enter your password. Enter your password and hit enter.



Once you've done that, you're in and you'll see something like this on your screen.



Nagios automatically adds in the 'localhost' to the config, and does load, current users, disk space, http and ssh checks.



Before leaving we have to do one more thing to make Nagios all set. We need to have it accept external commands so that we can acknowledge problems, add comments, etc. To do that, we need to edit a few files.

Start by opening /etc/nagios3/nagios.cfg with the following command.
$ sudo nano /etc/nagios3/nagios.cfg


Now, search for check_external_commands, and turn the check_external_commands=0 into check_external_commands=1.

Now, restart apache by running.

$ sudo service apache2 restart


Not done yet! We need to edit /etc/group. There should be a line like this in there.

nagios:x:114

Change it to
nagios:x:114:www-data

Save and close this file.

Now, we need to edit the /var/lib/nagios3/rw files permission with.
$ sudo chmod g+x /var/lib/nagios3/rw

And then (because of how permissions work) we need to edit the permissions of the directory above that with.
$ sudo chmod g+x /var/lib/nagios3

Now, restart nagios with.
$ sudo service nagios3 restart

And you should be good to go! Happy monitoring!

NOTE : Do not forget to let me know whether it worked for you or not.

How To Find A File In Linux(Through The Shell)

In *nix family of operating systems, we can find a file easily with the help of find command.

SYNTAX : $ find {directory-name} -name {filename}

To find a file in root directory :

Sometimes, it happens that we don't have any clue about the location of the file we are trying to search. In such a case we can search the entire system via the root directory (/). For example, if we want to search for a file named demo.txt, but we don't know where it could probably be present, then we would do something like this :

$ sudo find / -name 'demo.txt'

NOTE : Sometimes we may need special privileges to search for a particular file, so we'll use 'sudo' for that.

To find a file in a specific directory :

If we know the probable location of the file, but not sure about it, we can do this :

$ sudo find /path/of/the/directory -name 'demo.txt'

HOW TO SETUP AND CONFIGURE 'ssh' ON LINUX (UBUNTU)

SSH (Secure Shell) is a network protocol secure data communication, remote shell services or command execution and other secure network services between two networked computers that it connects via a secure channel over an insecure network. The ssh server runs on a machine (server) and ssh client runs on another machine (client).

ssh has 2 main components :
1- ssh : The command we use to connect to remote machines - the client. 
2- sshd : The daemon that is running on the server and allows clients to connect to the server.
ssh is pre-enabled on Linux, but in order to start sshd daemon, we need to install ssh first. Use this command to do that :

$ sudo apt-get install ssh
This will install ssh on your machine. In order to check if ssh is setup properly do this :

$ which ssh
It will throw this line on your terminal
/usr/bin/ssh

$ which sshd
It will throw this line on your terminal
/usr/bin/sshd

SSH uses public-key cryptography to authenticate the remote computer and allow it to authenticate the user, if necessary. Well, there are numerous post and links that explain about ssh in much detail. You can just google ssh if you want to learn about it. I'll now show the steps required to configure ssh.

1- First of all create a ssh-keypair using this command :
$ ssh-keygen -t rsa -P ""
Once you issue this command it will ask you for the name of directory where you want to store the key. Simple hit enter without giving any name, and your key will be created and saved to the default location i.e /.ssh directory inside your home directory. (Files and directories having names tarting with a dot (.) are hidden files and directories in Linux. To see these files and directories just go to your home folder and press Ctrl+h).

cluster@ubuntu:~$ ssh-keygen -t rsa -P ""

Generating public/private rsa key pair.

Enter file in which to save the key (/home/cluster/.ssh/id_rsa):

Hit enter, and you will see something like this :

Your identification has been saved in /home/cluster/.ssh/id_rsa.
Your public key has been saved in /home/cluster/.ssh/id_rsa.pub.
The key fingerprint is:
66:4f:72:26:2b:18:57:43:64:4f:3e:5a:58:d1:2c:30 cluster@ubuntu
The key's randomart image is:
+--[ RSA 2048]-----+
|             .E.++          |
|              o B. o         |
|              + =.           |
|              . + .            |
|           . . S +            |
|           + o O            |
|           . . . .              |
|               .                 |
|                                 |
+---------------------+
cluster@ubuntu:~$

Your keypair has been created now. The keypair contains the keys in 2 different files present under the .ssh directory. These are id_rsa (the private key) and id_rsa.pub (the public key). 

To connect to a remote machine, just give ssh command along with the hostname of that machine. 

NOTE : Hostname of the machine to which you want to ssh must be present in your /etc/hosts file along with its IP address.

For example, if you want to connect to machine called 'client', do this :

cluster@ubuntu:~$ ssh localhost 
cluster@localhost's password:

Enter the password of the client machine to login. Now, you are at the terminal of the client machine. Just use a few commands like ls or cat to cross check. Once you give the password and hit enter you will see something like this on your terminal :

cluster@ubuntu:~$ ssh client
cluster@client's password: 
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-26-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

90 packages can be updated.
10 updates are security updates.

Last login: Fri Jul 20 01:08:28 2012 from client
cluster@client:~$

NOTE : In some cases you may want to use passwordless ssh (for example while working with Apache Hadoop). To do that you just have to copy the public key, i'e the content of your id_rsa.pub file to the authorized_keys in the .ssh directory of the client machine. Use the following command to do that :

cluster@ubuntu:~$cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys

Now, if you do ssh client, you won't be asked for any password.

cluster@ubuntu:~$ ssh client 
Welcome to Ubuntu 12.04 LTS (GNU/Linux 3.2.0-26-generic x86_64)

 * Documentation:  https://help.ubuntu.com/

90 packages can be updated.
10 updates are security updates.

Last login: Fri Jul 20 01:08:28 2012 from client
cluster@client:~$

HOW TO INSTALL SUN(ORACLE) JAVA ON UBUNTU 12.04 IN 3 EASY STEPS

If you have upgraded to Ubuntu 12.04 or just made a fresh Ubuntu installation you might want to install sun(oracle) java on it. Although Ubuntu has its own jdk, the OpenJdk, but there certain things that demand for sun(oracle) java. You can follow the steps shown below to do that -

1 - Add the “WEBUPD8″ PPA :

    hadoop@master:~$ sudo add-apt-repository ppa:webupd8team/java 

2 - Update the repositories :

     hadoop@master:~$ sudo apt-get update

3 - Begin the installation :

     hadoop@master:~$ sudo apt-get install oracle-java7-installer

Now, to test if the installation was ok or not do this :

hadoop@master:~$ java -version

If everything was ok you must be able to see something like this on your terminal :

hadoop@master:~$ java -version

java version "1.7.0_05"

Java(TM) SE Runtime Environment (build 1.7.0_05-b05)

Java HotSpot(TM) 64-Bit Server VM (build 23.1-b03, mixed mode)

hadoop@master:~$ 

How to install maven3 on ubuntu 11.10

If you are trying to install maven2 that comes shipped with your ubutnu 11.10, and it is not working as intended you can try following steps to install maven3 :

1 - First of all add the repository for maven3. Use following command for this -
     $ sudo add-apt-repository ppa:natecarlson/maven3

2 - Now update the repositories - 
     $ sudo apt-get update

3 - Finally install maven3 - 
      $ sudo apt-get install maven3

NOTE : To check whether installation was done properly or not, issue the following 
              command -
              $ mvn --version