Is it ideal for the Name Node to also be the Secondary Name Node?

Question

I am practicing a hadoop cluster via Raspberry Pi,, according to this tutorial(http://www.widriksson.com/raspberry-pi-hadoop-cluster/) he sets the node1 in his hadoop masters file configuration which is confusing as he also uses node to start the hadoop daemons. I want to know the reason for his configuration as well

P.S. - Just ctrl+f the masters

It's not ideal. With Hadoop 2.x, Active Namenode will take over Namenode role if Namenode is down. Have a look at related SE question : http://stackoverflow.com/questions/19970461/name-node-vs-secondary-name-node/34716750#34716750 — Ravindra babu, Jan 27 '16 at 17:05

Mobin Ranjbar · Accepted Answer · 2016-01-06T08:37:19.733

1

No It is not ideal. It is up to you how to configure your cluster. In this tutorial, author decided to use node1 as P-NN and S-NN at the same time. Keep in mind that RPi Hadoop Cluster is suitable just for development and test and not production environment.

Pros and Cons of running Primary NameNode and Secondary NameNode on a separate machine(Based on This article from Cloudera):

1.Scalability. Creating the system snapshot requires about as much memory as the NameNode itself occupies. Since the memory available to the NameNode process is a primary limit on the size of the distributed filesystem, a large-scale cluster will require most or all of the available memory for the NameNode.

2.Durability. When the SecondaryNameNode creates a checkpoint, it does so in a separate copy of the filesystem metadata. Moving this process to another machine also creates a copy of the metadata file on an independent machine, increasing its durability.

edited Jan 06 '16 at 08:37

answered Jan 06 '16 at 07:51

Mobin Ranjbar

1,320
1
14
24

Oh ok, thank you for the answer. But what will be the effect of this configuration? Can you speak out the advantages and disadvantages? – Dean Christian Armada Jan 06 '16 at 08:32
@DeanChristianArmada I have edited my answer with pros and cons. – Mobin Ranjbar Jan 06 '16 at 08:37
I think another reason for seperation is in case the node 1 which the "name node" fails then there is an available secondary name node that it can point to – Dean Christian Armada Jan 06 '16 at 08:46
@DeanChristianArmada Secondary NameNode is not a backup for NameNode. – Mobin Ranjbar Jan 06 '16 at 08:49
Correct me if I'm wrong but Secondary Name Node runs when the Name Node has a failure right? – Dean Christian Armada Jan 06 '16 at 08:55
@DeanChristianArmada No It is wrong. It is not a backup node. Read this for the secret story behind S-NN: http://blog.madhukaraphatak.com/secondary-namenode---what-it-really-do/ – Mobin Ranjbar Jan 06 '16 at 08:58
Thanks for everything – Dean Christian Armada Jan 06 '16 at 09:02
I get the idea of the blog, but it can also be used as a backup for the namenode on such cases where the namenode malfunctions or failure right? – Dean Christian Armada Jan 06 '16 at 09:17
@DeanChristianArmada There is another node called BackupNode which does this. S-NN is just a Checkpoint Node. – Mobin Ranjbar Jan 06 '16 at 09:25
Ok,, thanks! Wow, you are very knowledgeable I'm impressed – Dean Christian Armada Jan 06 '16 at 09:26

Is it ideal for the Name Node to also be the Secondary Name Node?

1 Answers1