2

I started a small hadoop cluster for experiment purpose, in my own hardware, with three datanodes with 30GB disk space. Later I added two more nodes with 200GB and now my cluster has approximately 420GB. My replica factor is 2. Today my starting datanodes are full and even when the new datanodes have free space, writing on the cluster fail when the cluster try to replicate blocks. I assume that the namenode is trying to replicate blocks to full datanodes. I've been reading about a policy (dfs.datanode.fsdataset.volume.choosing.policy) to manage the write on volumes but seems that not have effect on replica process over datanodes.
Is there a way, or policy, to ignore some datanodes from the write or replica process when they are full.

Thanks in advance

0 Answers0