Sunday, September 23, 2012

Lily cluster setup - HDFS permission issue and solution

Really enjoy using Lily, a framework for easily working with HBase and Solr.  I hadn't set up a Lily cluster in awhile and was perplexed with these kinds of errors when starting the Lily service from my datanodes:

Caused by: org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.security.AccessControlException: Permission denied: user=lily, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x

But when I started the Lily server as root, it seemed to work but this wasn't an optimal solution.  This means my hdfs directory permissions weren't set properly since lily contacts hdfs as whatever user you start as.  I had to set the user/group of /lily to lily:lily from the namenode:

[user@master01 ~]$ sudo su hdfs
[sudo] password for user:
bash-4.1$ hadoop fs -chown -R lily:lily /lily

This made the hdfs folder happy and my datanodes were able to start the lily service without becoming root.

In some cases, you might need to create the lily directory just like you do when setting up mapred.  To do so:

[user@master01 ~]$ sudo su hdfs
[sudo] password for user:
bash-4.1$ hadoop fs -mkdir /lily