Configure security features

Hadoop Eco supports security features using Apache Ranger and Kerberos.

caution

HDE-1.0.1 version does not support security features. Some components that do not support Kerberos may not function properly,
and enabling security features may prevent Data Catalog from operating correctly.

Kerberos

Kerberos is a user authentication protocol developed by MIT.
When Kerberos authentication is applied, KDC is installed, and Kerberos authentication is applied to Hadoop, Hive, and HBase.

To use Hadoop, Hive, and HBase after enabling Kerberos, authentication must be performed via Kerberos, and inter-component authentication using a keytab will create a default keytab file at /etc/hadoopeco.keytab. You can verify the default user list with this keytab.

Check keytab

# Verify default users
klist -kt /etc/hadoopeco.keytab
 
# Authentication method
kinit -kt /etc/hadoopeco.keytab hdfs/{host_name}@{realm_name}

info

For details on Kerberos installation, refer to Installation and Integration Guide.

Change component ports

When Kerberos is configured, the default ports for certain components will change as follows:

HDE version	Component	Default port	Changed port	Notes
HDE-1.x	HDFS Namenode	50070	50470	Access via HTTPS
	HDFS SecondaryNamenode	50090	50091
	HDFS Datanode	50075	50475	Access via HTTPS
HDE-2.x	HDFS Namenode	9870	9871	Access via HTTPS
	HDFS SecondaryNamenode	9868	9869
	HDFS Datanode	9864	9867	Access via HTTPS

Ranger

Hadoop Eco uses Apache Ranger to apply ACLs and monitor audits.

The default Ranger ID is admin, and the password is the administrator password set during cluster creation. Ranger requires the following password creation rules:

• At least 8 characters
• At least one uppercase letter, one lowercase letter, and one number
• Certain special characters (\'""`) are not supported

Access Ranger

Ranger is installed on the master server in single-node setups and on the third master server in HA setups. Access is available via port 6080.

1. Verify the security group of the VM where Ranger is installed and add a public IP.
2. Log in using admin or rangerAdmin1.
3. By default, HDFS, YARN, and Hadoop SQL plugins are supported.

Ranger policies

After modifying Ranger policies, agents must fetch and apply these policies.
In Plugin Status, verify that policies are downloaded and marked as Active before executing tasks.

After modifying HDFS policies, verify in Access Enforcer that ranger-acl is applied to restrict user access.

Installation and integration methods

Installation method

During the Hadoop Eco creation process, you can install Kerberos and Ranger by adding the following information in Step 3: Cluster Configuration Settings (Cluster configuration settings):

Kerberos and Ranger installation method

    {
        "configurations":
        [
            {
                "classification": "kerberos-setting",
                "properties":
                {
                  "enabled": true,        // Enable Kerberos configuration
                  "passwd": "bigadmin",   // KDC password
                  "realm": "HADOOP.ECO"   // Default realm name
                }
            },
            {
                "classification": "ranger-setting",
                "properties":
                {
                    "enabled": true
                }
            }
        ]
    }

Integration method

Ranger integrates with Kerberos to verify users.
To validate tasks, add users to each node and issue Kerberos keytabs.

Add users

For Kerberos authentication, add users to the desired group on the master node.
On the master node, add users to the desired group, and on the worker nodes, add users to the hadoop group.

# Master node
groupadd {group_name}
useradd {user_name} -g {group_name}
usermod -G {group_name} {user_name}
    
# Worker node
useradd {user_name} -g hadoop
usermod -G hadoop {user_name}

After adding users, you must notify the NameNode and ResourceManager about the added users.

Apply to NameNode

kinit -kt /etc/hadoopeco.keytab hdfs/{host_name}@{realm_name}
hdfs dfsadmin -refreshServiceAcl
hdfs dfsadmin -refreshUserToGroupsMappings
hdfs dfsadmin -refreshSuperUserGroupsConfiguration

Apply to ResourceManager

kinit -kt /etc/hadoopeco.keytab yarn/{host_name}@{realm_name}
yarn rmadmin -refreshSuperUserGroupsConfiguration
yarn rmadmin -refreshUserToGroupsMappings
yarn rmadmin -refreshAdminAcls
yarn rmadmin -refreshServiceAcl

Issue keytab and log in

Register the policy (principal) of the user in KDC and issue a keytab.

Log in to kadmin

sudo kadmin.local

# Register user and issue keytab in kadmin CLI
addprinc -randkey {user_name}@{realm_name}
xst -k {keytab_file_name} {user_name}@{realm_name}

After issuing the keytab, verify that the file has been created at the specified location.

Verify keytab

# Check the list of users in the keytab
klist -kt {keytab_file_name}

# Log in using the keytab account
kinit -kt {keytab_file_name} {user_name}@{realm_name}

# Verify authenticated user
klist

Verify operation

After registering the user and executing tasks, you can verify that HDFS, YARN, and Hadoop SQL ACLs are applied as shown below.