Archive

Posts Tagged ‘kerberos’

How-to: Quickly Configure Kerberos for Your Apache Hadoop Cluster “http://blog.cloudera.com/blog/2015/03/how-to-quickly-configure-kerberos-for-your-apache-hadoop-cluster/”

March 11, 2016 Leave a comment

Use the scripts and screenshots below to configure a Kerberized cluster in minutes.

Kerberos is the foundation of securing your Apache Hadoop cluster. With Kerberos enabled, user authentication is required. Once users are authenticated, you can use projects like Apache Sentry (incubating) for role-based access control via GRANT/REVOKE statements.

Taming the three-headed dog that guards the gates of Hades is challenging, so Cloudera has put significant effort into making this process easier in Hadoop-based enterprise data hubs. In this post, you’ll learn how to stand-up a one-node cluster with Kerberos enforcing user authentication, using the Cloudera QuickStart VM as a demo environment.

If you want to read the product documentation, it’s available here. You should consider this reference material; I’d suggest reading it later to understand more details about what the scripts do.

Requirements

You need the following downloads to follow along.

Initial Configuration

Before you start the QuickStart VM, increase the memory allocation to 8GB RAM and increase the number of CPUs to two. You can get by with a little less RAM, but we will have everything including the Kerberos server running on one node.

Start up the VM and activate Cloudera Manager as shown here:

Give this script some time to run, it has to restart the cluster.

KDC Install and Setup Script

The script goKerberos_beforeCM.sh does all the setup work for the Kerberos server and the appropriate configuration parameters. The comments are designed to explain what is going on inline. (Do not copy and paste this script! It contains unprintable characters that are pretending to be spaces. Rather, download it.)

Cloudera Manager Kerberos Wizard

After running the script, you now have a working Kerberos server and can secure the Hadoop cluster. The wizard will do most of the heavy lifting; you just have to fill in a few values.

To start, log into Cloudera Manager by going to http://quickstart.cloudera:7180 in your browser. The userid is cloudera and the password is cloudera. (Almost needless to say but never use “cloudera” as a password in a real-world setting.)

There are lots of productivity tools here for managing the cluster but ignore them for now and head straight for the Administration > Kerberos wizard as shown in the next screenshot.

Click on the “Enable Kerberos” button.

The four checklist items were all completed by the script you’ve already run. Check off each item and select “Continue.”

The Kerberos Wizard needs to know the details of what the script configured. Fill in the entries as follows:

  • KDC Server Host: quickstart.cloudera
  • Kerberos Security Realm: CLOUDERA
  • Kerberos Encryption Types: aes256-cts-hmac-sha1-96

Click “Continue.”

Do you want Cloudera Manager to manage the krb5.conf files in your cluster? Remember, the whole point of this blog post is to make Kerberos easier. So, please check “Yes” and then select “Continue.”

The Kerberos Wizard is going to create Kerberos principals for the different services in the cluster. To do that it needs a Kerberos Administrator ID. The ID created is: cloudera-scm/admin@CLOUDERA.

The screen shot shows how to enter this information. Recall the password is: cloudera.

The next screen provides good news. It lets you know that the wizard was able to successfully authenticate.

OK, you’re ready to let the Kerberos Wizard do its work. Since this is a VM, you can safely select “I’m ready to restart the cluster now” and then click “Continue.” You now have time to go get a coffee or other beverage of your choice.

How long does that take? Just let it work.

Congrats, you are now running a Hadoop cluster secured with Kerberos.

Kerberos is Enabled. Now What?

The old method of su - hdfs will no longer provide administrator access to the HDFS filesystem. Here is how you become the hdfs user with Kerberos:

Now validate you can do hdfs user things:

Next, invalidate the Kerberos token so as not to break anything:

The min.user parameter needs to be fixed per the message below:

This is the error message you get without fixing min.user.id:

Save the changes shown above and restart the YARN service. Now validate that the cloudera user can use the cluster:

If you forget to kinit before trying to use the cluster you’ll get the errors below. The simple fix is to use kinit with the principal you wish to use.

Advertisements

How to Install Kerberos 5 KDC Server on Linux for Authentication

October 27, 2015 Leave a comment

Kerberos is a network authentication protocol.

Kerberos provides a strong cryptographic authentication against the devices which lets the client & servers to communicate in a more secured manner. It is designed to address network security problems.

When firewalls acts a solution to address the intrusion from the external networks, Kerberos usually used to address the intrusion and other security problems within the network.

The current version of Kerberos is version 5 which is called as KRB5.

To implement the Kerberos, we need to have the centralized authentication service running on server.

This service is called KEY DISTRIBUTION CENTER (KDC).

A server registered with KDC is trusted by all other computers in the Kerberos realm.

Sample krb5.conf File

Here’s an example krb5.conf file that contains all the REALM and domain to REALM mapping information,

# cat /etc/krb5.conf
[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = EXAMPLE.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 EXAMPLE.COM = {
  kdc = kerberos.example.com
  admin_server = kerberos.example.com
 }

[domain_realm]
 .example.com = EXAMPLE.COM
 example.com = EXAMPLE.COM

Install Kerberos KDC server

For security reason, it is recommended to run the Kerberos (KDC) server on a separate server.

Download and install the krb5 server package.

# rpm -ivh krb5-server-1.10.3-10.el6_4.6.x86_64.rpm
Preparing...       ########################################### [100%]
   1:krb5-server   ########################################### [100%]

Verify that the following rpm are installed before configuring KDC:

# rpm -qa | grep -i krb5
pam_krb5-2.3.11-9.el6.x86_64
krb5-server-1.10.3-10.el6_4.6.x86_64
krb5-workstation-1.10.3-10.el6_4.6.x86_64
krb5-libs-1.10.3-10.el6_4.6.x86_64

Modify /etc/krb5.conf File

Change /etc/krb5.conf to reflect like the below with the appropriate REALM and DOMAIN_REALM mappings.

# cat /etc/krb5.conf
[logging]
 default = FILE:/var/log/krb5libs.log
 kdc = FILE:/var/log/krb5kdc.log
 admin_server = FILE:/var/log/kadmind.log

[libdefaults]
 default_realm = MYREALM.COM
 dns_lookup_realm = false
 dns_lookup_kdc = false
 ticket_lifetime = 24h
 renew_lifetime = 7d
 forwardable = true

[realms]
 MYREALM.COM = {
  kdc = elserver1.example.com
  admin_server = elserver1.example.com
 }

[domain_realm]
 .myrealm.com = MYREALM.COM
 myrealm.com = MYREALM.COM

Modify kdc.conf File

Also the kdc.conf should be modified as shown below.

# cat /var/kerberos/krb5kdc/kdc.conf
[kdcdefaults]
 kdc_ports = 88
 kdc_tcp_ports = 88

[realms]
 MYREALM.COM = {
  #master_key_type = aes256-cts
  acl_file = /var/kerberos/krb5kdc/kadm5.acl
  dict_file = /usr/share/dict/words
  admin_keytab = /var/kerberos/krb5kdc/kadm5.keytab
  supported_enctypes = aes256-cts:normal aes128-cts:normal des3-hmac-sha1:normal 
  arcfour-hmac:normal des-hmac-sha1:normal des-cbc-md5:normal des-cbc-crc:normal
 }

Create KDC database

Next, create the KDC database using the kdb5_util command as shown below. As this stage, enter the appropriate pasword for the KDC database master key.

# /usr/sbin/kdb5_util create -s
Loading random data
Initializing database '/var/kerberos/krb5kdc/principal' for realm 'MYREALM.COM',
master key name 'K/M@MYREALM.COM'
You will be prompted for the database Master Password.
It is important that you NOT FORGET this password.
Enter KDC database master key:
Re-enter KDC database master key to verify:

Assign Administrator Privilege

The users can be granted administrator privileges to the database using the file /var/kerberos/krb5kdc/kadm5.acl.

# cat /var/kerberos/krb5kdc/kadm5.acl
*/admin@MYREALM.COM     *

In the above example, any principal in the MYREALM with an admin instance has all administrator privileges.

Create a Principal

Create the principal using the following command. In this example, I created the principal with the user name “eluser”.

# kadmin.local -q "addprinc eluser/admin"
Authenticating as principal root/admin@MYREALM.COM with password.
WARNING: no policy specified for eluser/admin@MYREALM.COM; defaulting to no policy
Enter password for principal "eluser/admin@MYREALM.COM":
Re-enter password for principal "eluser/admin@MYREALM.COM":
Principal "eluser/admin@MYREALM.COM" created.

Start the Kerberos Service

Start the KDC and kadmin daemons as shown below.

# service krb5kdc start
Starting Kerberos 5 KDC:               [  OK  ]

# service kadmin start
Starting Kerberos 5 Admin Server:      [  OK  ]

Configure Kerberos Client

Configure the Kerberos client to authenticate against the KDC database:

Now let’s see how to configure the krb5 client to authenticate against the Kerberos KDC database we created above.

Step 1: Install the krb5-libs and krb5-workstation packages on the client machine.

Step 2: Copy the /etc/krb5.conf from the KDC server to the client machine.

Step 3: Now we need to create the principal for the client in the KDC/Kerberos database.

You can use the below commands to create the principal for the client machine on the KDC master server. In the below example the I am creating a host principal for the client elserver3.example.com on the master KDC server elserver1.example.com

# kadmin.local -q "addprinc host/elserver3.example.com"
Authenticating as principal root/admin@MYREALM.COM with password.
WARNING: no policy specified for host/elserver1.example.com@MYREALM.COM; defaulting to no policy
Enter password for principal "host/elserver1.example.com@MYREALM.COM":
Re-enter password for principal "host/elserver1.example.com@MYREALM.COM":
Principal "host/elserver1.example.com@MYREALM.COM" created.

Step 4: Extract the krb5.keytab for the client from the KDC master server using the below command:

# kadmin.local -q "ktadd -k /etc/krb5.keytab host/elserver3.example.com"
Authenticating as principal root/admin@MYREALM.COM with password.
Entry for principal host/elserver3.example.com with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/elserver3.example.com with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/elserver3.example.com with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/elserver3.example.com with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/elserver3.example.com with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal host/elserver3.example.com with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:/etc/krb5.keytab.

This completes the configuration. You are all done at this stage.

From now on, everytime you establish a SSH, RSH connection the host verifies its identity against the KDC database using keytab file and it establishes secure connection over the Kerberos.

Ktadd is used a generate new keytab or add a principal to an existing keytab from the kadmin command.

Ktremove is used to remove the principal from an existing keytab. The command to remove the principal that we created above is,

kadmin.local -q "ktremove -k /etc/krb5.keytab –q all"

Delete a KDC database

For some reason, if you have to delete a KDC database, use the following command:

# kdb5_util -r MYREALM.COM destroy
kdb5_util: Deleting KDC database stored in /usr/local/var/krb5kdc/principal, you sure
(type yes to confirm)? <== yes
OK, deleting database '/usr/local/var/krb5kdc/principal'...

-f option in the above command forces the deletion of KDC database.

Backup and Restore KDC Database

To backup a KDC database to a file, use krb5_util_dump.

# kdb5_util dump kdcfile

# ls -l kdcfile
-rw-------. 1 root root 5382 Apr 10 07:25 kdcfile

To restore the KDC database from the dump file created in the above step, do the following:

# kdb5_util load kdcfile
Categories: kerberos, linue Tags: ,