This document describes how to modify the Hadoop configuration to enable it to access Kerberos. For secure clusters purchased through EMR, the required settings are already automatically configured by the system.
/var/krb5kdc/emr.keytab
).Hadoop mainly contains HDFS and Yarn services. You need to modify their configurations separately and restart the service processes.
hadoop.security.authentication= kerberos
hadoop.security.authorization= true
dfs.namenode.kerberos.principal= hadoop/_HOST@EMR
dfs.namenode.keytab.file= /var/krb5kdc/emr.keytab
dfs.namenode.kerberos.internal.spnego.principal= HTTP/_HOST@EMR
dfs.secondary.namenode.kerberos.principal= hadoop/_HOST@EMR
dfs.secondary.namenode.keytab.file= /var/krb5kdc/emr.keytab
dfs.secondary.namenode.kerberos.internal.spnego.principal= HTTP/_HOST@EMR
dfs.journalnode.kerberos.principal= hadoop/_HOST@EMR
dfs.journalnode.keytab.file= /var/krb5kdc/emr.keytab
dfs.journalnode.kerberos.internal.spnego.principal= HTTP/_HOST@EMR
dfs.datanode.kerberos.principal= hadoop/_HOST@EMR
dfs.datanode.keytab.file= /var/krb5kdc/emr.keytab
dfs.datanode.data.dir.perm= 700
dfs.web.authentication.kerberos.keytab= /var/krb5kdc/emr.keytab
dfs.web.authentication.kerberos.principal= HTTP/_HOST@EMR
ignore.secure.ports.for.testing= true
Note:
The
ignore.secure.ports.for.testing
option must be set to true; otherwise, the sasl mode has to be configured, and webhdfs has to have HTTPS enabled.
httpfs.authentication.type= kerberos
httpfs.hadoop.authentication.type= kerberos
httpfs.authentication.kerberos.principal= HTTP/_HOST@EMR
httpfs.hadoop.authentication.kerberos.principal= hadoop/_HOST@EMR
httpfs.authentication.kerberos.keytab= /var/krb5kdc/emr.keytab
httpfs.hadoop.authentication.kerberos.keytab= /var/krb5kdc/emr.keytab
yarn.resourcemanager.keytab= /var/krb5kdc/emr.keytab
yarn.resourcemanager.principal= hadoop/_HOST@EMR
yarn.nodemanager.keytab= /var/krb5kdc/emr.keytab
yarn.nodemanager.principal= hadoop/_HOST@EMR
mapreduce.jobhistory.keytab= /var/krb5kdc/emr.keytab
mapreduce.jobhistory.principal= hadoop/_HOST@EMR
Was this page helpful?