HADOOP_CONF_DIR
的相关配置文件。CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hadoop.username' = 'hive','dfs.nameservices'='your-nameservice','dfs.ha.namenodes.your-nameservice'='nn1,nn2','dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007','dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007','dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider');
type
和 hive.metastore.uris
两个必须参数外,还可以通过更多参数来传递连接所需要的信息。CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hadoop.username' = 'hive','dfs.nameservices'='your-nameservice','dfs.ha.namenodes.your-nameservice'='nn1,nn2','dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007','dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007','dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider');
CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hive.metastore.sasl.enabled' = 'true','hive.metastore.kerberos.principal' = 'your-hms-principal','dfs.nameservices'='your-nameservice','dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007','dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007','dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider','hadoop.security.authentication' = 'kerberos','hadoop.kerberos.keytab' = '/your-keytab-filepath/your.keytab','hadoop.kerberos.principal' = 'your-principal@YOUR.COM','yarn.resourcemanager.principal' = 'your-rm-principal');
BE
、FE
节点下放置 krb5.conf
文件和 keytab
认证文件,keytab
认证文件路径和配置保持一致,krb5.conf
文件默认放置在 /etc/krb5.conf
路径。
hive.metastore.kerberos.principal
的值需要和所连接的 hive metastore 的同名属性保持一致,可从 hive-site.xml
中获取。CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','dfs.encryption.key.provider.uri' = 'kms://http@kms_host:kms_port/kms');
CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hadoop.username' = 'root','fs.jfs.impl' = 'io.juicefs.JuiceFileSystem','fs.AbstractFileSystem.jfs.impl' = 'io.juicefs.JuiceFS','juicefs.meta' = 'xxx');
CREATE CATALOG hive PROPERTIES ("type"="hms","hive.metastore.type" = "glue","aws.region" = "us-east-1","aws.glue.access-key" = "ak","aws.glue.secret-key" = "sk","AWS_ENDPOINT" = "s3.us-east-1.amazonaws.com","AWS_REGION" = "us-east-1","AWS_ACCESS_KEY" = "ak","AWS_SECRET_KEY" = "sk","use_path_style" = "true");
# 1. 创建 ResourceCREATE RESOURCE hms_resource PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hadoop.username' = 'hive','dfs.nameservices'='your-nameservice','dfs.ha.namenodes.your-nameservice'='nn1,nn2','dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007','dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007','dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider');# 2. 创建 Catalog 并使用 Resource,这里的 Key Value 信息会覆盖 Resource 中的信息。CREATE CATALOG hive WITH RESOURCE hms_resource PROPERTIES('key' = 'value');
file.meta.cache.ttl-second
来设置 File Cache 自动失效时间,也可以将该值设置为 0 来禁用 File Cache。时间单位为:秒。示例如下:CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hadoop.username' = 'hive','dfs.nameservices'='your-nameservice','dfs.ha.namenodes.your-nameservice'='nn1,nn2','dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007','dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007','dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider','file.meta.cache.ttl-second' = '60');
CREATE CATALOG hive PROPERTIES ('type'='hms','hive.metastore.uris' = 'thrift://172.21.0.1:7004','hive.version' = '1.1.0');
HMS Type | Doris Type | Comment |
boolean | boolean | - |
tinyint | tinyint | - |
smallint | smallint | - |
int | int | - |
bigint | bigint | - |
date | date | - |
timestamp | datetime | - |
float | float | - |
double | double | - |
char | char | - |
varchar | varchar | - |
decimal | decimal | - |
array<type> | array<type> | 支持array嵌套,如 array<array<int>> |
map<KeyType, ValueType> | map<KeyType, ValueType> | 暂不支持嵌套,KeyType 和 ValueType 需要为基础类型 |
struct<col1: Type1, col2: Type2, ...> | struct<col1: Type1, col2: Type2, ...> | 暂不支持嵌套,Type1,Type2,... 需要为基础类型 |
other | unsupported | - |
"access_controller.properties.ranger.service.name" = "hive","access_controller.class" = "org.apache.doris.catalog.authorizer.RangerHiveAccessControllerFactory",
<?xml version="1.0" encoding="UTF-8"?><?xml-stylesheet type="text/xsl" href="configuration.xsl"?><configuration>#The directory for caching permission data, needs to be writable<property><name>ranger.plugin.hive.policy.cache.dir</name><value>/mnt/datadisk0/zhangdong/rangerdata</value></property>#The time interval for periodically pulling permission data<property><name>ranger.plugin.hive.policy.pollIntervalMs</name><value>30000</value></property><property><name>ranger.plugin.hive.policy.rest.client.connection.timeoutMs</name><value>60000</value></property><property><name>ranger.plugin.hive.policy.rest.client.read.timeoutMs</name><value>60000</value></property><property><name>ranger.plugin.hive.policy.rest.ssl.config.file</name><value></value></property><property><name>ranger.plugin.hive.policy.rest.url</name><value>http://172.21.0.32:6080</value></property><property><name>ranger.plugin.hive.policy.source.impl</name><value>org.apache.ranger.admin.client.RangerAdminRESTClient</value></property><property><name>ranger.plugin.hive.service.name</name><value>hive</value></property><property><name>xasecure.hive.update.xapolicies.on.grant.revoke</name><value>true</value></property></configuration>
本页内容是否解决了您的问题?