tencent cloud

Feedback

Configuring Hive Execution Engine

Last updated: 2024-10-30 11:34:16
    Hive supports using MapReduce, Tez, and Spark as execution engines, with MapReduce as the default in EMR. This document will show you how to configure the execution engine in Hive.
    It is generally recommended to use the Tez execution engine instead of MapReduce for improved computation efficiency.

    Development Preparation

    Make sure you have activated Tencent Cloud and created an EMR cluster. For more details, see Creating a Cluster.
    When an EMR cluster is created, select the Hive component in the software configuration interface. If you need to switch to the Tez or Spark execution engine, check the corresponding component when creating the cluster.

    Configuring Engines

    Log in to the Hive client, following the directions in Hive Connection Method. The execution engine for Hive is configured using the hive.execution.engine parameter, with the default engine being mr. You can use the following command to check the current execution engine:
    hive> set hive.execution.engine;
    hive.execution.engine=mr

    Switching Execution Engine in the Command Line

    Use the following command to modify the execution engine for the current client session:
    Modify the execution engine to Tez:
    hive> set hive.execution.engine=tez;
    Modify the execution engine to Spark:
    hive> set hive.execution.engine=spark;
    After execution, you can enter the following command to verify:
    hive> set hive.execution.engine;
    If the returned information matches the execution engine you set, it indicates that the configuration is successful.
    Note:
    The above execution engine switch operation is only effective for the current client session; the default execution engine will be used when accessing by other clients or after disconnecting and reconnecting.

    Switching Execution Engine in the Configuration File

    You can globally switch the default execution engine through the EMR console by navigating to Cluster Services > HIVE > Configuration Management. Locate the hive-site.xml configuration file and set the following parameter value:
    Modify the execution engine to Tez:
    hive.execution.engine = tez
    Modify the execution engine to Spark:
    hive.execution.engine = sparkP
    Perform Save Configuration > Save and Deploy, and Restart HiveServer2. After the restart is completed, the default execution engine of Hive will be modified.
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support