Data Lake Compute
- Getting Started
- Operation Guide
- Console Operation Introduction
- Data Development and Exploration
- Data Exploration
- Data Query Task
- Resource Management
- Engine Management
- SuperSQL Engine
- Standard Engine
- Resource Group
- Private Connection
- Storage Configuration
- Ops Management
- Insight Management
- System Management
- User and Permission Management
- Monitoring and Alarms
- Development Guide
- System Restraints
- Client Access
- JDBC Access
- Practical Tutorial
- SQL Statement
- SuperSQL Statement
- Unified Statement
- DDL Statement
- ALTER TABLE
- DML Statement
- DQL Statement
- Iceberg Table Statement
- API Documentation
- Making API Requests
- Data Table APIs
- Task APIs
- Metadata APIs
- Service Configuration APIs
- Permission Management APIs
- Data Source Connection APIs
- Data Optimization APIs
- Data Engine APIs
- General Reference
- Operation Guide on Connecting Third-Party Software to DLC
DocumentationData Lake ComputeOperation GuideConsole Operation IntroductionResource ManagementEngine ManagementData Engine Introduction
Data Engine Introduction
Last updated: 2025-04-15 16:25:35
The DLC data engine is the foundation of DLC's data analysis and computation services. All calculations performed by users within DLC require the use of this data engine. Depending on the specific use case, users can select the appropriate engine type.
Engine Types
DLC offers two types of data engines for users to choose from: Standard Engine and SuperSQL Engine. The primary difference between these two engines lies in the SQL syntax they support. The Standard Engine uses native Spark and Presto syntax from the community, while the SuperSQL Engine supports DLC's independently developed unified syntax. This unified SuperSQL syntax can run on both Spark and Presto engines, effectively masking the syntax differences between them. This feature can significantly reduce usage costs in scenes where different analytics engines need to be used together. Below are the main characteristics of each engine and recommendations for selection:
Engine Types | Available Types | Main Features | Usage Requirements | Purchase Recommendations |
Standard Engine | Spark Presto | Native syntax: Uses the native syntax from the Spark/Presto community, ensuring low learning and migration costs. Flexible usage: Supports both Hive JDBC and Presto JDBC. Integrated Spark: The standard Spark engine can execute SQL and Spark batch tasks. | Currently, a 2 CU specification free gateway is provided. If you need to upgrade the specification, upgrade the Gateway | 1. Require the use of native Spark/Presto syntax. 2. Need to purchase a Spark engine for batch processing and offline SQL tasks. 3. Prefer to use Hive JDBC and Presto JDBC. |
SuperSQL Engine | SparkSQL Spark jobs\\nPresto | Unified syntax: A set of syntax applies to both Spark and Presto engines. Supports federated queries. | You need to learn the SuperSQL unified syntax.\\nFor SQL/batch task scenes, it is recommended to purchase the corresponding engine type. | 1. Prefer to use a unified syntax for both Spark and Presto. 2. Need to perform federated queries. |
For more detailed information, see the comparison table below or review the documentation for the Standard Engine and SuperSQL Engine Description.
Detailed Comparison of Standard Engine and SuperSQL Engine
Feature | Standard Engine | SuperSQL Engine | Description |
Presto | ✓ | ✓ | Both engines support the Presto engine. |
Spark | ✓ | ✓ | The SuperSQL Engine is divided into SparkSQL and Spark job. The SparkSQL engine supports SQL jobs, while the Spark job engine supports Spark batch and streaming jobs as well as SQL jobs. The Standard Engine is an integrated Spark engine. |
Native syntax | Unified syntax | The Standard Engine supports native Spark and Presto syntax. The SuperSQL Engine supports DLC's self-developed unified syntax. | |
✓ | | DLC, based on Apache Kyuubi, has developed its own Serverless gateway service, providing a more stable, secure, and high-performance task submission experience. | |
✓ | | Resource groups are a unique feature of the Standard Spark Engine, allowing resources to be allocated as needed. SQL tasks can be submitted to a designated resource group for execution. | |
Shared Engine | | ✓ | The SuperSQL Engine supports a shared mode, which is suitable for scenes with low analysis frequency and smaller data volumes. |
✓ | | The Standard Engine supports submitting tasks using Hive JDBC. | |
✓ | | The Standard Engine supports submitting tasks using Presto JDBC. | |
✓ | ✓ | Both types of engines support submitting tasks using DLC JDBC. | |
TencentCloud API Task Submission | ✓ | ✓ | Both types of engines support submitting tasks using TencentCloud API or through the data exploration page in the console. |
Federated Query | | ✓ | The SuperSQL Engine provides federated query analysis capabilities. For instructions on adding a federated query data catalog, see Data Directory and DMC. The Standard Engine currently does not support federated queries. |
If you have any questions about choosing between the Standard Engine or SuperSQL Engine, you can Submit a Ticket to contact us.
Engine Pricing
Data engines support both monthly subscription and pay-as-you-go subscription. For more information, see Billing Overview.
Limitations
The name of the data engine should be globally unique and cannot be changed.
The billing mode of the data engine cannot be switched.
The data engine does not support changing regions.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No