tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Cloud WeData
    Documentation
    Download PDF

    DLC (DLC)

    Last updated: 2024-11-01 16:26:14
    Download PDF
      WeData supports the integration of the DLC computational storage separation engine, offering DLC Database/Table Management features, and enabling agile and efficient data lake analysis and computation. Leveraging Spark and Presto capabilities, you can perform federated analysis and computing with standard SQL on COS services (COS) and multi-source databases.

      Background

      Tencent Cloud DLC (Data Lake Compute, DLC) provides agile and efficient managed data lake analysis and computing services. Users do not need to perform traditional data layer modeling, significantly reducing the preparation time for massive data analysis and effectively enhancing enterprise data agility.

      Use Limits

      Limit
      Restrictions
      DLC
      WeData currently supports data management, analytical querying, and computing tasks for DLC Type Library Tables.
      The DLC currently supports the following computational engine versions:
      Spark SQL:SuperSQL-S 1.0
      Spark Job: Spark 2.4, Spark 3.2
      Presto:SuperSQL-P 1.0
      WeData
      The types of DLC tasks supported in WeData data development include: DLC SQL, DLC Spark.
      WeData supports the creation of DLC tables and DLC functions.

      Getting Started

      The main process of using DLC in WeData includes the following steps:

      Preparations

      Preparation Category
      Operation Description
      Reference Links
      DLC
      To ensure the smooth use of DLC-related table creation, data development, and data exploration features in WeData, the DLC cluster must meet basic configuration requirements. For example, to use the Spark job engine in WeData with DLC, you need to create a Spark job engine in DLC and grant the corresponding user permission to use the engine.
      Create and manage the DLC engine: Data Engine Description.
      WeData
      Bind the DLC cluster and obtain the latest cluster configuration from the DLC cluster.
      By default, new projects will automatically use dynamic keys to communicate with DLC.
      Data permissions and engine permission management: DLC Permission Management.
      

      Task Development

      Creating workflow

      Task development is based on data workflow orchestration to achieve the procedural execution of computing tasks. Before creating computing tasks, you need to create a data workflow and then orchestrate the computing task execution process within the workflow.

      Creating a DLC Node

      WeData is based on the DLC engine for task development. After binding the DLC cluster with a project in WeData, the DLC system data source will be integrated into WeData. Currently, the Orchestration Space DLC SQL only supports the DLC system source.

      Task Development

      After binding the DLC engine to the WeData project, create the type of computing tasks supported by DLC in the already created data workflow. During the configuration process of the task node, use the system data source provided by DLC for task development and debugging.

      Submitting the job

      After configuring the DLC system source data and ensuring it is correct, save the corresponding computing task. When the computing task is submitted and released, it can be scheduled and run in the Operation and Maintenance Center.

      Related Operations

      After completing DLC task development, you can perform DLC metadata management, task operation and maintenance monitoring, and data quality monitoring in WeData to ensure the normal production of DLC data. You can also conduct multi-source joint queries and data analysis in the Data Exploration feature.
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy