tencent cloud

PrevNext

Feedback

search by keyword

Recent Pages

Documentation
Tencent Cloud WeData
    Documentation
    Download PDF

    Metadata Collection

    Last updated: 2024-11-01 16:12:15
    Download PDF
      WeData provides users with a Definition metadata collection task feature. Administrators must collect metadata from the data source before they can perform visual management. Collection granularity supports libraries, and each database can only create one collection task. The collection task will run and update metadata information on a scheduled basis according to the configured cycle, and also supports manual execution, task editing, and other management operations.

      Supported Data Source Types

      Supported data source types for Metadata Collection:
      Information
      Type
      Big data
      Hive
      HBase
      DLC
      ClickHouse
      TCHouse-C
      Iceberg
      Greenplum
      Doris
      StarRocks
      TCHouse-D
      Relational database
      MySQL
      TencentDB for MySQL
      PostgreSQL
      Oracle
      SQL Server
      TCHouse-P
      TDSQL-PostgreSQL

      Collection Task Configuration

      Create Collection Task

      1. Go to the page Data Discovery > Metadata Collection and click Create Collection Task.
      2. Enter the Create Collection Task interface and select the data source type as Hive.
      
      
      
      3. Go to the Set Collection Targets page, fill in the following parameter information, and then click Next.
      
      
      
      Note:
      Each collection task supports binding to a single data source under the WeData project. Data sources cannot be bound to multiple collection tasks.
      Information
      Details
      Task Name
      The name of the collection task cannot be empty. Naming supports starting with letters or Chinese characters and can include letters, Chinese characters, numbers, hyphens (-), and underscores (_)
      Description
      Optional, description information of the collection task
      Affiliated Project
      Specify the project to which the data source belongs, binding the data source management permissions with the project
      Data Source
      The name of the data source corresponding to the collection task. This data source can be viewed in the project management module
      Database
      A database can only correspond to one collection task. Already collected databases cannot be selected
      Database Table
      Data tables to be collected
      Designated Table Owner
      Person in charge of data table management permissions
      Task Owner
      The responsible person has the permissions to view, stop, start, view logs, view details, rerun, and modify task information for this task
      Configure collection plan
      Configure the cycle, specific dates, and times for running the metadata collection task.
      Collection cycle: The current version supports hourly, daily, weekly, monthly, and one-time collection tasks.
      Collection date: Weekly and monthly tasks can specify one or more specific collection dates. After configuration, the task will run as planned on those days. For example, if the collection dates are set to the 1st, 5th, and 31st, the task will execute the metadata collection on these dates each month.
      Execution time: Specific execution time of the task
      Run immediately: After setting, the collection task will trigger once immediately after the task configuration is completed.

      Collection task list

      The collection task list provides information on all collection tasks under the current user, including task name, collection object, technical type, project affiliation, creator, etc. It supports operations such as viewing task details, logs, editing, deleting, and transferring.
      
      
      
      Information
      Details
      Task Name
      Collection task name
      Type
      Data source type
      Data Source
      Data source for the collection task
      Collection Database
      Collection database of the collection task
      Task Owner
      Account name of the current task responsible person
      Creator
      Account name that created the collection task
      Creation Time
      Task creation time
      Collection plan
      Collection task execution cycle
      Running Status
      Running status of the collection task
      Most recent execution time
      Date and time information of the last execution of the collection task
      Operation
      Provides details, editing, and deletion of the task, as well as a feature to view task details

      Run collection task

      Manually execute one-time/periodic tasks. Tasks not in execution state can be run manually.

      Editing collection task

      When not in execution state, you can edit items, data sources, update methods, deletion methods, and collection plans. Supports changes to the same type of data source. The collection task will collect data based on the newly bound collection object.

      Deleting collection task

      After deleting the collection task, collection for the data source will stop. In the current version, tables collected in WeData are not deleted.

      Transfer collection task

      In the collection task list, you can transfer the collection task to another task owner. The original owner will no longer have administrative privileges for the task.
      
      
      Contact Us

      Contact our sales team or business advisors to help your business.

      Contact Us
      Technical Support

      Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

      Submit a Ticket
      7x24 Phone Support
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Privacy PolicyLegalCookie Policy