tencent cloud

Background

NIO have been using public cloud storage for tens of petabytes of storage. Due to the increasing data increment, the data collected by vehicles and the data of the training platform need to access the public cloud increasing frequency, and the impact of network and storage on efficiency and cost is becoming more and more significant.

Also due to compliance requirements, sensitive data also needs to be stored in the local data center. Therefore, NIO choose to build their own private cloud storage to meet the data growth for the autonomous driving business.

Solution

Tstor OneCOS massive object storage adopts the homogenous architecture of Tencent Cloud Object Storage(COS), supports unlimited expansion of a single cluster and trillions of objects in a single bucket, easily meeting the large-capacity scale-out needs for private and hybrid cloud storage in the mobility industry.

As a distributed cloud storage foundation, TStor OneCOS provides a data lake or data center storage platforms for businesses such as autonomous driving. Through super storage and expansion capabilities, combined with big data solutions such as GooseFS, it supports data preparation, model training and reasoning full process data service.

Choosing TStor OneCOS massive object storage to build a cloud storage pool for the autonomous driving platform, NIO built a solution, combined with GooseFS, to meet data training and other needs.

Using EC 8+4 redundantly achieve high balance of space utilization and reliability. The fully self-developed YottaStore distributed storage adopts Top Return, log cache and other mechanisms to ensure performance, and ensure auto data balance and repair, so as to achieve rapid expansion without affecting business.

Using the StarLake server, and the end-to-end deep software and hardware adaptation and optimization, NIO secured a stable and energy-saving server solution. Multiple nodes are used to form a storage pool of hundreds PB, which greatly improves density and reduces costs.

Combined with GooseFS to accelerate the performance of massive data analysis, machine learning, artificial intelligence and other workloads, it solves the demand for massive small files and HDFS file access and realizes intelligent data preheating and higher performance access nearby.

Benefits

Using TStor OneCOS to build an autonomous driving cloud storage platform, the capacity, performance and cost requirements of NIO have been solved, meeting the needs of business growth, and realizing accelerated business expansion.

• In terms of capacity, it supports NIO’s EB-scale autonomous driving data storage and rapid growth, preparing NIO for upgrading autonomous driving models to L4 or L5 in the future.
• In terms of performance, the globally unified distributed metadata cluster and efficient load balancing gateway realize 1 trillion+ file indexes, support 100GB+ read and write bandwidth and 1 million+ request QPS access, and support on-demand expansion performance.
• In terms of reliability and maintenance, the ultra-high reliability of 12 nines is achieved, and data could be quickly restored when nodes are repaired from abnormal, ensuring rapid capacity expansion without perception, and the average speed of customer site scale expansion reaches 20GB/s.
• In terms of cost, the total cost of hardware resources is reduced by more than 50%, while reducing cabinet space consumption.