Elastic Kubernetes Service (EKS) is a TKE service mode that allows you to deploy workloads without purchasing any nodes. EKS is fully compatible with native Kubernetes, allowing you to purchase and manage resources natively. This service is billed based on the actual amount of resources used by containers. In addition, EKS provides extended support for Tencent Cloud products, such as storage and network products, and can ensure the secure isolation of containers. EKS is ready to use out-of-the-box.
Deploying GooseFS with Tencent Cloud EKS can make full use of the elastic computing resources from EKS and construct an on-demand, pay-as-you-go COS access acceleration service billed on a per second basis.
Architecture
The figure below shows the general architecture of deploying GooseFS with Tencent EKS.
As shown in the figure, the entire architecture consists of three parts: EKS Managed Components, User Resource Pool, and COS Server. User Resource Pool is mainly used to deploy GooseFS clusters, and COS Server is used as a remote storage system and can be replaced by CHDFS, a public cloud storage service. During the construction process:
Both GooseFS Master and Worker are deployed as Kubernetes StatefulSet.
Fluid is used to start a GooseFS cluster.
Fuse Client is integrated into the sandbox of the User Pod.
The usage method is the same as standard Kubernetes.
# A required parameter, which sets the VM specification of pods. Default value: 1c1g
resources:
requests:
cpu:8
memory:"16Gi"
limits:
cpu:8
memory:"16Gi"
replicas:1
# journal:
# volumeType: pvc
# storageClass: goosefs-hdd
jvmOptions:
-"-Xmx12G"
-"-XX:+UnlockExperimentalVMOptions"
-"-XX:ActiveProcessorCount=8"
-"-Xms10G"
worker:
jvmOptions:
-"-Xmx28G"
-"-Xms28G"
-"-XX:+UnlockExperimentalVMOptions"
-"-XX:MaxDirectMemorySize=28g"
-"-XX:ActiveProcessorCount=8"
resources:
requests:
cpu:16
memory:"32Gi"
limits:
cpu:16
memory:"32Gi"
fuse:
jvmOptions:
-"-Xmx4G"
-"-Xms4G"
-"-XX:+UseG1GC"
-"-XX:MaxDirectMemorySize=4g"
-"-XX:+UnlockExperimentalVMOptions"
-"-XX:ActiveProcessorCount=24"
5. Check the statuses of the cluster and PVC:
-> goosefs kubectl get pod
NAME READY STATUS RESTARTS AGE
slice1-master-0 2/2 Running 0 8m8s
slice1-worker-0 2/2 Running 0 8m8s
slice1-worker-1 2/2 Running 0 8m8s
slice1-worker-2 2/2 Running 0 8m8s
slice1-worker-3 2/2 Running 0 8m8s
slice1-worker-4 2/2 Running 0 8m8s
slice1-worker-5 2/2 Running 0 8m8s
-> goosefs kubectl get pvc
slice1 Bound default-slice1 100Gi ROX fluid 7m37s # PVC and dataset share the same name. 100Gi is a dummy value used as a placeholder.
Loading data
To preload data, you only need to create a resource with the yaml below (e.g. kubectl apply -f dataload.yaml). A response example after running is as follows:
apiVersion: data.fluid.io/v1alpha1
kind: DataLoad
metadata:
name: slice1-dataload
spec:
# Configuring the dataset that needs to load data
dataset:
name: slice1
namespace: default
After creating, you can check the status via kubectl get dataload slice1-dataload.
Mounting PVC to a service pod
The user service container should be used in accordance to the K8s instruction. Refer to Kubernetes documentation for details.
Terminating a GooseFS cluster
To terminate a GooseFS cluster, you can specify the master and worker nodes to be deleted and run the delete command. This is a high-risk operation. Make sure that there is no I/O operation on GooseFS in the service pod.
