Feature Request: use a storage class for data volume instead of host dir

[stephen-dahl]

Optionally use a stateful with PVCs for the StorageOS cluster. You already mount a local volume to each pod in the StorageOS cluster. Replacing that with another storage class would allow the cluster to use our cloud providers storage class to back StorageOS.

This has the following benefits:

• Simpler setup since nodes do not have to be provisioned with extra storage ahead of time.
• abstracts the hosts completely from the storage cluster.
• makes adding more storage to the cluster as simple as scaling up the stateful set.

[croomes]

Hi, this is a really interesting idea. Would it work for you if the nodes that provided storage ran as a StatefulSet, and the remaining nodes were provisioned as clients by a DaemonSet with anti-affinity on the storage nodes?

For provisioning the cluster initially, would it be enough to specify the number of storage nodes required and the PVC size? Just thinking how to expose this via the CR or Helm without making it too complicated.

Also need to think a bit more about how you'd convert a node from a storage provider to a client and vice versa.

Thanks for the request!

[stephen-dahl]

From your comment I am assuming that the current daemon set has logic that must run on every node. if that is the case then the best thing eventually would probably be to extract that logic into its own daemon set. this would make the data pods placement much more flexible and you don't have to worry about converting nodes as the stateful set grows.

This could probably be done in phases.

1. Current system + storage class option.
2. Second daemon set.

These are the options that it might eventually need.

storageProvider: local/pvc
storageClass: gp2/other (just not one provided by storageos)
storageVolumeSize: 100Gi
storageDeploymentType: DaemonSet/StatefulSet

I bring this up because I have been researching the different storage options. I think it would make StorageOS more plug and play and the clear winner for my clusters.