Support for Talos

[killcity]

With the dependency on OS binaries such as mount and mkdir, Trident cannot be used with more sophisticated/progressive operating systems like Talos. Will this dependency be lifted at some point?

[Untersander]

Would also be very interested to know if this would be possible at some point.

[Nauno33]

I am also interested in using Astra Trident on a Kubernetes cluster using Talos. Do you have a solution?

[k999o]

I'm working for a large retail(150+ ONTAP clusters)we'd really like to see Talos support. This keeps Netapp in the running as we vet out the best storage selection in our Kubernetes hybrid cloud environment.

We plan to move everything in that direction.

[stijoh]

Yes, this is needed. Shelling out on the nodes is not a good option. Getting rid of this dependency will benefit all Linux distros, not just Talos, as they would need much less tools installed on them.

[louhisuo]

+1 for Talos support.

[redbeard28]

+1 for Talos support.

[sempex]

+1 for Talos support.

[clementnuss]

for what it's worth, I managed to mount a trident share on talos, by using a debian:latest BASE image in the Dockerfile (cf this commit. through that, the basic binaries needed (e.g. mkdir, mount, mount.nfs) to mount NFS shares become accessible. This is not ideal as those binaries probably have some sort of correlation with the host kernel version, but for a workaround it does it.

There are some limitations though:

• NFSv3 with locks are not supported, because the rpc.statd daemon is not supported on Talos. that's all documented in NFS mount requires nolock mount option otherwise fails  siderolabs/talos#6582
• NFSv3 with the -o nolock mount option do not work either, I can't explain why. the error message is mount.nfs: Protocol not supported
• NFSv4 (i.e. mount option nfsvers=4) does work 🎉 and it seems I was able to use locks (tested that with flock) across different nodes.

I'm not yet sure if we are "ready" to change all our current workloads to NFSv4, I have to read this netapp article on the topic first, but at least we know that technically it is not fully impossible to mount a trident NFS share on talos.

[clementnuss]

TLDR; in theory it's possible, but it's tricky and I'm not going to invest more time in this for the time being.

here are my latest findings:

• at some point I thought that the fact the the nfsv3 or nfsv4 kernel modules couldn't be found was the reason for the Protocol not found error, but that didn't help. For reference, mounting /lib/modules (from Talos) on /lib/modules (trident-main container) makes those kernel mods discoverable by tools such as modinfo etc.
• the Protocol not supported error disappeared when I copied the /etc/protocols file from the kubelet rootfs to the trident-main container (the file was here to be precise: /run/containerd/io.containerd.runtime.v2.task/system/kubelet/rootfs/etc/protocols, thanks strace for finding that out)
• the nfs-utils binaries (which include mount.nfs, and rpc.statd) can be installed as described in this commit and they do work.
• NFSv3 with locks can work, provided you have start the rpcbind and rpc.statd daemons.

All of that being said, we are currently putting our trident exploration on hold, and might get back to this issue later. solving it would require:

1. building a system extension with the rpcbind and rpc.statd daemons, which is not trivial, partly because building those from scratch with the musl library requires some adaptations it appears.
2. starting those daemons in a dedicated pod, (e.g. in a daemonset with hostNetwork), however given how critical those daemons would be w.r.t. to locks, we do not want to adventure ourselves in this direction.

1 is much cleaner than 2, but requires too much development at this stage.

[paultrantelus]

+1 for Talos support.

[CaptainQwark]

+1 for Talos support

[torirevilla]

What protocols do you have an interest in being supported in Talos? This could help with prioritization.

[magicite]

We would use the following protocols in our talos environment:

1. NFSv4 (including pNFS)
2. NVMe/TCP