Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions docs/cli/certificate.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,10 +81,6 @@ The command performs integrity checks to confirm that the updated certificates a
If the updated data is acceptable, the datastore's encrypted bootstrap key is updated, and the new certificates and keys will be used the next time K3s starts.
If problems are encountered while validating the certificates and keys, an error is reported to the system log and the operation is cancelled without changes.

:::info Version Gate
Support for the `k3s certificate rotate-ca` command and the ability to use CA certificates signed by an external CA is available starting with the 2023-02 releases (v1.26.2+k3s1, v1.25.7+k3s1, v1.24.11+k3s1, v1.23.17+k3s1).
:::

### Using Custom CA Certificates

#### Cautions Against CA Reuse
Expand Down
4 changes: 0 additions & 4 deletions docs/cli/etcd-snapshot.md
Original file line number Diff line number Diff line change
Expand Up @@ -307,10 +307,6 @@ The token value can also be set in the K3s config file.

## ETCDSnapshotFile Custom Resources

:::info Version Gate
ETCDSnapshotFiles are available as of the November 2023 releases: v1.28.4+k3s2, v1.27.8+k3s2, v1.26.11+k3s2, v1.25.16+k3s4
:::

Snapshots can be viewed remotely using any Kubernetes client by listing or describing cluster-scoped `ETCDSnapshotFile` resources.
Unlike the `k3s etcd-snapshot list` command, which only shows snapshots visible to that node, `ETCDSnapshotFile` resources track all snapshots present on cluster members.

Expand Down
4 changes: 0 additions & 4 deletions docs/cli/token.md
Original file line number Diff line number Diff line change
Expand Up @@ -143,10 +143,6 @@ Flag | Description

#### `k3s token rotate`

:::info Version Gate
Available as of the October 2023 releases (v1.28.2+k3s1, v1.27.7+k3s1, v1.26.10+k3s1, v1.25.15+k3s1).
:::

Rotate original server token with a new server token. After running this command, all servers and any agents that originally joined with the old token must be restarted with the new token.

If you do not specify a new token, one will be generated for you.
Expand Down
6 changes: 0 additions & 6 deletions docs/installation/requirements.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,6 @@ K3s is available for the following architectures:
- armhf
- arm64/aarch64

:::warning ARM64 Page Size

Prior to May 2023 releases (v1.24.14+k3s1, v1.25.10+k3s1, v1.26.5+k3s1, v1.27.2+k3s1), on `aarch64/arm64` systems, the kernel must use 4k pages. **RHEL9**, **Ubuntu**, **Raspberry PI OS**, and **SLES** all meet this requirement.

:::

## Operating Systems

K3s is expected to work on most modern Linux systems.
Expand Down
73 changes: 0 additions & 73 deletions docs/known-issues.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,79 +36,6 @@ Iptables versions 1.8.0-1.8.4 also have known issues that can cause K3s to fail.

K3s includes a known-good version of iptables (v1.8.8) which has been tested to function properly. You can tell K3s to use its bundled version of iptables by starting K3s with the `--prefer-bundled-bin` option, or by uninstalling the iptables/nftables packages from your operating system.

:::info Version Gate

The `--prefer-bundled-bin` flag is available starting with the 2022-12 releases (v1.26.0+k3s1, v1.25.5+k3s1, v1.24.9+k3s1, v1.23.15+k3s1).

:::

### Rootless Mode

Running K3s with Rootless mode is experimental and has several [known issues.](./advanced.md#known-issues-with-rootless-mode)

### Upgrading Hardened Clusters from v1.24.x to v1.25.x {#hardened-125}

Kubernetes removed PodSecurityPolicy from v1.25 in favor of Pod Security Standards. You can read more about PSS in the [upstream documentation](https://kubernetes.io/docs/concepts/security/pod-security-standards/). For K3S, there are some manual steps that must be taken if any `PodSecurityPolicy` has been configured on the nodes.

1. On all nodes, update the `kube-apiserver-arg` value to remove the `PodSecurityPolicy` admission-plugin. Add the following arg value instead: `'admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml'`, but do NOT restart or upgrade K3S yet. Below is an example of what a configuration file might look like after this update for the node to be hardened:
```yaml
protect-kernel-defaults: true
secrets-encryption: true
kube-apiserver-arg:
- 'admission-control-config-file=/var/lib/rancher/k3s/server/psa.yaml'
- 'audit-log-path=/var/lib/rancher/k3s/server/logs/audit.log'
- 'audit-policy-file=/var/lib/rancher/k3s/server/audit.yaml'
- 'audit-log-maxage=30'
- 'audit-log-maxbackup=10'
- 'audit-log-maxsize=100'
kube-controller-manager-arg:
- 'terminated-pod-gc-threshold=10'
- 'use-service-account-credentials=true'
kubelet-arg:
- 'streaming-connection-idle-timeout=5m'
```
2. Create the `/var/lib/rancher/k3s/server/psa.yaml` file with the following contents. You may want to exempt more namespaces as well. The below example exempts `kube-system` (required), `cis-operator-system` (optional, but useful for when running security scans through Rancher), and `system-upgrade` (required if doing [Automated Upgrades](./upgrades/automated.md)).
```yaml
apiVersion: apiserver.config.k8s.io/v1
kind: AdmissionConfiguration
plugins:
- name: PodSecurity
configuration:
apiVersion: pod-security.admission.config.k8s.io/v1beta1
kind: PodSecurityConfiguration
defaults:
enforce: "restricted"
enforce-version: "latest"
audit: "restricted"
audit-version: "latest"
warn: "restricted"
warn-version: "latest"
exemptions:
usernames: []
runtimeClasses: []
namespaces: [kube-system, cis-operator-system, system-upgrade]
```
3. Perform the upgrade as normal. If doing [Automated Upgrades](./upgrades/automated.md), ensure that the namespace where the `system-upgrade-controller` pod is running in is setup to be privileged in accordance with the [Pod Security levels](https://kubernetes.io/docs/concepts/security/pod-security-admission/#pod-security-levels):
```yaml
apiVersion: v1
kind: Namespace
metadata:
name: system-upgrade
labels:
# This value must be privileged for the controller to run successfully.
pod-security.kubernetes.io/enforce: privileged
pod-security.kubernetes.io/enforce-version: v1.25
# We are setting these to our _desired_ `enforce` level, but note that these below values can be any of the available options.
pod-security.kubernetes.io/audit: privileged
pod-security.kubernetes.io/audit-version: v1.25
pod-security.kubernetes.io/warn: privileged
pod-security.kubernetes.io/warn-version: v1.25
```
4. After the upgrade is complete, remove any remaining PSP resources from the cluster. In many cases, there may be PodSecurityPolicies and associated RBAC resources in custom files used for hardening within `/var/lib/rancher/k3s/server/manifests/`. Remove those resources and k3s will update automatically. Sometimes, due to timing, some of these may be left in the cluster, in which case you will need to delete them manually. If the [Hardening Guide](./security/hardening-guide.md) was previously followed, you should be able to delete them via the following:
```sh
# Get the resources associated with PSPs
$ kubectl get roles,clusterroles,rolebindings,clusterrolebindings -A | grep -i psp

# Delete those resources:
$ kubectl delete clusterrole.rbac.authorization.k8s.io/psp:restricted-psp clusterrole.rbac.authorization.k8s.io/psp:svclb-psp clusterrole.rbac.authorization.k8s.io/psp:system-unrestricted-psp clusterrolebinding.rbac.authorization.k8s.io/default:restricted-psp clusterrolebinding.rbac.authorization.k8s.io/system-unrestricted-node-psp-rolebinding && kubectl delete -n kube-system rolebinding.rbac.authorization.k8s.io/svclb-psp-rolebinding rolebinding.rbac.authorization.k8s.io/system-unrestricted-svc-acct-psp-rolebinding
```
4 changes: 0 additions & 4 deletions docs/upgrades/automated.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,10 +149,6 @@ Jobs to execute upgrades for a plan will not be created outside the time window.

## Downgrade Prevention

:::info Version Gate
Starting with the 2023-07 releases ([v1.27.4+k3s1](https://github.com/k3s-io/k3s-upgrade/releases/tag/v1.27.4%2Bk3s1), [v1.26.7+k3s1](https://github.com/k3s-io/k3s-upgrade/releases/tag/v1.26.7%2Bk3s1), [v1.25.12+k3s1](https://github.com/k3s-io/k3s-upgrade/releases/tag/v1.25.12%2Bk3s1), [v1.24.16+k3s1](https://github.com/k3s-io/k3s-upgrade/releases/tag/v1.24.16%2Bk3s1))
:::

Kubernetes does not support downgrades of control-plane components. The k3s-upgrade image used by upgrade plans will refuse to downgrade K3s, failing the plan. Nodes with `cordon: true` configured in their plan will stay cordoned following the failure.

Here is an example cluster, showing failed upgrade pods and cordoned nodes:
Expand Down
Loading