diff --git a/etcd/README.md b/etcd/README.md new file mode 100644 index 00000000000..2a2895bc945 --- /dev/null +++ b/etcd/README.md @@ -0,0 +1,148 @@ +# etcd.openshift.io API Group + +This API group contains CRDs related to etcd cluster management in Two Node OpenShift with Fencing deployments. + +## API Versions + +### v1alpha1 + +Contains the `PacemakerCluster` custom resource for monitoring Pacemaker cluster health in Two Node OpenShift with Fencing deployments. + +#### PacemakerCluster + +- **Feature Gate**: `DualReplica` +- **Component**: `two-node-fencing` +- **Scope**: Cluster-scoped singleton resource (must be named "cluster") +- **Resource Path**: `pacemakerclusters.etcd.openshift.io` + +The `PacemakerCluster` resource provides visibility into the health and status of a Pacemaker-managed cluster. +It is periodically updated by the cluster-etcd-operator's status collector. + +### Pacemaker Resources + +A **pacemaker resource** is a unit of work managed by pacemaker. In pacemaker terminology, resources are services +or applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + +For Two Node OpenShift with Fencing, we manage three resources: +- **Kubelet**: The Kubernetes node agent and a prerequisite for etcd +- **Etcd**: The distributed key-value store +- **FencingAgent**: Used to isolate failed nodes during a quorum loss event + +### Status Structure + +> **Note on optional fields:** Per Kubernetes API conventions, all status fields are marked `+optional` in the Go +> type definitions to allow for partial updates and backward compatibility. However, CEL validation rules enforce +> that certain fields (like conditions and lastUpdated) cannot be removed once set. This means these fields are +> "optional on initial creation, but required once present." +> See: [kube-api-linter statusoptional](https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go) + +```yaml +status: + conditions: # Cluster-level conditions (required once set, min 3 items) + - type: Healthy + - type: InService + - type: NodeCountAsExpected + lastUpdated: # When status was last updated (required once set, cannot decrease) + nodes: # Per-node status (0-32 nodes, expects 2) + - name: # RFC 1123 subdomain name + ipAddresses: # List of global unicast IPv4 or IPv6 addresses in canonical form + - # First address used for etcd peer URLs + conditions: # Node-level conditions (required once set, min 7 items) + - type: Healthy + - type: Online + - type: InService + - type: Active + - type: Ready + - type: Clean + - type: Member + resources: # Array of pacemaker resources on this node (required once set, min 3 items) + - name: Kubelet # All three resources (Kubelet, Etcd, FencingAgent) must be present + conditions: # Resource-level conditions (required once set, min 8 items) + - type: Healthy + - type: InService + - type: Managed + - type: Enabled + - type: Operational + - type: Active + - type: Started + - type: Schedulable + - name: Etcd + conditions: [] + - name: FencingAgent + conditions: [] +``` + +### Cluster-Level Conditions + +**All three conditions are required once the status is populated.** + +| Condition | True | False | +|-----------|------|-------| +| `Healthy` | Cluster is healthy (`ClusterHealthy`) | Cluster has issues (`ClusterUnhealthy`) | +| `InService` | In service (`InService`) | In maintenance (`InMaintenance`) | +| `NodeCountAsExpected` | Node count is as expected (`AsExpected`) | Wrong count (`InsufficientNodes`, `ExcessiveNodes`) | + +### Node-Level Conditions + +**All seven conditions are required for each node once the status is populated.** + +| Condition | True | False | +|-----------|------|-------| +| `Healthy` | Node is healthy (`NodeHealthy`) | Node has issues (`NodeUnhealthy`) | +| `Online` | Node is online (`Online`) | Node is offline (`Offline`) | +| `InService` | In service (`InService`) | In maintenance (`InMaintenance`) | +| `Active` | Node is active (`Active`) | Node is in standby (`Standby`) | +| `Ready` | Node is ready (`Ready`) | Node is pending (`Pending`) | +| `Clean` | Node is clean (`Clean`) | Node is unclean (`Unclean`) | +| `Member` | Node is a member (`Member`) | Not a member (`NotMember`) | + +### Resource-Level Conditions + +Each resource in the `resources` array has its own conditions. **All eight conditions are required for each resource once the status is populated.** + +| Condition | True | False | +|-----------|------|-------| +| `Healthy` | Resource is healthy (`ResourceHealthy`) | Resource has issues (`ResourceUnhealthy`) | +| `InService` | In service (`InService`) | In maintenance (`InMaintenance`) | +| `Managed` | Managed by pacemaker (`Managed`) | Not managed (`Unmanaged`) | +| `Enabled` | Resource is enabled (`Enabled`) | Resource is disabled (`Disabled`) | +| `Operational` | Resource is operational (`Operational`) | Resource has failed (`Failed`) | +| `Active` | Resource is active (`Active`) | Resource is not active (`Inactive`) | +| `Started` | Resource is started (`Started`) | Resource is stopped (`Stopped`) | +| `Schedulable` | Resource is schedulable (`Schedulable`) | Resource is not schedulable (`Unschedulable`) | + +### Validation Rules + +**Resource naming:** +- Resource name must be "cluster" (singleton) + +**Node name validation:** +- Must be a lowercase RFC 1123 subdomain name +- Consists of lowercase alphanumeric characters, '-' or '.' +- Must start and end with an alphanumeric character +- Maximum 253 characters + +**IP address validation:** +- Pacemaker allows multiple IP addresses for Corosync communication between nodes (1-8 addresses) +- The first address in the list is used for IP-based peer URLs for etcd membership +- Each address must be in canonical form (e.g., `192.168.1.1` not `192.168.001.001`, or `2001:db8::1` not `2001:0db8::1`) +- Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses) +- Excludes loopback, link-local, and multicast addresses + +**Timestamp validation:** +- `lastUpdated` is optional on initial creation, but once set it cannot be removed and must always increase (prevents stale updates) + +**Required conditions (once status is populated):** +- All cluster-level conditions must be present (MinItems=3) +- All node-level conditions must be present for each node (MinItems=7) +- All resource-level conditions must be present for each resource in the `resources` array (MinItems=8) + +**Resource names:** +- Valid values are: `Kubelet`, `Etcd`, `FencingAgent` +- All three resources must be present in each node's `resources` array + +### Usage + +The cluster-etcd-operator healthcheck controller watches this resource and updates operator conditions based on +the cluster state. The aggregate `Healthy` conditions at each level (cluster, node, resource) provide a quick +way to determine overall health. diff --git a/etcd/install.go b/etcd/install.go new file mode 100644 index 00000000000..7e7474152c4 --- /dev/null +++ b/etcd/install.go @@ -0,0 +1,26 @@ +package etcd + +import ( + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" + + v1alpha1 "github.com/openshift/api/etcd/v1alpha1" +) + +const ( + GroupName = "etcd.openshift.io" +) + +var ( + schemeBuilder = runtime.NewSchemeBuilder(v1alpha1.Install) + // Install is a function which adds every version of this group to a scheme + Install = schemeBuilder.AddToScheme +) + +func Resource(resource string) schema.GroupResource { + return schema.GroupResource{Group: GroupName, Resource: resource} +} + +func Kind(kind string) schema.GroupKind { + return schema.GroupKind{Group: GroupName, Kind: kind} +} diff --git a/etcd/v1alpha1/Makefile b/etcd/v1alpha1/Makefile new file mode 100644 index 00000000000..3d019662af6 --- /dev/null +++ b/etcd/v1alpha1/Makefile @@ -0,0 +1,3 @@ +.PHONY: test +test: + make -C ../../tests test GINKGO_EXTRA_ARGS=--focus="etcd.openshift.io/v1alpha1" diff --git a/etcd/v1alpha1/doc.go b/etcd/v1alpha1/doc.go new file mode 100644 index 00000000000..aea92fb381a --- /dev/null +++ b/etcd/v1alpha1/doc.go @@ -0,0 +1,6 @@ +// +k8s:deepcopy-gen=package,register +// +k8s:defaulter-gen=TypeMeta +// +k8s:openapi-gen=true +// +openshift:featuregated-schema-gen=true +// +groupName=etcd.openshift.io +package v1alpha1 diff --git a/etcd/v1alpha1/register.go b/etcd/v1alpha1/register.go new file mode 100644 index 00000000000..1dc6482f832 --- /dev/null +++ b/etcd/v1alpha1/register.go @@ -0,0 +1,39 @@ +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" + "k8s.io/apimachinery/pkg/runtime" + "k8s.io/apimachinery/pkg/runtime/schema" +) + +var ( + GroupName = "etcd.openshift.io" + GroupVersion = schema.GroupVersion{Group: GroupName, Version: "v1alpha1"} + schemeBuilder = runtime.NewSchemeBuilder(addKnownTypes) + // Install is a function which adds this version to a scheme + Install = schemeBuilder.AddToScheme + + // SchemeGroupVersion generated code relies on this name + // Deprecated + SchemeGroupVersion = GroupVersion + // AddToScheme exists solely to keep the old generators creating valid code + // DEPRECATED + AddToScheme = schemeBuilder.AddToScheme +) + +// Resource generated code relies on this being here, but it logically belongs to the group +// DEPRECATED +func Resource(resource string) schema.GroupResource { + return schema.GroupResource{Group: GroupName, Resource: resource} +} + +func addKnownTypes(scheme *runtime.Scheme) error { + metav1.AddToGroupVersion(scheme, GroupVersion) + + scheme.AddKnownTypes(GroupVersion, + &PacemakerCluster{}, + &PacemakerClusterList{}, + ) + + return nil +} diff --git a/etcd/v1alpha1/tests/pacemakerclusters.etcd.openshift.io/DualReplica.yaml b/etcd/v1alpha1/tests/pacemakerclusters.etcd.openshift.io/DualReplica.yaml new file mode 100644 index 00000000000..11909113892 --- /dev/null +++ b/etcd/v1alpha1/tests/pacemakerclusters.etcd.openshift.io/DualReplica.yaml @@ -0,0 +1,1192 @@ +apiVersion: apiextensions.k8s.io/v1 # Hack because controller-gen complains if we don't have this +name: "PacemakerCluster" +crdName: pacemakerclusters.etcd.openshift.io +featureGate: DualReplica +tests: + onCreate: + - name: Should be able to create a minimal PacemakerCluster + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + expected: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + - name: Should reject PacemakerCluster with wrong name + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: wrong-name + expectedError: "PacemakerCluster must be named 'cluster'" + onUpdate: + # Full healthy cluster status test + - name: Should accept full healthy cluster status with all required conditions + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + updated: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ClusterHealthy + message: "Cluster is healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected nodes present" + lastUpdated: "2024-01-01T00:00:01Z" + nodes: + - name: node1.example.com + ipAddresses: + - "192.168.1.1" + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: NodeHealthy + message: "Node healthy" + - type: Online + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Online + message: "Online" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Ready + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Ready + message: "Ready" + - type: Clean + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Clean + message: "Clean" + - type: Member + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Member + message: "Member" + resources: + - name: Kubelet + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: Etcd + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: FencingAgent + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + expected: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ClusterHealthy + message: "Cluster is healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected nodes present" + lastUpdated: "2024-01-01T00:00:01Z" + nodes: + - name: node1.example.com + ipAddresses: + - "192.168.1.1" + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: NodeHealthy + message: "Node healthy" + - type: Online + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Online + message: "Online" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Ready + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Ready + message: "Ready" + - type: Clean + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Clean + message: "Clean" + - type: Member + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Member + message: "Member" + resources: + - name: Kubelet + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: Etcd + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: FencingAgent + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + # IP address validation tests + - name: Should reject invalid IP address + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + updated: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ClusterHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected" + lastUpdated: "2024-01-01T00:00:01Z" + nodes: + - name: node1.example.com + ipAddresses: + - "invalid-ip" + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: NodeHealthy + message: "Healthy" + - type: Online + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Online + message: "Online" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Ready + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Ready + message: "Ready" + - type: Clean + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Clean + message: "Clean" + - type: Member + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Member + message: "Member" + resources: + - name: Kubelet + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: Etcd + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: FencingAgent + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + expectedStatusError: "must be a valid global unicast IPv4 or IPv6 address in canonical form" + - name: Should reject loopback IP address + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + updated: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ClusterHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected" + lastUpdated: "2024-01-01T00:00:01Z" + nodes: + - name: node1.example.com + ipAddresses: + - "127.0.0.1" + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: NodeHealthy + message: "Healthy" + - type: Online + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Online + message: "Online" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Ready + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Ready + message: "Ready" + - type: Clean + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Clean + message: "Clean" + - type: Member + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Member + message: "Member" + resources: + - name: Kubelet + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: Etcd + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: FencingAgent + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + expectedStatusError: "must be a valid global unicast IPv4 or IPv6 address in canonical form" + # Node name validation + - name: Should reject node name with uppercase + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + updated: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ClusterHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected" + lastUpdated: "2024-01-01T00:00:01Z" + nodes: + - name: "Node1" + ipAddresses: + - "192.168.1.1" + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: NodeHealthy + message: "Healthy" + - type: Online + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Online + message: "Online" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Ready + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Ready + message: "Ready" + - type: Clean + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Clean + message: "Clean" + - type: Member + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Member + message: "Member" + resources: + - name: Kubelet + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: Etcd + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: FencingAgent + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + expectedStatusError: "name must be a lowercase RFC 1123 subdomain" + # Missing required conditions tests + - name: Should reject cluster status missing Healthy condition + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + updated: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected" + - type: Extra + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Extra + message: "Extra" + lastUpdated: "2024-01-01T00:00:01Z" + expectedStatusError: "conditions must contain a condition of type Healthy" + # Missing required resource tests + - name: Should reject node missing Etcd resource + initial: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + updated: | + apiVersion: etcd.openshift.io/v1alpha1 + kind: PacemakerCluster + metadata: + name: cluster + status: + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ClusterHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: NodeCountAsExpected + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: AsExpected + message: "Expected" + lastUpdated: "2024-01-01T00:00:01Z" + nodes: + - name: node1.example.com + ipAddresses: + - "192.168.1.1" + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: NodeHealthy + message: "Healthy" + - type: Online + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Online + message: "Online" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Ready + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Ready + message: "Ready" + - type: Clean + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Clean + message: "Clean" + - type: Member + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Member + message: "Member" + resources: + - name: Kubelet + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + - name: FencingAgent + conditions: + - type: Healthy + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: ResourceHealthy + message: "Healthy" + - type: InService + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: InService + message: "In service" + - type: Managed + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Managed + message: "Managed" + - type: Enabled + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Enabled + message: "Enabled" + - type: Operational + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Operational + message: "Operational" + - type: Active + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Active + message: "Active" + - type: Started + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Started + message: "Started" + - type: Schedulable + status: "True" + lastTransitionTime: "2024-01-01T00:00:00Z" + reason: Schedulable + message: "Schedulable" + expectedStatusError: "resources must contain a resource named Etcd" diff --git a/etcd/v1alpha1/types_pacemakercluster.go b/etcd/v1alpha1/types_pacemakercluster.go new file mode 100644 index 00000000000..07261dae7c0 --- /dev/null +++ b/etcd/v1alpha1/types_pacemakercluster.go @@ -0,0 +1,594 @@ +package v1alpha1 + +import ( + metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" +) + +// PacemakerCluster is used in Two Node OpenShift with Fencing deployments to monitor the health +// of etcd running under pacemaker. + +// Cluster-level condition types for PacemakerCluster.status.conditions +const ( + // ClusterHealthyConditionType tracks the overall health of the pacemaker cluster. + // This is an aggregate condition that reflects the health of all cluster-level conditions and node health. + // Specifically, it aggregates the following conditions: + // - ClusterInServiceConditionType + // - ClusterNodeCountAsExpectedConditionType + // - NodeHealthyConditionType (for each node) + // When True, the cluster is healthy with reason "ClusterHealthy". + // When False, the cluster is unhealthy with reason "ClusterUnhealthy". + ClusterHealthyConditionType = "Healthy" + + // ClusterInServiceConditionType tracks whether the cluster is in service (not in maintenance mode). + // Maintenance mode is a cluster-wide setting that prevents pacemaker from starting or stopping resources. + // When True, the cluster is in service with reason "InService". This is the normal operating state. + // When False, the cluster is in maintenance mode with reason "InMaintenance". This is an unexpected state. + ClusterInServiceConditionType = "InService" + + // ClusterNodeCountAsExpectedConditionType tracks whether the cluster has the expected number of nodes. + // For Two Node OpenShift with Fencing, we are expecting exactly 2 nodes. + // When True, the expected number of nodes are present with reason "AsExpected". + // When False, the node count is incorrect with reason "InsufficientNodes" or "ExcessiveNodes". + ClusterNodeCountAsExpectedConditionType = "NodeCountAsExpected" +) + +// ClusterHealthy condition reasons +const ( + // ClusterHealthyReasonHealthy means the pacemaker cluster is healthy and operating normally. + ClusterHealthyReasonHealthy = "ClusterHealthy" + + // ClusterHealthyReasonUnhealthy means the pacemaker cluster has issues that need investigation. + ClusterHealthyReasonUnhealthy = "ClusterUnhealthy" +) + +// ClusterInService condition reasons +const ( + // ClusterInServiceReasonInService means the cluster is in service (not in maintenance mode). + // This is the normal operating state. + ClusterInServiceReasonInService = "InService" + + // ClusterInServiceReasonInMaintenance means the cluster is in maintenance mode. + // In maintenance mode, pacemaker will not start or stop any resources. Entering and exiting this state requires + // manual user intervention, and is unexpected during normal cluster operation. + ClusterInServiceReasonInMaintenance = "InMaintenance" +) + +// ClusterNodeCountAsExpected condition reasons +const ( + // ClusterNodeCountAsExpectedReasonAsExpected means the expected number of nodes are present. + // For Two Node OpenShift with Fencing, we are expecting exactly 2 nodes. This is the expected healthy state. + ClusterNodeCountAsExpectedReasonAsExpected = "AsExpected" + + // ClusterNodeCountAsExpectedReasonInsufficientNodes means fewer nodes than expected are present. + // For Two Node OpenShift with Fencing, this means that less than 2 nodes are present. Under normal operation, this will only happen during + // a node replacement operation. It's also possible to enter this state with manual user intervention, but + // will also require user intervention to restore normal functionality. + ClusterNodeCountAsExpectedReasonInsufficientNodes = "InsufficientNodes" + + // ClusterNodeCountAsExpectedReasonExcessiveNodes means more nodes than expected are present. + // For Two Node OpenShift with Fencing, this means more than 2 nodes are present. This should be investigated as it is unexpected and should + // never happen during normal cluster operation. It is possible to enter this state with manual user intervention, + // but will also require user intervention to restore normal functionality. + ClusterNodeCountAsExpectedReasonExcessiveNodes = "ExcessiveNodes" +) + +// Node-level condition types for PacemakerCluster.status.nodes[].conditions +const ( + // NodeHealthyConditionType tracks the overall health of a node in the pacemaker cluster. + // This is an aggregate condition that reflects the health of all node-level conditions and resource health. + // Specifically, it aggregates the following conditions: + // - NodeOnlineConditionType + // - NodeInServiceConditionType + // - NodeActiveConditionType + // - NodeReadyConditionType + // - NodeCleanConditionType + // - NodeMemberConditionType + // - ResourceHealthyConditionType (for each resource in the node's resources list) + // When True, the node is healthy with reason "NodeHealthy". + // When False, the node is unhealthy with reason "NodeUnhealthy". + NodeHealthyConditionType = "Healthy" + + // NodeOnlineConditionType tracks whether a node is online. + // When True, the node is online with reason "Online". This is the normal operating state. + // When False, the node is offline with reason "Offline". This can occur during reboots, failures, maintenance, or replacement. + NodeOnlineConditionType = "Online" + + // NodeInServiceConditionType tracks whether a node is in service (not in maintenance mode). + // A node in maintenance mode is ignored by pacemaker while maintenance mode is active. + // When True, the node is in service with reason "InService". This is the normal operating state. + // When False, the node is in maintenance mode with reason "InMaintenance". This is an unexpected state. + NodeInServiceConditionType = "InService" + + // NodeActiveConditionType tracks whether a node is active (not in standby mode). + // When a node enters standby mode, pacemaker moves its resources to other nodes in the cluster. + // In Two Node OpenShift with Fencing, we do not use standby mode during normal operation. + // When True, the node is active with reason "Active". This is the normal operating state. + // When False, the node is in standby mode with reason "Standby". This is an unexpected state. + NodeActiveConditionType = "Active" + + // NodeReadyConditionType tracks whether a node is ready (not in a pending state). + // A node in a pending state is in the process of joining or leaving the cluster. + // When True, the node is ready with reason "Ready". This is the normal operating state. + // When False, the node is pending with reason "Pending". This is expected to be temporary. + NodeReadyConditionType = "Ready" + + // NodeCleanConditionType tracks whether a node is in a clean state. + // An unclean state means that pacemaker was unable to confirm the node's state, which signifies issues + // in fencing, communication, or configuration. + // When True, the node is clean with reason "Clean". This is the normal operating state. + // When False, the node is unclean with reason "Unclean". This is an unexpected state. + NodeCleanConditionType = "Clean" + + // NodeMemberConditionType tracks whether a node is a member of the cluster. + // Some configurations may use remote nodes or ping nodes, which are nodes that are not members. + // For Two Node OpenShift with Fencing, we expect both nodes to be members. + // When True, the node is a member with reason "Member". This is the normal operating state. + // When False, the node is not a member with reason "NotMember". This is an unexpected state. + NodeMemberConditionType = "Member" +) + +// NodeHealthy condition reasons +const ( + // NodeHealthyReasonHealthy means the node is healthy and operating normally. + NodeHealthyReasonHealthy = "NodeHealthy" + + // NodeHealthyReasonUnhealthy means the node has issues that need investigation. + NodeHealthyReasonUnhealthy = "NodeUnhealthy" +) + +// NodeOnline condition reasons +const ( + // NodeOnlineReasonOnline means the node is online. This is the normal operating state. + NodeOnlineReasonOnline = "Online" + + // NodeOnlineReasonOffline means the node is offline. + NodeOnlineReasonOffline = "Offline" +) + +// NodeInService condition reasons +const ( + // NodeInServiceReasonInService means the node is in service (not in maintenance mode). + // This is the normal operating state. + NodeInServiceReasonInService = "InService" + + // NodeInServiceReasonInMaintenance means the node is in maintenance mode. + // This is an unexpected state. + NodeInServiceReasonInMaintenance = "InMaintenance" +) + +// NodeActive condition reasons +const ( + // NodeActiveReasonActive means the node is active (not in standby mode). + // This is the normal operating state. + NodeActiveReasonActive = "Active" + + // NodeActiveReasonStandby means the node is in standby mode. + // This is an unexpected state. + NodeActiveReasonStandby = "Standby" +) + +// NodeReady condition reasons +const ( + // NodeReadyReasonReady means the node is ready (not in a pending state). + // This is the normal operating state. + NodeReadyReasonReady = "Ready" + + // NodeReadyReasonPending means the node is joining or leaving the cluster. + // This state is expected to be temporary. + NodeReadyReasonPending = "Pending" +) + +// NodeClean condition reasons +const ( + // NodeCleanReasonClean means the node is in a clean state. + // This is the normal operating state. + NodeCleanReasonClean = "Clean" + + // NodeCleanReasonUnclean means the node is in an unclean state. + // Pacemaker was unable to confirm the node's state, which signifies issues in fencing, communication, or configuration. + // This is an unexpected state. + NodeCleanReasonUnclean = "Unclean" +) + +// NodeMember condition reasons +const ( + // NodeMemberReasonMember means the node is a member of the cluster. + // For Two Node OpenShift with Fencing, we expect both nodes to be members. This is the normal operating state. + NodeMemberReasonMember = "Member" + + // NodeMemberReasonNotMember means the node is not a member of the cluster. + // Some configurations may use remote nodes or ping nodes, which are nodes that are not members. + // This is an unexpected state. + NodeMemberReasonNotMember = "NotMember" +) + +// Resource-level condition types for PacemakerCluster.status.nodes[].resources[].conditions +const ( + // ResourceHealthyConditionType tracks the overall health of a pacemaker resource. + // This is an aggregate condition that reflects the health of all resource-level conditions. + // Specifically, it aggregates the following conditions: + // - ResourceInServiceConditionType + // - ResourceManagedConditionType + // - ResourceEnabledConditionType + // - ResourceOperationalConditionType + // - ResourceActiveConditionType + // - ResourceStartedConditionType + // - ResourceSchedulableConditionType + // When True, the resource is healthy with reason "ResourceHealthy". + // When False, the resource is unhealthy with reason "ResourceUnhealthy". + ResourceHealthyConditionType = "Healthy" + + // ResourceInServiceConditionType tracks whether a resource is in service (not in maintenance mode). + // Resources in maintenance mode are not monitored or moved by pacemaker. + // In Two Node OpenShift with Fencing, we do not expect any resources to be in maintenance mode. + // When True, the resource is in service with reason "InService". This is the normal operating state. + // When False, the resource is in maintenance mode with reason "InMaintenance". This is an unexpected state. + ResourceInServiceConditionType = "InService" + + // ResourceManagedConditionType tracks whether a resource is managed by pacemaker. + // Resources that are not managed by pacemaker are effectively invisible to the pacemaker HA logic. + // For Two Node OpenShift with Fencing, all resources are expected to be managed. + // When True, the resource is managed with reason "Managed". This is the normal operating state. + // When False, the resource is not managed with reason "Unmanaged". This is an unexpected state. + ResourceManagedConditionType = "Managed" + + // ResourceEnabledConditionType tracks whether a resource is enabled. + // Resources that are disabled are stopped and not automatically managed or started by the cluster. + // In Two Node OpenShift with Fencing, we do not expect any resources to be disabled. + // When True, the resource is enabled with reason "Enabled". This is the normal operating state. + // When False, the resource is disabled with reason "Disabled". This is an unexpected state. + ResourceEnabledConditionType = "Enabled" + + // ResourceOperationalConditionType tracks whether a resource is operational (not failed). + // A failed resource is one that is not able to start or is in an error state. + // When True, the resource is operational with reason "Operational". This is the normal operating state. + // When False, the resource has failed with reason "Failed". This is an unexpected state. + ResourceOperationalConditionType = "Operational" + + // ResourceActiveConditionType tracks whether a resource is active. + // An active resource is running on a cluster node. + // In Two Node OpenShift with Fencing, all resources are expected to be active. + // When True, the resource is active with reason "Active". This is the normal operating state. + // When False, the resource is not active with reason "Inactive". This is an unexpected state. + ResourceActiveConditionType = "Active" + + // ResourceStartedConditionType tracks whether a resource is started. + // It's normal for a resource like etcd to become stopped in the event of a quorum loss event because + // the pacemaker recovery logic will fence a node and restore etcd quorum on the surviving node as a cluster-of-one. + // A resource that stays stopped for an extended period of time is an unexpected state and should be investigated. + // When True, the resource is started with reason "Started". This is the normal operating state. + // When False, the resource is not started with reason "Stopped". This is expected to be temporary. + ResourceStartedConditionType = "Started" + + // ResourceSchedulableConditionType tracks whether a resource is schedulable (not blocked). + // A resource that is not schedulable is unable to start or move to a different node. + // In Two Node OpenShift with Fencing, we do not expect any resources to be unschedulable. + // When True, the resource is schedulable with reason "Schedulable". This is the normal operating state. + // When False, the resource is not schedulable with reason "Unschedulable". This is an unexpected state. + ResourceSchedulableConditionType = "Schedulable" +) + +// ResourceHealthy condition reasons +const ( + // ResourceHealthyReasonHealthy means the resource is healthy and operating normally. + ResourceHealthyReasonHealthy = "ResourceHealthy" + + // ResourceHealthyReasonUnhealthy means the resource has issues that need investigation. + ResourceHealthyReasonUnhealthy = "ResourceUnhealthy" +) + +// ResourceInService condition reasons +const ( + // ResourceInServiceReasonInService means the resource is in service (not in maintenance mode). + // This is the normal operating state. + ResourceInServiceReasonInService = "InService" + + // ResourceInServiceReasonInMaintenance means the resource is in maintenance mode. + // Resources in maintenance mode are not monitored or moved by pacemaker. This is an unexpected state. + ResourceInServiceReasonInMaintenance = "InMaintenance" +) + +// ResourceManaged condition reasons +const ( + // ResourceManagedReasonManaged means the resource is managed by pacemaker. + // This is the normal operating state. + ResourceManagedReasonManaged = "Managed" + + // ResourceManagedReasonUnmanaged means the resource is not managed by pacemaker. + // Resources that are not managed by pacemaker are effectively invisible to the pacemaker HA logic. + // This is an unexpected state. + ResourceManagedReasonUnmanaged = "Unmanaged" +) + +// ResourceEnabled condition reasons +const ( + // ResourceEnabledReasonEnabled means the resource is enabled. + // This is the normal operating state. + ResourceEnabledReasonEnabled = "Enabled" + + // ResourceEnabledReasonDisabled means the resource is disabled. + // Resources that are disabled are stopped and not automatically managed or started by the cluster. + // This is an unexpected state. + ResourceEnabledReasonDisabled = "Disabled" +) + +// ResourceOperational condition reasons +const ( + // ResourceOperationalReasonOperational means the resource is operational (not failed). + // This is the normal operating state. + ResourceOperationalReasonOperational = "Operational" + + // ResourceOperationalReasonFailed means the resource has failed. + // A failed resource is one that is not able to start or is in an error state. This is an unexpected state. + ResourceOperationalReasonFailed = "Failed" +) + +// ResourceActive condition reasons +const ( + // ResourceActiveReasonActive means the resource is active. + // An active resource is running on a cluster node. This is the normal operating state. + ResourceActiveReasonActive = "Active" + + // ResourceActiveReasonInactive means the resource is not active. + // This is an unexpected state. + ResourceActiveReasonInactive = "Inactive" +) + +// ResourceStarted condition reasons +const ( + // ResourceStartedReasonStarted means the resource is started. + // This is the normal operating state. + ResourceStartedReasonStarted = "Started" + + // ResourceStartedReasonStopped means the resource is stopped. + // It's normal for a resource like etcd to become stopped in the event of a quorum loss event because + // the pacemaker recovery logic will fence a node and restore etcd quorum on the surviving node as a cluster-of-one. + // A resource that stays stopped for an extended period of time is an unexpected state and should be investigated. + ResourceStartedReasonStopped = "Stopped" +) + +// ResourceSchedulable condition reasons +const ( + // ResourceSchedulableReasonSchedulable means the resource is schedulable (not blocked). + // This is the normal operating state. + ResourceSchedulableReasonSchedulable = "Schedulable" + + // ResourceSchedulableReasonUnschedulable means the resource is not schedulable (blocked). + // A resource that is not schedulable is unable to start or move to a different node. This is an unexpected state. + ResourceSchedulableReasonUnschedulable = "Unschedulable" +) + +// PacemakerClusterResourceName represents the name of a pacemaker resource. +// +kubebuilder:validation:Enum=Kubelet;Etcd;FencingAgent +type PacemakerClusterResourceName string + +// PacemakerClusterResourceName values +const ( + // PacemakerClusterResourceNameKubelet is the kubelet pacemaker resource. + // The kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + PacemakerClusterResourceNameKubelet PacemakerClusterResourceName = "Kubelet" + + // PacemakerClusterResourceNameEtcd is the etcd pacemaker resource. + // The etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + PacemakerClusterResourceNameEtcd PacemakerClusterResourceName = "Etcd" + + // PacemakerClusterResourceNameFencingAgent is the fencing agent pacemaker resource. + // The fencing agent is used to fence the other node during a quorum loss event. + PacemakerClusterResourceNameFencingAgent PacemakerClusterResourceName = "FencingAgent" +) + +// +genclient +// +genclient:nonNamespaced +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object +// PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. +// PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This +// resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. +// +// Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. +// +openshift:compatibility-gen:level=4 +// +kubebuilder:object:root=true +// +kubebuilder:resource:path=pacemakerclusters,scope=Cluster,singular=pacemakercluster +// +kubebuilder:subresource:status +// +openshift:api-approved.openshift.io=https://github.com/openshift/api/pull/2544 +// +openshift:file-pattern=cvoRunLevel=0000_25,operatorName=etcd,operatorOrdering=01,operatorComponent=two-node-fencing +// +openshift:enable:FeatureGate=DualReplica +// +kubebuilder:validation:XValidation:rule="self.metadata.name == 'cluster'",message="PacemakerCluster must be named 'cluster'" +type PacemakerCluster struct { + metav1.TypeMeta `json:",inline"` + + // metadata is the standard object's metadata. + // More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata + // +required + metav1.ObjectMeta `json:"metadata,omitempty"` + + // status contains the actual pacemaker cluster status information collected from the cluster. + // The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + // In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + // +optional + Status *PacemakerClusterStatus `json:"status,omitempty"` +} + +// PacemakerClusterStatus contains the actual pacemaker cluster status information. As part of validating the status +// object, we need to ensure that the lastUpdated timestamp cannot be removed once set and cannot be set to an earlier +// timestamp than the current value. +// +kubebuilder:validation:XValidation:rule="!has(oldSelf.lastUpdated) || has(self.lastUpdated)",message="lastUpdated cannot be removed once set" +// +kubebuilder:validation:XValidation:rule="!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated >= oldSelf.lastUpdated",message="lastUpdated cannot be set to an earlier timestamp" +type PacemakerClusterStatus struct { + // conditions represent the observations of the pacemaker cluster's current state. + // Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + // The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + // The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + // The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + // Each of these conditions is required, so the array must contain at least 3 items. + // +listType=map + // +listMapKey=type + // +kubebuilder:validation:MinItems=3 + // +kubebuilder:validation:MaxItems=8 + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Healthy')",message="conditions must contain a condition of type Healthy" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'InService')",message="conditions must contain a condition of type InService" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'NodeCountAsExpected')",message="conditions must contain a condition of type NodeCountAsExpected" + // Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + // after observing state. The MinItems validation ensures conditions are present once the controller + // has reported status. + // See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + // +optional + Conditions []metav1.Condition `json:"conditions,omitempty"` + + // lastUpdated is the timestamp when this status was last updated. This is useful for identifying + // stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + // be removed and cannot be set to an earlier timestamp than the current value. + // This field is marked optional because all status fields must be optional to allow for partial updates + // and backward compatibility. However, once set it must always be present. + // See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + // +kubebuilder:validation:Format=date-time + // +optional + LastUpdated metav1.Time `json:"lastUpdated,omitempty"` + + // nodes provides detailed information about each node in the cluster including per-node resource health. + // Each node entry includes the node's name, IP address, conditions, and resource status. + // The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + // For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + // are currently reporting status. + // +listType=map + // +listMapKey=name + // +kubebuilder:validation:MaxItems=32 + // +optional + Nodes []PacemakerClusterNodeStatus `json:"nodes,omitempty"` +} + +// PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including +// the node's conditions and the health of critical resources running on that node. +type PacemakerClusterNodeStatus struct { + // conditions represent the observations of the node's current state. + // Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + // The "Healthy" condition is an aggregate that tracks the overall health of the node. + // The "Online" condition tracks whether the node is online. + // The "InService" condition tracks whether the node is in service (not in maintenance mode). + // The "Active" condition tracks whether the node is active (not in standby mode). + // The "Ready" condition tracks whether the node is ready (not in a pending state). + // The "Clean" condition tracks whether the node is in a clean (status known) state. + // The "Member" condition tracks whether the node is a member of the cluster. + // Each of these conditions is required, so the array must contain at least 7 items. + // +listType=map + // +listMapKey=type + // +kubebuilder:validation:MinItems=7 + // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Healthy')",message="conditions must contain a condition of type Healthy" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Online')",message="conditions must contain a condition of type Online" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'InService')",message="conditions must contain a condition of type InService" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Active')",message="conditions must contain a condition of type Active" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Ready')",message="conditions must contain a condition of type Ready" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Clean')",message="conditions must contain a condition of type Clean" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Member')",message="conditions must contain a condition of type Member" + // Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + // after observing state. The MinItems validation ensures conditions are present once the controller + // has reported status. + // See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + // +optional + Conditions []metav1.Condition `json:"conditions,omitempty"` + + // name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + // RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + // an alphanumeric character, and be at most 253 characters in length. + // +kubebuilder:validation:MinLength=1 + // +kubebuilder:validation:MaxLength=253 + // +kubebuilder:validation:XValidation:rule="!format.dns1123Subdomain().validate(self).hasValue()",message="name must be a lowercase RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', and must start and end with an alphanumeric character" + // +required + Name string `json:"name,omitempty"` + + // ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + // Pacemaker allows multiple IP addresses for Corosync communication between nodes. + // The first address in this list is used for IP-based peer URLs for etcd membership. + // Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + // Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + // This excludes special addresses like unspecified, loopback, link-local, and multicast. + // +listType=set + // +kubebuilder:validation:MinItems=1 + // +kubebuilder:validation:MaxItems=8 + // +kubebuilder:validation:items:MaxLength=39 + // +kubebuilder:validation:items:XValidation:rule="isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast()",message="must be a valid global unicast IPv4 or IPv6 address in canonical form" + // +required + IPAddresses []string `json:"ipAddresses,omitempty"` + + // resources contains the status of pacemaker resources on this node. + // Each resource entry includes the resource name and its health conditions. + // For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + // All three resources are required to be present, so the array must contain at least 3 items. + // Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + // +listType=map + // +listMapKey=name + // +kubebuilder:validation:MinItems=3 + // +kubebuilder:validation:MaxItems=8 + // +kubebuilder:validation:XValidation:rule="self.exists(r, r.name == 'Kubelet')",message="resources must contain a resource named Kubelet" + // +kubebuilder:validation:XValidation:rule="self.exists(r, r.name == 'Etcd')",message="resources must contain a resource named Etcd" + // +kubebuilder:validation:XValidation:rule="self.exists(r, r.name == 'FencingAgent')",message="resources must contain a resource named FencingAgent" + // +required + Resources []PacemakerClusterResourceStatus `json:"resources,omitempty"` +} + +// PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. +// A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or +// applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. +// For Two Node OpenShift with Fencing, we manage three resources: +// - Kubelet (the Kubernetes node agent and a prerequisite for etcd), +// - Etcd (the distributed key-value store) +// - FencingAgent (used to isolate failed nodes during a quorum loss event) +type PacemakerClusterResourceStatus struct { + // conditions represent the observations of the resource's current state. + // Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + // "Active", "Started", "Schedulable". + // The "Healthy" condition is an aggregate that tracks the overall health of the resource. + // The "InService" condition tracks whether the resource is in service (not in maintenance mode). + // The "Managed" condition tracks whether the resource is managed by pacemaker. + // The "Enabled" condition tracks whether the resource is enabled. + // The "Operational" condition tracks whether the resource is operational (not failed). + // The "Active" condition tracks whether the resource is active (available to be used). + // The "Started" condition tracks whether the resource is started. + // The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + // Each of these conditions is required, so the array must contain at least 8 items. + // Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + // +listType=map + // +listMapKey=type + // +kubebuilder:validation:MinItems=8 + // +kubebuilder:validation:MaxItems=16 + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Healthy')",message="conditions must contain a condition of type Healthy" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'InService')",message="conditions must contain a condition of type InService" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Managed')",message="conditions must contain a condition of type Managed" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Enabled')",message="conditions must contain a condition of type Enabled" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Operational')",message="conditions must contain a condition of type Operational" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Active')",message="conditions must contain a condition of type Active" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Started')",message="conditions must contain a condition of type Started" + // +kubebuilder:validation:XValidation:rule="self.exists(c, c.type == 'Schedulable')",message="conditions must contain a condition of type Schedulable" + // Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + // after observing state. The MinItems validation ensures conditions are present once the controller + // has reported status. + // See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + // +optional + Conditions []metav1.Condition `json:"conditions,omitempty"` + + // name is the name of the pacemaker resource. + // Valid values are "Kubelet", "Etcd", and "FencingAgent". + // The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + // The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + // The FencingAgent resource is used to fence the other node during a quorum loss event. + // +required + Name PacemakerClusterResourceName `json:"name,omitempty"` +} + +// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object +// PacemakerClusterList contains a list of PacemakerCluster objects. PacemakerCluster is a cluster-scoped singleton +// resource; only one instance named "cluster" may exist. This list type exists only to satisfy Kubernetes API +// conventions. +// +// Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. +// +openshift:compatibility-gen:level=4 +type PacemakerClusterList struct { + metav1.TypeMeta `json:",inline"` + + // metadata is the standard list's metadata. + // More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata + metav1.ListMeta `json:"metadata,omitempty"` + + // items is a list of PacemakerCluster objects. + Items []PacemakerCluster `json:"items"` +} diff --git a/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-CustomNoUpgrade.crd.yaml b/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-CustomNoUpgrade.crd.yaml new file mode 100644 index 00000000000..b07c5e94876 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-CustomNoUpgrade.crd.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/merged-by-featuregates: "true" + include.release.openshift.io/ibm-cloud-managed: "true" + include.release.openshift.io/self-managed-high-availability: "true" + release.openshift.io/feature-set: CustomNoUpgrade + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {} diff --git a/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-DevPreviewNoUpgrade.crd.yaml b/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-DevPreviewNoUpgrade.crd.yaml new file mode 100644 index 00000000000..22ef80423b7 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-DevPreviewNoUpgrade.crd.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/merged-by-featuregates: "true" + include.release.openshift.io/ibm-cloud-managed: "true" + include.release.openshift.io/self-managed-high-availability: "true" + release.openshift.io/feature-set: DevPreviewNoUpgrade + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {} diff --git a/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-TechPreviewNoUpgrade.crd.yaml b/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-TechPreviewNoUpgrade.crd.yaml new file mode 100644 index 00000000000..87775a56941 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters-TechPreviewNoUpgrade.crd.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/merged-by-featuregates: "true" + include.release.openshift.io/ibm-cloud-managed: "true" + include.release.openshift.io/self-managed-high-availability: "true" + release.openshift.io/feature-set: TechPreviewNoUpgrade + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {} diff --git a/etcd/v1alpha1/zz_generated.crd-manifests/doc.go b/etcd/v1alpha1/zz_generated.crd-manifests/doc.go new file mode 100644 index 00000000000..fbefa7bcc2c --- /dev/null +++ b/etcd/v1alpha1/zz_generated.crd-manifests/doc.go @@ -0,0 +1 @@ +package etcd_v1alpha1_crdmanifests diff --git a/etcd/v1alpha1/zz_generated.deepcopy.go b/etcd/v1alpha1/zz_generated.deepcopy.go new file mode 100644 index 00000000000..d23c981e443 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.deepcopy.go @@ -0,0 +1,164 @@ +//go:build !ignore_autogenerated +// +build !ignore_autogenerated + +// Code generated by codegen. DO NOT EDIT. + +package v1alpha1 + +import ( + v1 "k8s.io/apimachinery/pkg/apis/meta/v1" + runtime "k8s.io/apimachinery/pkg/runtime" +) + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PacemakerCluster) DeepCopyInto(out *PacemakerCluster) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ObjectMeta.DeepCopyInto(&out.ObjectMeta) + if in.Status != nil { + in, out := &in.Status, &out.Status + *out = new(PacemakerClusterStatus) + (*in).DeepCopyInto(*out) + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PacemakerCluster. +func (in *PacemakerCluster) DeepCopy() *PacemakerCluster { + if in == nil { + return nil + } + out := new(PacemakerCluster) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PacemakerCluster) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PacemakerClusterList) DeepCopyInto(out *PacemakerClusterList) { + *out = *in + out.TypeMeta = in.TypeMeta + in.ListMeta.DeepCopyInto(&out.ListMeta) + if in.Items != nil { + in, out := &in.Items, &out.Items + *out = make([]PacemakerCluster, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PacemakerClusterList. +func (in *PacemakerClusterList) DeepCopy() *PacemakerClusterList { + if in == nil { + return nil + } + out := new(PacemakerClusterList) + in.DeepCopyInto(out) + return out +} + +// DeepCopyObject is an autogenerated deepcopy function, copying the receiver, creating a new runtime.Object. +func (in *PacemakerClusterList) DeepCopyObject() runtime.Object { + if c := in.DeepCopy(); c != nil { + return c + } + return nil +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PacemakerClusterNodeStatus) DeepCopyInto(out *PacemakerClusterNodeStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + if in.IPAddresses != nil { + in, out := &in.IPAddresses, &out.IPAddresses + *out = make([]string, len(*in)) + copy(*out, *in) + } + if in.Resources != nil { + in, out := &in.Resources, &out.Resources + *out = make([]PacemakerClusterResourceStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PacemakerClusterNodeStatus. +func (in *PacemakerClusterNodeStatus) DeepCopy() *PacemakerClusterNodeStatus { + if in == nil { + return nil + } + out := new(PacemakerClusterNodeStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PacemakerClusterResourceStatus) DeepCopyInto(out *PacemakerClusterResourceStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PacemakerClusterResourceStatus. +func (in *PacemakerClusterResourceStatus) DeepCopy() *PacemakerClusterResourceStatus { + if in == nil { + return nil + } + out := new(PacemakerClusterResourceStatus) + in.DeepCopyInto(out) + return out +} + +// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil. +func (in *PacemakerClusterStatus) DeepCopyInto(out *PacemakerClusterStatus) { + *out = *in + if in.Conditions != nil { + in, out := &in.Conditions, &out.Conditions + *out = make([]v1.Condition, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + in.LastUpdated.DeepCopyInto(&out.LastUpdated) + if in.Nodes != nil { + in, out := &in.Nodes, &out.Nodes + *out = make([]PacemakerClusterNodeStatus, len(*in)) + for i := range *in { + (*in)[i].DeepCopyInto(&(*out)[i]) + } + } + return +} + +// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new PacemakerClusterStatus. +func (in *PacemakerClusterStatus) DeepCopy() *PacemakerClusterStatus { + if in == nil { + return nil + } + out := new(PacemakerClusterStatus) + in.DeepCopyInto(out) + return out +} diff --git a/etcd/v1alpha1/zz_generated.featuregated-crd-manifests.yaml b/etcd/v1alpha1/zz_generated.featuregated-crd-manifests.yaml new file mode 100644 index 00000000000..f5a64682ab2 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.featuregated-crd-manifests.yaml @@ -0,0 +1,23 @@ +pacemakerclusters.etcd.openshift.io: + Annotations: {} + ApprovedPRNumber: https://github.com/openshift/api/pull/2544 + CRDName: pacemakerclusters.etcd.openshift.io + Capability: "" + Category: "" + FeatureGates: + - DualReplica + FilenameOperatorName: etcd + FilenameOperatorOrdering: "01" + FilenameRunLevel: "0000_25" + GroupName: etcd.openshift.io + HasStatus: true + KindName: PacemakerCluster + Labels: {} + PluralName: pacemakerclusters + PrinterColumns: [] + Scope: Cluster + ShortNames: null + TopLevelFeatureGates: + - DualReplica + Version: v1alpha1 + diff --git a/etcd/v1alpha1/zz_generated.featuregated-crd-manifests/pacemakerclusters.etcd.openshift.io/DualReplica.yaml b/etcd/v1alpha1/zz_generated.featuregated-crd-manifests/pacemakerclusters.etcd.openshift.io/DualReplica.yaml new file mode 100644 index 00000000000..3181d768572 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.featuregated-crd-manifests/pacemakerclusters.etcd.openshift.io/DualReplica.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/filename-cvo-runlevel: "0000_25" + api.openshift.io/filename-operator: etcd + api.openshift.io/filename-ordering: "01" + feature-gate.release.openshift.io/DualReplica: "true" + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {} diff --git a/etcd/v1alpha1/zz_generated.swagger_doc_generated.go b/etcd/v1alpha1/zz_generated.swagger_doc_generated.go new file mode 100644 index 00000000000..325cdee9573 --- /dev/null +++ b/etcd/v1alpha1/zz_generated.swagger_doc_generated.go @@ -0,0 +1,67 @@ +package v1alpha1 + +// This file contains a collection of methods that can be used from go-restful to +// generate Swagger API documentation for its models. Please read this PR for more +// information on the implementation: https://github.com/emicklei/go-restful/pull/215 +// +// TODOs are ignored from the parser (e.g. TODO(andronat):... || TODO:...) if and only if +// they are on one line! For multiple line or blocks that you want to ignore use ---. +// Any context after a --- is ignored. +// +// Those methods can be generated by using hack/update-swagger-docs.sh + +// AUTO-GENERATED FUNCTIONS START HERE +var map_PacemakerCluster = map[string]string{ + "": "PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is \"cluster\". This resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments.\n\nCompatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support.", + "metadata": "metadata is the standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata", + "status": "status contains the actual pacemaker cluster status information collected from the cluster. The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources.", +} + +func (PacemakerCluster) SwaggerDoc() map[string]string { + return map_PacemakerCluster +} + +var map_PacemakerClusterList = map[string]string{ + "": "PacemakerClusterList contains a list of PacemakerCluster objects. PacemakerCluster is a cluster-scoped singleton resource; only one instance named \"cluster\" may exist. This list type exists only to satisfy Kubernetes API conventions.\n\nCompatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support.", + "metadata": "metadata is the standard list's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata", + "items": "items is a list of PacemakerCluster objects.", +} + +func (PacemakerClusterList) SwaggerDoc() map[string]string { + return map_PacemakerClusterList +} + +var map_PacemakerClusterNodeStatus = map[string]string{ + "": "PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including the node's conditions and the health of critical resources running on that node.", + "conditions": "conditions represent the observations of the node's current state. Known condition types are: \"Healthy\", \"Online\", \"InService\", \"Active\", \"Ready\", \"Clean\", \"Member\". The \"Healthy\" condition is an aggregate that tracks the overall health of the node. The \"Online\" condition tracks whether the node is online. The \"InService\" condition tracks whether the node is in service (not in maintenance mode). The \"Active\" condition tracks whether the node is active (not in standby mode). The \"Ready\" condition tracks whether the node is ready (not in a pending state). The \"Clean\" condition tracks whether the node is in a clean (status known) state. The \"Member\" condition tracks whether the node is a member of the cluster. Each of these conditions is required, so the array must contain at least 7 items. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + "name": "name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with an alphanumeric character, and be at most 253 characters in length.", + "ipAddresses": "ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. Pacemaker allows multiple IP addresses for Corosync communication between nodes. The first address in this list is used for IP-based peer URLs for etcd membership. Each address must be in canonical form (e.g., \"192.168.1.1\" not \"192.168.001.001\", or \"2001:db8::1\" not \"2001:0db8::1\"). Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). This excludes special addresses like unspecified, loopback, link-local, and multicast.", + "resources": "resources contains the status of pacemaker resources on this node. Each resource entry includes the resource name and its health conditions. For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. All three resources are required to be present, so the array must contain at least 3 items. Valid resource names are \"Kubelet\", \"Etcd\", and \"FencingAgent\".", +} + +func (PacemakerClusterNodeStatus) SwaggerDoc() map[string]string { + return map_PacemakerClusterNodeStatus +} + +var map_PacemakerClusterResourceStatus = map[string]string{ + "": "PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. For Two Node OpenShift with Fencing, we manage three resources:\n - Kubelet (the Kubernetes node agent and a prerequisite for etcd),\n - Etcd (the distributed key-value store)\n - FencingAgent (used to isolate failed nodes during a quorum loss event)", + "conditions": "conditions represent the observations of the resource's current state. Known condition types are: \"Healthy\", \"InService\", \"Managed\", \"Enabled\", \"Operational\", \"Active\", \"Started\", \"Schedulable\". The \"Healthy\" condition is an aggregate that tracks the overall health of the resource. The \"InService\" condition tracks whether the resource is in service (not in maintenance mode). The \"Managed\" condition tracks whether the resource is managed by pacemaker. The \"Enabled\" condition tracks whether the resource is enabled. The \"Operational\" condition tracks whether the resource is operational (not failed). The \"Active\" condition tracks whether the resource is active (available to be used). The \"Started\" condition tracks whether the resource is started. The \"Schedulable\" condition tracks whether the resource is schedulable (not blocked). Each of these conditions is required, so the array must contain at least 8 items. Conditions is idiomatically the first field in status structs per Kubernetes API conventions. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + "name": "name is the name of the pacemaker resource. Valid values are \"Kubelet\", \"Etcd\", and \"FencingAgent\". The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. The FencingAgent resource is used to fence the other node during a quorum loss event.", +} + +func (PacemakerClusterResourceStatus) SwaggerDoc() map[string]string { + return map_PacemakerClusterResourceStatus +} + +var map_PacemakerClusterStatus = map[string]string{ + "": "PacemakerClusterStatus contains the actual pacemaker cluster status information. As part of validating the status object, we need to ensure that the lastUpdated timestamp cannot be removed once set and cannot be set to an earlier timestamp than the current value.", + "conditions": "conditions represent the observations of the pacemaker cluster's current state. Known condition types are: \"Healthy\", \"InService\", \"NodeCountAsExpected\". The \"Healthy\" condition is an aggregate that tracks the overall health of the cluster. The \"InService\" condition tracks whether the cluster is in service (not in maintenance mode). The \"NodeCountAsExpected\" condition tracks whether the expected number of nodes are present. Each of these conditions is required, so the array must contain at least 3 items. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + "lastUpdated": "lastUpdated is the timestamp when this status was last updated. This is useful for identifying stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot be removed and cannot be set to an earlier timestamp than the current value. This field is marked optional because all status fields must be optional to allow for partial updates and backward compatibility. However, once set it must always be present. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go", + "nodes": "nodes provides detailed information about each node in the cluster including per-node resource health. Each node entry includes the node's name, IP address, conditions, and resource status. The list can contain up to 32 nodes (the upper limit imposed by pacemaker). For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes are currently reporting status.", +} + +func (PacemakerClusterStatus) SwaggerDoc() map[string]string { + return map_PacemakerClusterStatus +} + +// AUTO-GENERATED FUNCTIONS END HERE diff --git a/hack/update-payload-crds.sh b/hack/update-payload-crds.sh index 1c49d4d7bae..17ed1e7b452 100755 --- a/hack/update-payload-crds.sh +++ b/hack/update-payload-crds.sh @@ -30,6 +30,7 @@ crd_globs="\ operator/v1/zz_generated.crd-manifests/*_csi-driver_01_clustercsidrivers*.crd.yaml insights/v1alpha2/zz_generated.crd-manifests/0000_10_insights_01_datagathers*.crd.yaml config/v1alpha2/zz_generated.crd-manifests/0000_10_config-operator_01_insightsdatagathers*.crd.yaml + etcd/v1alpha1/zz_generated.crd-manifests/0000_25_etcd_01_pacemakerclusters*.crd.yaml " # To allow the crd_globs to be sourced in the verify script, diff --git a/install.go b/install.go index ea5f349708c..e4574e7c4f5 100644 --- a/install.go +++ b/install.go @@ -55,6 +55,7 @@ import ( "github.com/openshift/api/cloudnetwork" "github.com/openshift/api/config" "github.com/openshift/api/console" + "github.com/openshift/api/etcd" "github.com/openshift/api/helm" "github.com/openshift/api/image" "github.com/openshift/api/imageregistry" @@ -91,6 +92,7 @@ var ( build.Install, config.Install, console.Install, + etcd.Install, helm.Install, image.Install, imageregistry.Install, diff --git a/openapi/generated_openapi/zz_generated.openapi.go b/openapi/generated_openapi/zz_generated.openapi.go index 93f00307951..fe851837ef2 100644 --- a/openapi/generated_openapi/zz_generated.openapi.go +++ b/openapi/generated_openapi/zz_generated.openapi.go @@ -531,6 +531,11 @@ func GetOpenAPIDefinitions(ref common.ReferenceCallback) map[string]common.OpenA "github.com/openshift/api/console/v1.ConsoleYAMLSampleSpec": schema_openshift_api_console_v1_ConsoleYAMLSampleSpec(ref), "github.com/openshift/api/console/v1.Link": schema_openshift_api_console_v1_Link(ref), "github.com/openshift/api/console/v1.NamespaceDashboardSpec": schema_openshift_api_console_v1_NamespaceDashboardSpec(ref), + "github.com/openshift/api/etcd/v1alpha1.PacemakerCluster": schema_openshift_api_etcd_v1alpha1_PacemakerCluster(ref), + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterList": schema_openshift_api_etcd_v1alpha1_PacemakerClusterList(ref), + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterNodeStatus": schema_openshift_api_etcd_v1alpha1_PacemakerClusterNodeStatus(ref), + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterResourceStatus": schema_openshift_api_etcd_v1alpha1_PacemakerClusterResourceStatus(ref), + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterStatus": schema_openshift_api_etcd_v1alpha1_PacemakerClusterStatus(ref), "github.com/openshift/api/example/v1.CELUnion": schema_openshift_api_example_v1_CELUnion(ref), "github.com/openshift/api/example/v1.EvolvingUnion": schema_openshift_api_example_v1_EvolvingUnion(ref), "github.com/openshift/api/example/v1.FormatMarkerExamples": schema_openshift_api_example_v1_FormatMarkerExamples(ref), @@ -25786,6 +25791,297 @@ func schema_openshift_api_console_v1_NamespaceDashboardSpec(ref common.Reference } } +func schema_openshift_api_etcd_v1alpha1_PacemakerCluster(ref common.ReferenceCallback) common.OpenAPIDefinition { + return common.OpenAPIDefinition{ + Schema: spec.Schema{ + SchemaProps: spec.SchemaProps{ + Description: "PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is \"cluster\". This resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments.\n\nCompatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support.", + Type: []string{"object"}, + Properties: map[string]spec.Schema{ + "kind": { + SchemaProps: spec.SchemaProps{ + Description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds", + Type: []string{"string"}, + Format: "", + }, + }, + "apiVersion": { + SchemaProps: spec.SchemaProps{ + Description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources", + Type: []string{"string"}, + Format: "", + }, + }, + "metadata": { + SchemaProps: spec.SchemaProps{ + Description: "metadata is the standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata", + Default: map[string]interface{}{}, + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta"), + }, + }, + "status": { + SchemaProps: spec.SchemaProps{ + Description: "status contains the actual pacemaker cluster status information collected from the cluster. The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources.", + Ref: ref("github.com/openshift/api/etcd/v1alpha1.PacemakerClusterStatus"), + }, + }, + }, + Required: []string{"metadata"}, + }, + }, + Dependencies: []string{ + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterStatus", "k8s.io/apimachinery/pkg/apis/meta/v1.ObjectMeta"}, + } +} + +func schema_openshift_api_etcd_v1alpha1_PacemakerClusterList(ref common.ReferenceCallback) common.OpenAPIDefinition { + return common.OpenAPIDefinition{ + Schema: spec.Schema{ + SchemaProps: spec.SchemaProps{ + Description: "PacemakerClusterList contains a list of PacemakerCluster objects. PacemakerCluster is a cluster-scoped singleton resource; only one instance named \"cluster\" may exist. This list type exists only to satisfy Kubernetes API conventions.\n\nCompatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support.", + Type: []string{"object"}, + Properties: map[string]spec.Schema{ + "kind": { + SchemaProps: spec.SchemaProps{ + Description: "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds", + Type: []string{"string"}, + Format: "", + }, + }, + "apiVersion": { + SchemaProps: spec.SchemaProps{ + Description: "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources", + Type: []string{"string"}, + Format: "", + }, + }, + "metadata": { + SchemaProps: spec.SchemaProps{ + Description: "metadata is the standard list's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata", + Default: map[string]interface{}{}, + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.ListMeta"), + }, + }, + "items": { + SchemaProps: spec.SchemaProps{ + Description: "items is a list of PacemakerCluster objects.", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: map[string]interface{}{}, + Ref: ref("github.com/openshift/api/etcd/v1alpha1.PacemakerCluster"), + }, + }, + }, + }, + }, + }, + Required: []string{"items"}, + }, + }, + Dependencies: []string{ + "github.com/openshift/api/etcd/v1alpha1.PacemakerCluster", "k8s.io/apimachinery/pkg/apis/meta/v1.ListMeta"}, + } +} + +func schema_openshift_api_etcd_v1alpha1_PacemakerClusterNodeStatus(ref common.ReferenceCallback) common.OpenAPIDefinition { + return common.OpenAPIDefinition{ + Schema: spec.Schema{ + SchemaProps: spec.SchemaProps{ + Description: "PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including the node's conditions and the health of critical resources running on that node.", + Type: []string{"object"}, + Properties: map[string]spec.Schema{ + "conditions": { + VendorExtensible: spec.VendorExtensible{ + Extensions: spec.Extensions{ + "x-kubernetes-list-map-keys": []interface{}{ + "type", + }, + "x-kubernetes-list-type": "map", + }, + }, + SchemaProps: spec.SchemaProps{ + Description: "conditions represent the observations of the node's current state. Known condition types are: \"Healthy\", \"Online\", \"InService\", \"Active\", \"Ready\", \"Clean\", \"Member\". The \"Healthy\" condition is an aggregate that tracks the overall health of the node. The \"Online\" condition tracks whether the node is online. The \"InService\" condition tracks whether the node is in service (not in maintenance mode). The \"Active\" condition tracks whether the node is active (not in standby mode). The \"Ready\" condition tracks whether the node is ready (not in a pending state). The \"Clean\" condition tracks whether the node is in a clean (status known) state. The \"Member\" condition tracks whether the node is a member of the cluster. Each of these conditions is required, so the array must contain at least 7 items. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: map[string]interface{}{}, + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Condition"), + }, + }, + }, + }, + }, + "name": { + SchemaProps: spec.SchemaProps{ + Description: "name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with an alphanumeric character, and be at most 253 characters in length.", + Type: []string{"string"}, + Format: "", + }, + }, + "ipAddresses": { + VendorExtensible: spec.VendorExtensible{ + Extensions: spec.Extensions{ + "x-kubernetes-list-type": "set", + }, + }, + SchemaProps: spec.SchemaProps{ + Description: "ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. Pacemaker allows multiple IP addresses for Corosync communication between nodes. The first address in this list is used for IP-based peer URLs for etcd membership. Each address must be in canonical form (e.g., \"192.168.1.1\" not \"192.168.001.001\", or \"2001:db8::1\" not \"2001:0db8::1\"). Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). This excludes special addresses like unspecified, loopback, link-local, and multicast.", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: "", + Type: []string{"string"}, + Format: "", + }, + }, + }, + }, + }, + "resources": { + VendorExtensible: spec.VendorExtensible{ + Extensions: spec.Extensions{ + "x-kubernetes-list-map-keys": []interface{}{ + "name", + }, + "x-kubernetes-list-type": "map", + }, + }, + SchemaProps: spec.SchemaProps{ + Description: "resources contains the status of pacemaker resources on this node. Each resource entry includes the resource name and its health conditions. For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. All three resources are required to be present, so the array must contain at least 3 items. Valid resource names are \"Kubelet\", \"Etcd\", and \"FencingAgent\".", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: map[string]interface{}{}, + Ref: ref("github.com/openshift/api/etcd/v1alpha1.PacemakerClusterResourceStatus"), + }, + }, + }, + }, + }, + }, + Required: []string{"name", "ipAddresses", "resources"}, + }, + }, + Dependencies: []string{ + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterResourceStatus", "k8s.io/apimachinery/pkg/apis/meta/v1.Condition"}, + } +} + +func schema_openshift_api_etcd_v1alpha1_PacemakerClusterResourceStatus(ref common.ReferenceCallback) common.OpenAPIDefinition { + return common.OpenAPIDefinition{ + Schema: spec.Schema{ + SchemaProps: spec.SchemaProps{ + Description: "PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. For Two Node OpenShift with Fencing, we manage three resources:\n - Kubelet (the Kubernetes node agent and a prerequisite for etcd),\n - Etcd (the distributed key-value store)\n - FencingAgent (used to isolate failed nodes during a quorum loss event)", + Type: []string{"object"}, + Properties: map[string]spec.Schema{ + "conditions": { + VendorExtensible: spec.VendorExtensible{ + Extensions: spec.Extensions{ + "x-kubernetes-list-map-keys": []interface{}{ + "type", + }, + "x-kubernetes-list-type": "map", + }, + }, + SchemaProps: spec.SchemaProps{ + Description: "conditions represent the observations of the resource's current state. Known condition types are: \"Healthy\", \"InService\", \"Managed\", \"Enabled\", \"Operational\", \"Active\", \"Started\", \"Schedulable\". The \"Healthy\" condition is an aggregate that tracks the overall health of the resource. The \"InService\" condition tracks whether the resource is in service (not in maintenance mode). The \"Managed\" condition tracks whether the resource is managed by pacemaker. The \"Enabled\" condition tracks whether the resource is enabled. The \"Operational\" condition tracks whether the resource is operational (not failed). The \"Active\" condition tracks whether the resource is active (available to be used). The \"Started\" condition tracks whether the resource is started. The \"Schedulable\" condition tracks whether the resource is schedulable (not blocked). Each of these conditions is required, so the array must contain at least 8 items. Conditions is idiomatically the first field in status structs per Kubernetes API conventions. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: map[string]interface{}{}, + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Condition"), + }, + }, + }, + }, + }, + "name": { + SchemaProps: spec.SchemaProps{ + Description: "name is the name of the pacemaker resource. Valid values are \"Kubelet\", \"Etcd\", and \"FencingAgent\". The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. The FencingAgent resource is used to fence the other node during a quorum loss event.", + Type: []string{"string"}, + Format: "", + }, + }, + }, + Required: []string{"name"}, + }, + }, + Dependencies: []string{ + "k8s.io/apimachinery/pkg/apis/meta/v1.Condition"}, + } +} + +func schema_openshift_api_etcd_v1alpha1_PacemakerClusterStatus(ref common.ReferenceCallback) common.OpenAPIDefinition { + return common.OpenAPIDefinition{ + Schema: spec.Schema{ + SchemaProps: spec.SchemaProps{ + Description: "PacemakerClusterStatus contains the actual pacemaker cluster status information. As part of validating the status object, we need to ensure that the lastUpdated timestamp cannot be removed once set and cannot be set to an earlier timestamp than the current value.", + Type: []string{"object"}, + Properties: map[string]spec.Schema{ + "conditions": { + VendorExtensible: spec.VendorExtensible{ + Extensions: spec.Extensions{ + "x-kubernetes-list-map-keys": []interface{}{ + "type", + }, + "x-kubernetes-list-type": "map", + }, + }, + SchemaProps: spec.SchemaProps{ + Description: "conditions represent the observations of the pacemaker cluster's current state. Known condition types are: \"Healthy\", \"InService\", \"NodeCountAsExpected\". The \"Healthy\" condition is an aggregate that tracks the overall health of the cluster. The \"InService\" condition tracks whether the cluster is in service (not in maintenance mode). The \"NodeCountAsExpected\" condition tracks whether the expected number of nodes are present. Each of these conditions is required, so the array must contain at least 3 items. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: map[string]interface{}{}, + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Condition"), + }, + }, + }, + }, + }, + "lastUpdated": { + SchemaProps: spec.SchemaProps{ + Description: "lastUpdated is the timestamp when this status was last updated. This is useful for identifying stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot be removed and cannot be set to an earlier timestamp than the current value. This field is marked optional because all status fields must be optional to allow for partial updates and backward compatibility. However, once set it must always be present. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go", + Ref: ref("k8s.io/apimachinery/pkg/apis/meta/v1.Time"), + }, + }, + "nodes": { + VendorExtensible: spec.VendorExtensible{ + Extensions: spec.Extensions{ + "x-kubernetes-list-map-keys": []interface{}{ + "name", + }, + "x-kubernetes-list-type": "map", + }, + }, + SchemaProps: spec.SchemaProps{ + Description: "nodes provides detailed information about each node in the cluster including per-node resource health. Each node entry includes the node's name, IP address, conditions, and resource status. The list can contain up to 32 nodes (the upper limit imposed by pacemaker). For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes are currently reporting status.", + Type: []string{"array"}, + Items: &spec.SchemaOrArray{ + Schema: &spec.Schema{ + SchemaProps: spec.SchemaProps{ + Default: map[string]interface{}{}, + Ref: ref("github.com/openshift/api/etcd/v1alpha1.PacemakerClusterNodeStatus"), + }, + }, + }, + }, + }, + }, + }, + }, + Dependencies: []string{ + "github.com/openshift/api/etcd/v1alpha1.PacemakerClusterNodeStatus", "k8s.io/apimachinery/pkg/apis/meta/v1.Condition", "k8s.io/apimachinery/pkg/apis/meta/v1.Time"}, + } +} + func schema_openshift_api_example_v1_CELUnion(ref common.ReferenceCallback) common.OpenAPIDefinition { return common.OpenAPIDefinition{ Schema: spec.Schema{ diff --git a/openapi/openapi.json b/openapi/openapi.json index 77420dcbaa9..2d807ed4fd2 100644 --- a/openapi/openapi.json +++ b/openapi/openapi.json @@ -10158,7 +10158,7 @@ "$ref": "#/definitions/com.github.openshift.api.config.v1.PKI" }, "policyType": { - "description": "policyType is a required field specifies the type of the policy for verification. This field must correspond to how the policy was generated. Allowed values are \"PublicKey\", \"FulcioCAWithRekor\", and \"PKI\". When set to \"PublicKey\", the policy relies on a sigstore publicKey and may optionally use a Rekor verification. When set to \"FulcioCAWithRekor\", the policy is based on the Fulcio certification and incorporates a Rekor verification. When set to \"PKI\", the policy is based on the certificates from Bring Your Own Public Key Infrastructure (BYOPKI). This value is enabled by turning on the SigstoreImageVerificationPKI feature gate.", + "description": "policyType is a required field specifies the type of the policy for verification. This field must correspond to how the policy was generated. Allowed values are \"PublicKey\", \"FulcioCAWithRekor\", and \"PKI\". When set to \"PublicKey\", the policy relies on a sigstore publicKey and may optionally use a Rekor verification. When set to \"FulcioCAWithRekor\", the policy is based on the Fulcio certification and incorporates a Rekor verification. When set to \"PKI\", the policy is based on the certificates from Bring Your Own Public Key Infrastructure (BYOPKI).", "type": "string", "default": "" }, @@ -14287,6 +14287,169 @@ } } }, + "com.github.openshift.api.etcd.v1alpha1.PacemakerCluster": { + "description": "PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is \"cluster\". This resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments.\n\nCompatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support.", + "type": "object", + "required": [ + "metadata" + ], + "properties": { + "apiVersion": { + "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources", + "type": "string" + }, + "kind": { + "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds", + "type": "string" + }, + "metadata": { + "description": "metadata is the standard object's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata", + "default": {}, + "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ObjectMeta" + }, + "status": { + "description": "status contains the actual pacemaker cluster status information collected from the cluster. The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources.", + "$ref": "#/definitions/com.github.openshift.api.etcd.v1alpha1.PacemakerClusterStatus" + } + } + }, + "com.github.openshift.api.etcd.v1alpha1.PacemakerClusterList": { + "description": "PacemakerClusterList contains a list of PacemakerCluster objects. PacemakerCluster is a cluster-scoped singleton resource; only one instance named \"cluster\" may exist. This list type exists only to satisfy Kubernetes API conventions.\n\nCompatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support.", + "type": "object", + "required": [ + "items" + ], + "properties": { + "apiVersion": { + "description": "APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources", + "type": "string" + }, + "items": { + "description": "items is a list of PacemakerCluster objects.", + "type": "array", + "items": { + "default": {}, + "$ref": "#/definitions/com.github.openshift.api.etcd.v1alpha1.PacemakerCluster" + } + }, + "kind": { + "description": "Kind is a string value representing the REST resource this object represents. Servers may infer this from the endpoint the client submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds", + "type": "string" + }, + "metadata": { + "description": "metadata is the standard list's metadata. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#metadata", + "default": {}, + "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.ListMeta" + } + } + }, + "com.github.openshift.api.etcd.v1alpha1.PacemakerClusterNodeStatus": { + "description": "PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including the node's conditions and the health of critical resources running on that node.", + "type": "object", + "required": [ + "name", + "ipAddresses", + "resources" + ], + "properties": { + "conditions": { + "description": "conditions represent the observations of the node's current state. Known condition types are: \"Healthy\", \"Online\", \"InService\", \"Active\", \"Ready\", \"Clean\", \"Member\". The \"Healthy\" condition is an aggregate that tracks the overall health of the node. The \"Online\" condition tracks whether the node is online. The \"InService\" condition tracks whether the node is in service (not in maintenance mode). The \"Active\" condition tracks whether the node is active (not in standby mode). The \"Ready\" condition tracks whether the node is ready (not in a pending state). The \"Clean\" condition tracks whether the node is in a clean (status known) state. The \"Member\" condition tracks whether the node is a member of the cluster. Each of these conditions is required, so the array must contain at least 7 items. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + "type": "array", + "items": { + "default": {}, + "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.Condition" + }, + "x-kubernetes-list-map-keys": [ + "type" + ], + "x-kubernetes-list-type": "map" + }, + "ipAddresses": { + "description": "ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. Pacemaker allows multiple IP addresses for Corosync communication between nodes. The first address in this list is used for IP-based peer URLs for etcd membership. Each address must be in canonical form (e.g., \"192.168.1.1\" not \"192.168.001.001\", or \"2001:db8::1\" not \"2001:0db8::1\"). Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). This excludes special addresses like unspecified, loopback, link-local, and multicast.", + "type": "array", + "items": { + "type": "string", + "default": "" + }, + "x-kubernetes-list-type": "set" + }, + "name": { + "description": "name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with an alphanumeric character, and be at most 253 characters in length.", + "type": "string" + }, + "resources": { + "description": "resources contains the status of pacemaker resources on this node. Each resource entry includes the resource name and its health conditions. For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. All three resources are required to be present, so the array must contain at least 3 items. Valid resource names are \"Kubelet\", \"Etcd\", and \"FencingAgent\".", + "type": "array", + "items": { + "default": {}, + "$ref": "#/definitions/com.github.openshift.api.etcd.v1alpha1.PacemakerClusterResourceStatus" + }, + "x-kubernetes-list-map-keys": [ + "name" + ], + "x-kubernetes-list-type": "map" + } + } + }, + "com.github.openshift.api.etcd.v1alpha1.PacemakerClusterResourceStatus": { + "description": "PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. For Two Node OpenShift with Fencing, we manage three resources:\n - Kubelet (the Kubernetes node agent and a prerequisite for etcd),\n - Etcd (the distributed key-value store)\n - FencingAgent (used to isolate failed nodes during a quorum loss event)", + "type": "object", + "required": [ + "name" + ], + "properties": { + "conditions": { + "description": "conditions represent the observations of the resource's current state. Known condition types are: \"Healthy\", \"InService\", \"Managed\", \"Enabled\", \"Operational\", \"Active\", \"Started\", \"Schedulable\". The \"Healthy\" condition is an aggregate that tracks the overall health of the resource. The \"InService\" condition tracks whether the resource is in service (not in maintenance mode). The \"Managed\" condition tracks whether the resource is managed by pacemaker. The \"Enabled\" condition tracks whether the resource is enabled. The \"Operational\" condition tracks whether the resource is operational (not failed). The \"Active\" condition tracks whether the resource is active (available to be used). The \"Started\" condition tracks whether the resource is started. The \"Schedulable\" condition tracks whether the resource is schedulable (not blocked). Each of these conditions is required, so the array must contain at least 8 items. Conditions is idiomatically the first field in status structs per Kubernetes API conventions. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + "type": "array", + "items": { + "default": {}, + "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.Condition" + }, + "x-kubernetes-list-map-keys": [ + "type" + ], + "x-kubernetes-list-type": "map" + }, + "name": { + "description": "name is the name of the pacemaker resource. Valid values are \"Kubelet\", \"Etcd\", and \"FencingAgent\". The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. The FencingAgent resource is used to fence the other node during a quorum loss event.", + "type": "string" + } + } + }, + "com.github.openshift.api.etcd.v1alpha1.PacemakerClusterStatus": { + "description": "PacemakerClusterStatus contains the actual pacemaker cluster status information. As part of validating the status object, we need to ensure that the lastUpdated timestamp cannot be removed once set and cannot be set to an earlier timestamp than the current value.", + "type": "object", + "properties": { + "conditions": { + "description": "conditions represent the observations of the pacemaker cluster's current state. Known condition types are: \"Healthy\", \"InService\", \"NodeCountAsExpected\". The \"Healthy\" condition is an aggregate that tracks the overall health of the cluster. The \"InService\" condition tracks whether the cluster is in service (not in maintenance mode). The \"NodeCountAsExpected\" condition tracks whether the expected number of nodes are present. Each of these conditions is required, so the array must contain at least 3 items. Conditions is marked optional per Kubernetes API conventions because controllers populate conditions after observing state. The MinItems validation ensures conditions are present once the controller has reported status. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go", + "type": "array", + "items": { + "default": {}, + "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.Condition" + }, + "x-kubernetes-list-map-keys": [ + "type" + ], + "x-kubernetes-list-type": "map" + }, + "lastUpdated": { + "description": "lastUpdated is the timestamp when this status was last updated. This is useful for identifying stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot be removed and cannot be set to an earlier timestamp than the current value. This field is marked optional because all status fields must be optional to allow for partial updates and backward compatibility. However, once set it must always be present. See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go", + "$ref": "#/definitions/io.k8s.apimachinery.pkg.apis.meta.v1.Time" + }, + "nodes": { + "description": "nodes provides detailed information about each node in the cluster including per-node resource health. Each node entry includes the node's name, IP address, conditions, and resource status. The list can contain up to 32 nodes (the upper limit imposed by pacemaker). For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes are currently reporting status.", + "type": "array", + "items": { + "default": {}, + "$ref": "#/definitions/com.github.openshift.api.etcd.v1alpha1.PacemakerClusterNodeStatus" + }, + "x-kubernetes-list-map-keys": [ + "name" + ], + "x-kubernetes-list-type": "map" + } + } + }, "com.github.openshift.api.example.v1.CELUnion": { "description": "CELUnion demonstrates how to use a discriminated union and how to validate it using CEL.", "type": "object", diff --git a/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-CustomNoUpgrade.crd.yaml b/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-CustomNoUpgrade.crd.yaml new file mode 100644 index 00000000000..b07c5e94876 --- /dev/null +++ b/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-CustomNoUpgrade.crd.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/merged-by-featuregates: "true" + include.release.openshift.io/ibm-cloud-managed: "true" + include.release.openshift.io/self-managed-high-availability: "true" + release.openshift.io/feature-set: CustomNoUpgrade + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {} diff --git a/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-DevPreviewNoUpgrade.crd.yaml b/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-DevPreviewNoUpgrade.crd.yaml new file mode 100644 index 00000000000..22ef80423b7 --- /dev/null +++ b/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-DevPreviewNoUpgrade.crd.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/merged-by-featuregates: "true" + include.release.openshift.io/ibm-cloud-managed: "true" + include.release.openshift.io/self-managed-high-availability: "true" + release.openshift.io/feature-set: DevPreviewNoUpgrade + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {} diff --git a/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-TechPreviewNoUpgrade.crd.yaml b/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-TechPreviewNoUpgrade.crd.yaml new file mode 100644 index 00000000000..87775a56941 --- /dev/null +++ b/payload-manifests/crds/0000_25_etcd_01_pacemakerclusters-TechPreviewNoUpgrade.crd.yaml @@ -0,0 +1,455 @@ +apiVersion: apiextensions.k8s.io/v1 +kind: CustomResourceDefinition +metadata: + annotations: + api-approved.openshift.io: https://github.com/openshift/api/pull/2544 + api.openshift.io/merged-by-featuregates: "true" + include.release.openshift.io/ibm-cloud-managed: "true" + include.release.openshift.io/self-managed-high-availability: "true" + release.openshift.io/feature-set: TechPreviewNoUpgrade + name: pacemakerclusters.etcd.openshift.io +spec: + group: etcd.openshift.io + names: + kind: PacemakerCluster + listKind: PacemakerClusterList + plural: pacemakerclusters + singular: pacemakercluster + scope: Cluster + versions: + - name: v1alpha1 + schema: + openAPIV3Schema: + description: |- + PacemakerCluster represents the current state of the pacemaker cluster as reported by the pcs status command. + PacemakerCluster is a cluster-scoped singleton resource. The name of this instance is "cluster". This + resource provides a view into the health and status of a pacemaker-managed cluster in Two Node OpenShift with Fencing deployments. + + Compatibility level 4: No compatibility is provided, the API can change at any point for any reason. These capabilities should not be used by applications needing long term support. + properties: + apiVersion: + description: |- + APIVersion defines the versioned schema of this representation of an object. + Servers should convert recognized schemas to the latest internal value, and + may reject unrecognized values. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources + type: string + kind: + description: |- + Kind is a string value representing the REST resource this object represents. + Servers may infer this from the endpoint the client submits requests to. + Cannot be updated. + In CamelCase. + More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds + type: string + metadata: + type: object + status: + description: |- + status contains the actual pacemaker cluster status information collected from the cluster. + The goal of this status is to be able to quickly identify if pacemaker is in a healthy state. + In Two Node OpenShift with Fencing, a healthy pacemaker cluster has 2 nodes, both of which have healthy kubelet, etcd, and fencing resources. + properties: + conditions: + description: |- + conditions represent the observations of the pacemaker cluster's current state. + Known condition types are: "Healthy", "InService", "NodeCountAsExpected". + The "Healthy" condition is an aggregate that tracks the overall health of the cluster. + The "InService" condition tracks whether the cluster is in service (not in maintenance mode). + The "NodeCountAsExpected" condition tracks whether the expected number of nodes are present. + Each of these conditions is required, so the array must contain at least 3 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of the current + state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type NodeCountAsExpected + rule: self.exists(c, c.type == 'NodeCountAsExpected') + lastUpdated: + description: |- + lastUpdated is the timestamp when this status was last updated. This is useful for identifying + stale status reports. It must be a valid timestamp in RFC3339 format. Once set, this field cannot + be removed and cannot be set to an earlier timestamp than the current value. + This field is marked optional because all status fields must be optional to allow for partial updates + and backward compatibility. However, once set it must always be present. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/statusoptional/doc.go + format: date-time + type: string + nodes: + description: |- + nodes provides detailed information about each node in the cluster including per-node resource health. + Each node entry includes the node's name, IP address, conditions, and resource status. + The list can contain up to 32 nodes (the upper limit imposed by pacemaker). + For Two Node OpenShift with Fencing, exactly 2 nodes are expected in a healthy cluster. An empty list indicates no nodes + are currently reporting status. + items: + description: |- + PacemakerClusterNodeStatus represents the status of a single node in the pacemaker cluster including + the node's conditions and the health of critical resources running on that node. + properties: + conditions: + description: |- + conditions represent the observations of the node's current state. + Known condition types are: "Healthy", "Online", "InService", "Active", "Ready", "Clean", "Member". + The "Healthy" condition is an aggregate that tracks the overall health of the node. + The "Online" condition tracks whether the node is online. + The "InService" condition tracks whether the node is in service (not in maintenance mode). + The "Active" condition tracks whether the node is active (not in standby mode). + The "Ready" condition tracks whether the node is ready (not in a pending state). + The "Clean" condition tracks whether the node is in a clean (status known) state. + The "Member" condition tracks whether the node is a member of the cluster. + Each of these conditions is required, so the array must contain at least 7 items. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect of + the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, False, + Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 7 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type Online + rule: self.exists(c, c.type == 'Online') + - message: conditions must contain a condition of type InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type Ready + rule: self.exists(c, c.type == 'Ready') + - message: conditions must contain a condition of type Clean + rule: self.exists(c, c.type == 'Clean') + - message: conditions must contain a condition of type Member + rule: self.exists(c, c.type == 'Member') + ipAddresses: + description: |- + ipAddresses is a list of IPv4 or IPv6 addresses for the node in canonical form. + Pacemaker allows multiple IP addresses for Corosync communication between nodes. + The first address in this list is used for IP-based peer URLs for etcd membership. + Each address must be in canonical form (e.g., "192.168.1.1" not "192.168.001.001", or "2001:db8::1" not "2001:0db8::1"). + Each address must be a valid global unicast IPv4 or IPv6 address (including private/RFC1918 addresses). + This excludes special addresses like unspecified, loopback, link-local, and multicast. + items: + maxLength: 39 + type: string + x-kubernetes-validations: + - message: must be a valid global unicast IPv4 or IPv6 address + in canonical form + rule: isIP(self) && ip.isCanonical(self) && ip(self).isGlobalUnicast() + maxItems: 8 + minItems: 1 + type: array + x-kubernetes-list-type: set + name: + description: |- + name is the name of the node. This is expected to match the Kubernetes node's name, which must be a lowercase + RFC 1123 subdomain consisting of lowercase alphanumeric characters, '-' or '.', starting and ending with + an alphanumeric character, and be at most 253 characters in length. + maxLength: 253 + minLength: 1 + type: string + x-kubernetes-validations: + - message: name must be a lowercase RFC 1123 subdomain consisting + of lowercase alphanumeric characters, '-' or '.', and must + start and end with an alphanumeric character + rule: '!format.dns1123Subdomain().validate(self).hasValue()' + resources: + description: |- + resources contains the status of pacemaker resources on this node. + Each resource entry includes the resource name and its health conditions. + For Two Node OpenShift with Fencing, we manage three resources: Kubelet, Etcd, and FencingAgent. + All three resources are required to be present, so the array must contain at least 3 items. + Valid resource names are "Kubelet", "Etcd", and "FencingAgent". + items: + description: |- + PacemakerClusterResourceStatus represents the status of a pacemaker resource on a node. + A pacemaker resource is a unit of work managed by pacemaker. In pacemaker terminology, resources are services or + applications that pacemaker monitors, starts, stops, and moves between nodes to maintain high availability. + For Two Node OpenShift with Fencing, we manage three resources: + - Kubelet (the Kubernetes node agent and a prerequisite for etcd), + - Etcd (the distributed key-value store) + - FencingAgent (used to isolate failed nodes during a quorum loss event) + properties: + conditions: + description: |- + conditions represent the observations of the resource's current state. + Known condition types are: "Healthy", "InService", "Managed", "Enabled", "Operational", + "Active", "Started", "Schedulable". + The "Healthy" condition is an aggregate that tracks the overall health of the resource. + The "InService" condition tracks whether the resource is in service (not in maintenance mode). + The "Managed" condition tracks whether the resource is managed by pacemaker. + The "Enabled" condition tracks whether the resource is enabled. + The "Operational" condition tracks whether the resource is operational (not failed). + The "Active" condition tracks whether the resource is active (available to be used). + The "Started" condition tracks whether the resource is started. + The "Schedulable" condition tracks whether the resource is schedulable (not blocked). + Each of these conditions is required, so the array must contain at least 8 items. + Conditions is idiomatically the first field in status structs per Kubernetes API conventions. + Conditions is marked optional per Kubernetes API conventions because controllers populate conditions + after observing state. The MinItems validation ensures conditions are present once the controller + has reported status. + See: https://github.com/kubernetes-sigs/kube-api-linter/blob/main/pkg/analysis/conditions/doc.go + items: + description: Condition contains details for one aspect + of the current state of this API Resource. + properties: + lastTransitionTime: + description: |- + lastTransitionTime is the last time the condition transitioned from one status to another. + This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable. + format: date-time + type: string + message: + description: |- + message is a human readable message indicating details about the transition. + This may be an empty string. + maxLength: 32768 + type: string + observedGeneration: + description: |- + observedGeneration represents the .metadata.generation that the condition was set based upon. + For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date + with respect to the current state of the instance. + format: int64 + minimum: 0 + type: integer + reason: + description: |- + reason contains a programmatic identifier indicating the reason for the condition's last transition. + Producers of specific condition types may define expected values and meanings for this field, + and whether the values are considered a guaranteed API. + The value should be a CamelCase string. + This field may not be empty. + maxLength: 1024 + minLength: 1 + pattern: ^[A-Za-z]([A-Za-z0-9_,:]*[A-Za-z0-9_])?$ + type: string + status: + description: status of the condition, one of True, + False, Unknown. + enum: + - "True" + - "False" + - Unknown + type: string + type: + description: type of condition in CamelCase or in + foo.example.com/CamelCase. + maxLength: 316 + pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$ + type: string + required: + - lastTransitionTime + - message + - reason + - status + - type + type: object + maxItems: 16 + minItems: 8 + type: array + x-kubernetes-list-map-keys: + - type + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: conditions must contain a condition of type + Healthy + rule: self.exists(c, c.type == 'Healthy') + - message: conditions must contain a condition of type + InService + rule: self.exists(c, c.type == 'InService') + - message: conditions must contain a condition of type + Managed + rule: self.exists(c, c.type == 'Managed') + - message: conditions must contain a condition of type + Enabled + rule: self.exists(c, c.type == 'Enabled') + - message: conditions must contain a condition of type + Operational + rule: self.exists(c, c.type == 'Operational') + - message: conditions must contain a condition of type + Active + rule: self.exists(c, c.type == 'Active') + - message: conditions must contain a condition of type + Started + rule: self.exists(c, c.type == 'Started') + - message: conditions must contain a condition of type + Schedulable + rule: self.exists(c, c.type == 'Schedulable') + name: + description: |- + name is the name of the pacemaker resource. + Valid values are "Kubelet", "Etcd", and "FencingAgent". + The Kubelet resource is a prerequisite for etcd in Two Node OpenShift with Fencing deployments. + The Etcd resource may temporarily transition to stopped during pacemaker quorum-recovery operations. + The FencingAgent resource is used to fence the other node during a quorum loss event. + enum: + - Kubelet + - Etcd + - FencingAgent + type: string + required: + - name + type: object + maxItems: 8 + minItems: 3 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + x-kubernetes-validations: + - message: resources must contain a resource named Kubelet + rule: self.exists(r, r.name == 'Kubelet') + - message: resources must contain a resource named Etcd + rule: self.exists(r, r.name == 'Etcd') + - message: resources must contain a resource named FencingAgent + rule: self.exists(r, r.name == 'FencingAgent') + required: + - ipAddresses + - name + - resources + type: object + maxItems: 32 + type: array + x-kubernetes-list-map-keys: + - name + x-kubernetes-list-type: map + type: object + x-kubernetes-validations: + - message: lastUpdated cannot be removed once set + rule: '!has(oldSelf.lastUpdated) || has(self.lastUpdated)' + - message: lastUpdated cannot be set to an earlier timestamp + rule: '!has(oldSelf.lastUpdated) || !has(self.lastUpdated) || self.lastUpdated + >= oldSelf.lastUpdated' + required: + - metadata + type: object + x-kubernetes-validations: + - message: PacemakerCluster must be named 'cluster' + rule: self.metadata.name == 'cluster' + served: true + storage: true + subresources: + status: {}