OSD-28241: Add multi-environment OCM and backplane connection support

[nephomaniac]

OSD-28241: Add multi-environment OCM and backplane connection support

Summary

This PR makes use of recent backplane-cli changes allowing OCM connections to be passed and persisted to the backplane functions. This PR adds support for managing OCM and backplane connections to clusters across different OCM environments (production, staging, integration). This enables workflows where target clusters and their hive clusters exist in different OCM environments.

Problem Statement

Currently, osdctl assumes that target clusters and their corresponding hive clusters exist in the same OCM environment. This limitation prevents SREs from managing clusters that span multiple OCM environments, such as:

• Target cluster in staging/integration, hive cluster in production
• Testing scenarios requiring connections to multiple OCM environments
• Cross-environment cluster management workflows

Solution

This PR introduces new utility functions and client helpers that accept OCM SDK connections as parameters, allowing explicit control over which OCM environment is used for each operation.

Key Features

1. Multi-Environment OCM Connections

• Support for URL aliases: production, staging, integration
• Ability to create connections to different OCM environments simultaneously
• Explicit connection management with proper cleanup

2. Cross-Environment Hive Access

• GetHiveBPClientForCluster() - Connect to hive clusters in different OCM environments
• GetHiveClusterWithConn() - Fetch hive cluster metadata using custom connections
• Supports both elevated (cluster-admin) and standard access

3. Connection-Based Client Helpers

• NewWithConn() / NewAsBackplaneClusterAdminWithConn() - Create k8s clients with custom OCM connections
• GetKubeConfigAndClientWithConn() - Get kubeconfig and clientset with custom connections
• Maintains backward compatibility with existing functions

Changes

New Functions

pkg/utils/ocm.go (+285 lines)

• GetHiveBPClientForCluster() - Create backplane client for hive cluster with optional OCM URL
• GetHiveClusterWithConn() - Fetch hive cluster using separate target and hive OCM connections
• GetHiveShardWithConn() - Get hive shard URL with custom OCM connection
• CreateConnectionWithUrl() - Create OCM connection with URL or alias (production/staging/integration)
• GetOCMConfigFromEnv() - Load OCM configuration from environment variables
• GetOcmConfigFromFilePath() - Load OCM configuration from JSON file
• GetOCMSdkConnBuilderFromConfig() - Build OCM connection from config object
• GetOCMSdkConnBuilderFromFilePath() - Build OCM connection from config file
• GetOCMSdkConnFromFilePath() - Create OCM connection directly from file

pkg/k8s/client.go (+41 lines)

• NewWithConn() - Create Kubernetes client with provided OCM connection
• NewAsBackplaneClusterAdminWithConn() - Create elevated Kubernetes client with provided OCM connection

cmd/common/helpers.go (+29 lines)

• GetKubeConfigAndClientWithConn() - Get kubeconfig and clients using provided OCM connection

cmd/hive/loginhivetests.go (NEW, 519 lines)

• Comprehensive integration test command for validating client connections
• Tests OCM connections, hive cluster discovery, and backplane access across environments
• 13 individual test functions covering all new functionality

New Test Files

cmd/common/helpers_test.go (+70 lines)

• Tests for GetKubeConfigAndClientWithConn()
• Validates nil connection handling and parameter validation

pkg/k8s/client_test.go (+117 lines)

• Tests for NewWithConn() and NewAsBackplaneClusterAdminWithConn()
• Covers nil connection, empty cluster ID, and edge cases

pkg/utils/ocm_test.go (+602 lines)

• Comprehensive tests for all new OCM utility functions
• Tests config loading, connection building, URL aliases
• Tests hive cluster operations and connection management

Testing

Test Coverage

• 768 lines of unit tests across 3 test files
• 519 lines of integration tests in login-tests command

Manual Testing

(Update: this test has been moved to https://github.com/nephomaniac/osdctl/tree/integration-tests with information on how to build and run the tests in the osdctl/test/ dir)

The new osdctl hive login-tests command provides real-world validation:

• Connects to target cluster using standard OCM environment
• Discovers and connects to hive cluster (potentially in different OCM environment)
• Tests both elevated and non-elevated access
• Validates all new connection helper functions

Usage Examples

Example 1: Access hive cluster in production when target is in staging

// Target cluster is in staging (using default OCM env vars)
clusterID := "abc123"

// Connect to hive cluster in production
hiveClient, err := utils.GetHiveBPClientForCluster(
    clusterID,
    client.Options{},
    "investigating cluster issue",  // elevation reason
    "production",                    // hive OCM URL alias
)

Example 2: Use custom OCM connections

// Create connection to staging environment
stagingConn, _ := utils.CreateConnectionWithUrl("staging")
defer stagingConn.Close()

// Create connection to production environment
prodConn, _ := utils.CreateConnectionWithUrl("production")
defer prodConn.Close()

// Get hive cluster using both connections
hiveCluster, err := utils.GetHiveClusterWithConn(
    clusterID,
    stagingConn,  // for target cluster
    prodConn,     // for hive cluster
)

Example 3: Integration test command

# Test connections to cluster in staging with hive in production
osdctl hive login-tests \
    --cluster-id abc123 \
    --hive-ocm-url production

# Or use config file
osdctl hive login-tests \
    --cluster-id abc123 \
    --hive-ocm-config ~/.ocm-prod.json

Backward Compatibility

✅ All existing functions remain unchanged

• New functions follow the "WithConn" naming convention
• Existing code continues to work without modifications
• No breaking changes to public APIs

[openshift-ci-robot]

@nephomaniac: This pull request references OSD-28241 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

OSD-28241: Add multi-environment OCM and backplane connection support

Summary

This PR adds support for managing OCM and backplane connections to clusters across different OCM environments (production, staging, integration). This enables workflows where target clusters and their hive clusters exist in different OCM environments.

Problem Statement

Currently, osdctl assumes that target clusters and their corresponding hive clusters exist in the same OCM environment. This limitation prevents SREs from managing clusters that span multiple OCM environments, such as:

• Target cluster in staging/integration, hive cluster in production
• Testing scenarios requiring connections to multiple OCM environments
• Cross-environment cluster management workflows

Solution

This PR introduces new utility functions and client helpers that accept OCM SDK connections as parameters, allowing explicit control over which OCM environment is used for each operation.

Key Features

1. Multi-Environment OCM Connections

• Support for URL aliases: production, staging, integration
• Ability to create connections to different OCM environments simultaneously
• Explicit connection management with proper cleanup

2. Cross-Environment Hive Access

• GetHiveBPClientForCluster() - Connect to hive clusters in different OCM environments
• GetHiveClusterWithConn() - Fetch hive cluster metadata using custom connections
• Supports both elevated (cluster-admin) and standard access

3. Connection-Based Client Helpers

• NewWithConn() / NewAsBackplaneClusterAdminWithConn() - Create k8s clients with custom OCM connections
• GetKubeConfigAndClientWithConn() - Get kubeconfig and clientset with custom connections
• Maintains backward compatibility with existing functions

Changes

New Functions

pkg/utils/ocm.go (+285 lines)

• GetHiveBPClientForCluster() - Create backplane client for hive cluster with optional OCM URL
• GetHiveClusterWithConn() - Fetch hive cluster using separate target and hive OCM connections
• GetHiveShardWithConn() - Get hive shard URL with custom OCM connection
• CreateConnectionWithUrl() - Create OCM connection with URL or alias (production/staging/integration)
• GetOCMConfigFromEnv() - Load OCM configuration from environment variables
• GetOcmConfigFromFilePath() - Load OCM configuration from JSON file
• GetOCMSdkConnBuilderFromConfig() - Build OCM connection from config object
• GetOCMSdkConnBuilderFromFilePath() - Build OCM connection from config file
• GetOCMSdkConnFromFilePath() - Create OCM connection directly from file

pkg/k8s/client.go (+41 lines)

• NewWithConn() - Create Kubernetes client with provided OCM connection
• NewAsBackplaneClusterAdminWithConn() - Create elevated Kubernetes client with provided OCM connection

cmd/common/helpers.go (+29 lines)

• GetKubeConfigAndClientWithConn() - Get kubeconfig and clients using provided OCM connection

cmd/hive/loginhivetests.go (NEW, 519 lines)

• Comprehensive integration test command for validating client connections
• Tests OCM connections, hive cluster discovery, and backplane access across environments
• 13 individual test functions covering all new functionality

New Test Files

cmd/common/helpers_test.go (+70 lines)

• Tests for GetKubeConfigAndClientWithConn()
• Validates nil connection handling and parameter validation

pkg/k8s/client_test.go (+117 lines)

• Tests for NewWithConn() and NewAsBackplaneClusterAdminWithConn()
• Covers nil connection, empty cluster ID, and edge cases

pkg/utils/ocm_test.go (+602 lines)

• Comprehensive tests for all new OCM utility functions
• Tests config loading, connection building, URL aliases
• Tests hive cluster operations and connection management

Testing

Test Coverage

• 768 lines of unit tests across 3 test files
• 519 lines of integration tests in login-tests command

Manual Testing

The new osdctl hive login-tests command provides real-world validation:

• Connects to target cluster using standard OCM environment
• Discovers and connects to hive cluster (potentially in different OCM environment)
• Tests both elevated and non-elevated access
• Validates all new connection helper functions

Usage Examples

Example 1: Access hive cluster in production when target is in staging

// Target cluster is in staging (using default OCM env vars)
clusterID := "abc123"

// Connect to hive cluster in production
hiveClient, err := utils.GetHiveBPClientForCluster(
   clusterID,
   client.Options{},
   "investigating cluster issue",  // elevation reason
   "production",                    // hive OCM URL alias
)

Example 2: Use custom OCM connections

// Create connection to staging environment
stagingConn, _ := utils.CreateConnectionWithUrl("staging")
defer stagingConn.Close()

// Create connection to production environment
prodConn, _ := utils.CreateConnectionWithUrl("production")
defer prodConn.Close()

// Get hive cluster using both connections
hiveCluster, err := utils.GetHiveClusterWithConn(
   clusterID,
   stagingConn,  // for target cluster
   prodConn,     // for hive cluster
)

Example 3: Integration test command

# Test connections to cluster in staging with hive in production
osdctl hive login-tests \
   --cluster-id abc123 \
   --hive-ocm-url production

# Or use config file
osdctl hive login-tests \
   --cluster-id abc123 \
   --hive-ocm-config ~/.ocm-prod.json

Backward Compatibility

✅ All existing functions remain unchanged

• New functions follow the "WithConn" naming convention
• Existing code continues to work without modifications
• No breaking changes to public APIs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: nephomaniac
Once this PR has been reviewed and has the lgtm label, please assign devppratik for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci-robot]

@nephomaniac: This pull request references OSD-28241 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

OSD-28241: Add multi-environment OCM and backplane connection support

Summary

This PR makes use of recent backplane-cli changes allowing OCM connections to be passed and persisted to the backplane functions. This PR adds support for managing OCM and backplane connections to clusters across different OCM environments (production, staging, integration). This enables workflows where target clusters and their hive clusters exist in different OCM environments.

Problem Statement

Currently, osdctl assumes that target clusters and their corresponding hive clusters exist in the same OCM environment. This limitation prevents SREs from managing clusters that span multiple OCM environments, such as:

• Target cluster in staging/integration, hive cluster in production
• Testing scenarios requiring connections to multiple OCM environments
• Cross-environment cluster management workflows

Solution

This PR introduces new utility functions and client helpers that accept OCM SDK connections as parameters, allowing explicit control over which OCM environment is used for each operation.

Key Features

1. Multi-Environment OCM Connections

• Support for URL aliases: production, staging, integration
• Ability to create connections to different OCM environments simultaneously
• Explicit connection management with proper cleanup

2. Cross-Environment Hive Access

• GetHiveBPClientForCluster() - Connect to hive clusters in different OCM environments
• GetHiveClusterWithConn() - Fetch hive cluster metadata using custom connections
• Supports both elevated (cluster-admin) and standard access

3. Connection-Based Client Helpers

• NewWithConn() / NewAsBackplaneClusterAdminWithConn() - Create k8s clients with custom OCM connections
• GetKubeConfigAndClientWithConn() - Get kubeconfig and clientset with custom connections
• Maintains backward compatibility with existing functions

Changes

New Functions

pkg/utils/ocm.go (+285 lines)

• GetHiveBPClientForCluster() - Create backplane client for hive cluster with optional OCM URL
• GetHiveClusterWithConn() - Fetch hive cluster using separate target and hive OCM connections
• GetHiveShardWithConn() - Get hive shard URL with custom OCM connection
• CreateConnectionWithUrl() - Create OCM connection with URL or alias (production/staging/integration)
• GetOCMConfigFromEnv() - Load OCM configuration from environment variables
• GetOcmConfigFromFilePath() - Load OCM configuration from JSON file
• GetOCMSdkConnBuilderFromConfig() - Build OCM connection from config object
• GetOCMSdkConnBuilderFromFilePath() - Build OCM connection from config file
• GetOCMSdkConnFromFilePath() - Create OCM connection directly from file

pkg/k8s/client.go (+41 lines)

• NewWithConn() - Create Kubernetes client with provided OCM connection
• NewAsBackplaneClusterAdminWithConn() - Create elevated Kubernetes client with provided OCM connection

cmd/common/helpers.go (+29 lines)

• GetKubeConfigAndClientWithConn() - Get kubeconfig and clients using provided OCM connection

cmd/hive/loginhivetests.go (NEW, 519 lines)

• Comprehensive integration test command for validating client connections
• Tests OCM connections, hive cluster discovery, and backplane access across environments
• 13 individual test functions covering all new functionality

New Test Files

cmd/common/helpers_test.go (+70 lines)

• Tests for GetKubeConfigAndClientWithConn()
• Validates nil connection handling and parameter validation

pkg/k8s/client_test.go (+117 lines)

• Tests for NewWithConn() and NewAsBackplaneClusterAdminWithConn()
• Covers nil connection, empty cluster ID, and edge cases

pkg/utils/ocm_test.go (+602 lines)

• Comprehensive tests for all new OCM utility functions
• Tests config loading, connection building, URL aliases
• Tests hive cluster operations and connection management

Testing

Test Coverage

• 768 lines of unit tests across 3 test files
• 519 lines of integration tests in login-tests command

Manual Testing

The new osdctl hive login-tests command provides real-world validation:

• Connects to target cluster using standard OCM environment
• Discovers and connects to hive cluster (potentially in different OCM environment)
• Tests both elevated and non-elevated access
• Validates all new connection helper functions

Usage Examples

Example 1: Access hive cluster in production when target is in staging

// Target cluster is in staging (using default OCM env vars)
clusterID := "abc123"

// Connect to hive cluster in production
hiveClient, err := utils.GetHiveBPClientForCluster(
   clusterID,
   client.Options{},
   "investigating cluster issue",  // elevation reason
   "production",                    // hive OCM URL alias
)

Example 2: Use custom OCM connections

// Create connection to staging environment
stagingConn, _ := utils.CreateConnectionWithUrl("staging")
defer stagingConn.Close()

// Create connection to production environment
prodConn, _ := utils.CreateConnectionWithUrl("production")
defer prodConn.Close()

// Get hive cluster using both connections
hiveCluster, err := utils.GetHiveClusterWithConn(
   clusterID,
   stagingConn,  // for target cluster
   prodConn,     // for hive cluster
)

Example 3: Integration test command

# Test connections to cluster in staging with hive in production
osdctl hive login-tests \
   --cluster-id abc123 \
   --hive-ocm-url production

# Or use config file
osdctl hive login-tests \
   --cluster-id abc123 \
   --hive-ocm-config ~/.ocm-prod.json

Backward Compatibility

✅ All existing functions remain unchanged

• New functions follow the "WithConn" naming convention
• Existing code continues to work without modifications
• No breaking changes to public APIs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[nephomaniac]

/hold

[nephomaniac]

/label tide/merge-method-squash

[iamkirkbater]

Smol change on whether or not this command should be present within the shipped binary (versus having this separate as an integration test artifact).

[iamkirkbater]

Does this need to be an actual command? This feels more like an integration test to me, and shouldn't be in the "public API" of the CLI itself.

I'm not sure what the work required to "refactor" this out to be just an external test to be called elsewhere would be - but if I'm understanding the purpose of this command I'd argue that we should keep this separate from the cmd directory and run this as its own separate binary or something as an integration test.

[iamkirkbater]

To clarify a bit - I love that this exists as an artifact that "proves" that the other libraries all work - I just think it doesn't need to be an actual osdctl command. We could either package this as a separate binary in like, /hack or something, or at a minimum we could make this a "hidden" command via https://pkg.go.dev/github.com/spf13/cobra#section-readme:~:text=of%20available%20commands.-,Hidden%20bool,-//%20SilenceErrors%20is%20an

[nephomaniac]

Thanks for the comments @iamkirkbater!
Correct, this is just an integration/manual test+example usage to help others with this PR/review. It doesn't necessarily even need to be in a build path or this PR/repo, but I also wasn't sure where things like should live(?). Easy to (re)move/refactor it out.

[nephomaniac]

I've removed this test command, and updated the summary with info on where to find, build and run this test or other integration tests:
"This test has been moved to https://github.com/nephomaniac/osdctl/tree/integration-tests with information on how to build and run the tests in the osdctl/test/ directory"
This can be a follow up PR if an integration test directory/structure is desired.
Thanks!

[openshift-ci-robot]

@nephomaniac: This pull request references OSD-28241 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.22.0" version, but no target version was set.

Details

In response to this:

OSD-28241: Add multi-environment OCM and backplane connection support

Summary

This PR makes use of recent backplane-cli changes allowing OCM connections to be passed and persisted to the backplane functions. This PR adds support for managing OCM and backplane connections to clusters across different OCM environments (production, staging, integration). This enables workflows where target clusters and their hive clusters exist in different OCM environments.

Problem Statement

Currently, osdctl assumes that target clusters and their corresponding hive clusters exist in the same OCM environment. This limitation prevents SREs from managing clusters that span multiple OCM environments, such as:

• Target cluster in staging/integration, hive cluster in production
• Testing scenarios requiring connections to multiple OCM environments
• Cross-environment cluster management workflows

Solution

This PR introduces new utility functions and client helpers that accept OCM SDK connections as parameters, allowing explicit control over which OCM environment is used for each operation.

Key Features

1. Multi-Environment OCM Connections

• Support for URL aliases: production, staging, integration
• Ability to create connections to different OCM environments simultaneously
• Explicit connection management with proper cleanup

2. Cross-Environment Hive Access

• GetHiveBPClientForCluster() - Connect to hive clusters in different OCM environments
• GetHiveClusterWithConn() - Fetch hive cluster metadata using custom connections
• Supports both elevated (cluster-admin) and standard access

3. Connection-Based Client Helpers

• NewWithConn() / NewAsBackplaneClusterAdminWithConn() - Create k8s clients with custom OCM connections
• GetKubeConfigAndClientWithConn() - Get kubeconfig and clientset with custom connections
• Maintains backward compatibility with existing functions

Changes

New Functions

pkg/utils/ocm.go (+285 lines)

• GetHiveBPClientForCluster() - Create backplane client for hive cluster with optional OCM URL
• GetHiveClusterWithConn() - Fetch hive cluster using separate target and hive OCM connections
• GetHiveShardWithConn() - Get hive shard URL with custom OCM connection
• CreateConnectionWithUrl() - Create OCM connection with URL or alias (production/staging/integration)
• GetOCMConfigFromEnv() - Load OCM configuration from environment variables
• GetOcmConfigFromFilePath() - Load OCM configuration from JSON file
• GetOCMSdkConnBuilderFromConfig() - Build OCM connection from config object
• GetOCMSdkConnBuilderFromFilePath() - Build OCM connection from config file
• GetOCMSdkConnFromFilePath() - Create OCM connection directly from file

pkg/k8s/client.go (+41 lines)

• NewWithConn() - Create Kubernetes client with provided OCM connection
• NewAsBackplaneClusterAdminWithConn() - Create elevated Kubernetes client with provided OCM connection

cmd/common/helpers.go (+29 lines)

• GetKubeConfigAndClientWithConn() - Get kubeconfig and clients using provided OCM connection

cmd/hive/loginhivetests.go (NEW, 519 lines)

• Comprehensive integration test command for validating client connections
• Tests OCM connections, hive cluster discovery, and backplane access across environments
• 13 individual test functions covering all new functionality

New Test Files

cmd/common/helpers_test.go (+70 lines)

• Tests for GetKubeConfigAndClientWithConn()
• Validates nil connection handling and parameter validation

pkg/k8s/client_test.go (+117 lines)

• Tests for NewWithConn() and NewAsBackplaneClusterAdminWithConn()
• Covers nil connection, empty cluster ID, and edge cases

pkg/utils/ocm_test.go (+602 lines)

• Comprehensive tests for all new OCM utility functions
• Tests config loading, connection building, URL aliases
• Tests hive cluster operations and connection management

Testing

Test Coverage

• 768 lines of unit tests across 3 test files
• 519 lines of integration tests in login-tests command

Manual Testing

(Update: this test has been moved to https://github.com/nephomaniac/osdctl/tree/integration-tests with information on how to build and run the tests in the osdctl/test/ dir)

The new osdctl hive login-tests command provides real-world validation:

• Connects to target cluster using standard OCM environment
• Discovers and connects to hive cluster (potentially in different OCM environment)
• Tests both elevated and non-elevated access
• Validates all new connection helper functions

Usage Examples

Example 1: Access hive cluster in production when target is in staging

// Target cluster is in staging (using default OCM env vars)
clusterID := "abc123"

// Connect to hive cluster in production
hiveClient, err := utils.GetHiveBPClientForCluster(
   clusterID,
   client.Options{},
   "investigating cluster issue",  // elevation reason
   "production",                    // hive OCM URL alias
)

Example 2: Use custom OCM connections

// Create connection to staging environment
stagingConn, _ := utils.CreateConnectionWithUrl("staging")
defer stagingConn.Close()

// Create connection to production environment
prodConn, _ := utils.CreateConnectionWithUrl("production")
defer prodConn.Close()

// Get hive cluster using both connections
hiveCluster, err := utils.GetHiveClusterWithConn(
   clusterID,
   stagingConn,  // for target cluster
   prodConn,     // for hive cluster
)

Example 3: Integration test command

# Test connections to cluster in staging with hive in production
osdctl hive login-tests \
   --cluster-id abc123 \
   --hive-ocm-url production

# Or use config file
osdctl hive login-tests \
   --cluster-id abc123 \
   --hive-ocm-config ~/.ocm-prod.json

Backward Compatibility

✅ All existing functions remain unchanged

• New functions follow the "WithConn" naming convention
• Existing code continues to work without modifications
• No breaking changes to public APIs

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

@nephomaniac: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.