Skip to content

command/pr: Support for only running specifically named test patterns (still with auto-detection) #25

@bflad

Description

@bflad

Description

Currently changes to shared files with large amounts of unit testing introduces a large amount of testing noise when running via tctest:

tctest pr 13143
Discovering tests for pr #13143 (https://github.com/terraform-providers/terraform-provider-aws/pull/13143)...
    TestValidateOnceADayWindowFormat
    TestValidateDmsReplicationTaskId
    TestValidateDbParamGroupName
    TestValidateKmsKey
    TestValidateDxConnectionBandWidth
    TestValidateArn
    TestValidateAwsKmsName
    TestValidateCognitoUserPoolEmailVerificationMessage
    TestValidateCognitoUserPoolSmsAuthenticationMessage
    TestValidateIamRoleDescription
    TestValidateTypeStringNullableFloat
    TestValidateLogGroupNamePrefix
    TestValidateDmsReplicationSubnetGroupId
    TestValidateSSMDocumentPermissions
    TestValidateSecurityGroupRuleDescription
    TestValidateLaunchTemplateName
    TestValidateCIDRNetworkAddress
    TestValidateCognitoProviderDeveloperName
    TestResourceAWSElastiCacheReplicationGroupAuthTokenValidation
    TestValidateLambdaPermissionEventSourceToken
    TestValidateEC2AutomateARN
    TestValidateSagemakerName
    TestValidateApiGatewayUsagePlanQuotaSettings
    TestValidateNeptuneEventSubscriptionNamePrefix
    TestValidateDbSubnetGroupName
    TestValidateNeptuneParamGroupNamePrefix
    TestValidateCloudFrontPublicKeyName
    TestValidateAwsKmsGrantName
    TestValidateWafMetricName
    TestValidateCognitoUserGroupName
    TestValidateSecretManagerSecretNamePrefix
    TestValidateLogMetricTransformationName
    TestValidateS3BucketLifecycleTimestamp
    TestValidateSQSFifoQueueName
    TestValidateStepFunctionStateMachineName
    TestValidateAccountAlias
    TestValidateIamRoleProfileNamePrefix
    TestValidateAwsSSMName
    TestValidateCognitoRoleMappingsRulesConfiguration
    TestValidateNeptuneParamGroupName
    TestAccAWSSSMDocument
    TestValidateCloudWatchEventRuleName
    TestValidateAwsAccountId
    TestValidatePolicyStatementId
    TestValidateLogGroupName
    TestValidateIAMPolicyJsonString
    TestValidateEcsPlacementStrategy
    TestValidateDmsCertificateId
    TestValidateNeptuneEventSubscriptionName
    TestValidateDbOptionGroupNamePrefix
    TestValidateCognitoUserPoolEmailVerificationSubject
    TestValidateRoute53ResolverName
    TestValidateLambdaFunctionName
    TestValidateDocDBIdentifier
    TestValidateDbOptionGroupName
    TestValidateOpenIdURL
    TestValidateCognitoSupportedLoginProviders
    TestValidateCognitoIdentityProvidersProviderName
    TestValidateBatchName
    TestValidateCloudFrontPublicKeyNamePrefix
    TestValidateSQSQueueName
    TestValidateDmsEndpointId
    TestValidateCognitoRoleMappingsAmbiguousRoleResolutionAgainstType
    TestValidateCognitoUserPoolId
    TestValidateTypeStringNullableBoolean
    TestValidateLbTargetGroupName
    TestValidateLbTargetGroupNamePrefix
    TestValidateSecretManagerSecretName
    TestAccAWSCognitoIdentityPoolRolesAttachment
    TestValidateCloudWatchDashboardName
    TestValidateLambdaPermissionAction
    TestValidateOnceAWeekWindowFormat
    TestValidateEcsPlacementConstraint
    TestValidateEmrCustomAmiId
    TestValidateIamRoleProfileName
    TestValidateNeptuneSubnetGroupName
    TestValidateDbSubnetGroupNamePrefix
    TestValidateCognitoRoles
    TestValidateElbNamePrefix
    TestValidateNeptuneSubnetGroupNamePrefix
    TestValidateCognitoIdentityPoolName
    TestValidateBatchPrefix
    TestValidateLambdaQualifier
    TestValidateDbEventSubscriptionName
    TestValidateCognitoUserPoolSmsVerificationMessage
    TestValidateLogMetricFilterName
    TestValidateCloudFormationTemplate
    TestValidateDmsReplicationInstanceId
    TestValidateElbName
    TestValidateCognitoIdentityProvidersClientId
    TestValidateAmazonSideAsn
    TestValidateLaunchTemplateId
triggering refs/pull/13143/merge for (TestValidateOnceADayWindowFormat|TestValidateDmsReplicationTaskId|TestValidateDbParamGroupName|TestValidateKmsKey|TestValidateDxConnectionBandWidth|TestValidateArn|TestValidateAwsKmsName|TestValidateCognitoUserPoolEmailVerificationMessage|TestValidateCognitoUserPoolSmsAuthenticationMessage|TestValidateIamRoleDescription|TestValidateTypeStringNullableFloat|TestValidateLogGroupNamePrefix|TestValidateDmsReplicationSubnetGroupId|TestValidateSSMDocumentPermissions|TestValidateSecurityGroupRuleDescription|TestValidateLaunchTemplateName|TestValidateCIDRNetworkAddress|TestValidateCognitoProviderDeveloperName|TestResourceAWSElastiCacheReplicationGroupAuthTokenValidation|TestValidateLambdaPermissionEventSourceToken|TestValidateEC2AutomateARN|TestValidateSagemakerName|TestValidateApiGatewayUsagePlanQuotaSettings|TestValidateNeptuneEventSubscriptionNamePrefix|TestValidateDbSubnetGroupName|TestValidateNeptuneParamGroupNamePrefix|TestValidateCloudFrontPublicKeyName|TestValidateAwsKmsGrantName|TestValidateWafMetricName|TestValidateCognitoUserGroupName|TestValidateSecretManagerSecretNamePrefix|TestValidateLogMetricTransformationName|TestValidateS3BucketLifecycleTimestamp|TestValidateSQSFifoQueueName|TestValidateStepFunctionStateMachineName|TestValidateAccountAlias|TestValidateIamRoleProfileNamePrefix|TestValidateAwsSSMName|TestValidateCognitoRoleMappingsRulesConfiguration|TestValidateNeptuneParamGroupName|TestAccAWSSSMDocument|TestValidateCloudWatchEventRuleName|TestValidateAwsAccountId|TestValidatePolicyStatementId|TestValidateLogGroupName|TestValidateIAMPolicyJsonString|TestValidateEcsPlacementStrategy|TestValidateDmsCertificateId|TestValidateNeptuneEventSubscriptionName|TestValidateDbOptionGroupNamePrefix|TestValidateCognitoUserPoolEmailVerificationSubject|TestValidateRoute53ResolverName|TestValidateLambdaFunctionName|TestValidateDocDBIdentifier|TestValidateDbOptionGroupName|TestValidateOpenIdURL|TestValidateCognitoSupportedLoginProviders|TestValidateCognitoIdentityProvidersProviderName|TestValidateBatchName|TestValidateCloudFrontPublicKeyNamePrefix|TestValidateSQSQueueName|TestValidateDmsEndpointId|TestValidateCognitoRoleMappingsAmbiguousRoleResolutionAgainstType|TestValidateCognitoUserPoolId|TestValidateTypeStringNullableBoolean|TestValidateLbTargetGroupName|TestValidateLbTargetGroupNamePrefix|TestValidateSecretManagerSecretName|TestAccAWSCognitoIdentityPoolRolesAttachment|TestValidateCloudWatchDashboardName|TestValidateLambdaPermissionAction|TestValidateOnceAWeekWindowFormat|TestValidateEcsPlacementConstraint|TestValidateEmrCustomAmiId|TestValidateIamRoleProfileName|TestValidateNeptuneSubnetGroupName|TestValidateDbSubnetGroupNamePrefix|TestValidateCognitoRoles|TestValidateElbNamePrefix|TestValidateNeptuneSubnetGroupNamePrefix|TestValidateCognitoIdentityPoolName|TestValidateBatchPrefix|TestValidateLambdaQualifier|TestValidateDbEventSubscriptionName|TestValidateCognitoUserPoolSmsVerificationMessage|TestValidateLogMetricFilterName|TestValidateCloudFormationTemplate|TestValidateDmsReplicationInstanceId|TestValidateElbName|TestValidateCognitoIdentityProvidersClientId|TestValidateAmazonSideAsn|TestValidateLaunchTemplateId)...
  bflad@--OMITTED--#Aws_ProviderAwsAlternate
  build 111059 queued: https://--OMITTED--/viewQueued.html?itemId=111059

It would be great if tctest provided a command option to control this behavior, such as --filter or --include-name-pattern or --include-file-pattern, beyond the hardcoded specifying the explicit test name patterns to match the auto-detected pattern. 👍 e.g.

$ tctest pr 13143 --include-name-pattern TestAcc
Discovering tests for pr #13143 (https://github.com/terraform-providers/terraform-provider-aws/pull/13143)...
    TestAccAWSSSMDocument
    TestAccAWSCognitoIdentityPoolRolesAttachment
triggering refs/pull/13143/merge for (TestAccAWSSSMDocument|TestAccAWSCognitoIdentityPoolRolesAttachment)...
  bflad@--OMITTED--#Aws_ProviderAwsAlternate
  build 111059 queued: https://--OMITTED--/viewQueued.html?itemId=111059

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions