[Test-ObjectGraph] When the schema specifies that a parameter is required and another parameter is not correct, that other parameter is not listed as failed.

[ykuijs]

When the schema specifies that a parameter is required and another parameter is not correct (incorrect type or using parameter that is unknown), those other parameters are not listed as failed.

Reproduction:

• DefaultLength is specified as string, but the schema expects and int.
• The property DoesNotExist is not specified in the schema.

$data = @{
    NonNodeData = @{
        AzureAD = @{
            AuthenticationMethodPolicy = @(
                @{
                    DefaultLength = 'string'
                    DoesNotExist = 'string'
                    DefaultLifetimeInMinutes = 10
                }
            )
        }
    }
}

$schema = @{
    NonNodeData = @{
        '@Type' = 'PSMapNode'
        AzureAD = @{
            '@Type' = 'PSMapNode'
            AuthenticationMethodPolicy = @(
                @{
                    '@Type' = 'PSMapNode'
                    DefaultLength = @{ '@Type' = 'Int' }
                    DefaultLifetimeInMinutes = @{ '@Type' = 'Int' }
                    Ensure = @{ '@Type' = 'String' }
                    Id = @{ '@Type' = 'String'; '@Required' = $true }
                    IncludeTargets = @(
                        @{
                            '@Type' = 'PSMapNode'
                            Id = @{ '@Type' = 'String' }
                            TargetType = @{ '@Type' = 'String' }
                        }
                    )
                    IsUsableOnce = @{ '@Type' = 'Bool' }
                    MaximumLifetimeInMinutes = @{ '@Type' = 'Int' }
                    MinimumLifetimeInMinutes = @{ '@Type' = 'Int' }
                    State = @{ '@Type' = 'String' }
                }
            )
        }
    }
}

$result = $data | Test-Object $schema
$result

Output:

Path                                              Value                                    Valid Issue
----                                              -----                                    ----- -----
NonNodeData.AzureAD.AuthenticationMethodPolicy[0] @{DefaultLeng…='string';DefaultLife…=10}  [X]  The required node condition 'Id' is not met
NonNodeData.AzureAD.AuthenticationMethodPolicy    @(@{DefaultLeng…=…;…})                    [X]  The following nodes are not accepted: 0
NonNodeData.AzureAD                               @{Authenticat…=@(…)}                      [X]  The following nodes are not accepted: Authenticat…
NonNodeData                                       @{AzureAD=@{Authenticat…=…}}              [X]  The following nodes are not accepted: AzureAD
                                                  @{NonNodeData=@{AzureAD=…}}               [X]  The following nodes are not accepted: NonNodeData

The incorrect type of the DefaultLength parameter and the fact that the DoesNotExist parameter is unknown are missing in the list. I would have expected that the code would output all discovered issues.

[iRon7]

Based on this feedback, I have implemented an result class to better handle the result output.
The output of the concerned test is now:

Path                                                            Value                                                          Valid Issue
----                                                            -----                                                          ----- -----
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node Id does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0].DefaultLength 'string'                                                        [X]  The node 'string' is not of type 'Int'
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node IsUsableOnce does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node Ensure does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node IncludeTargets does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node MaximumLifetimeInMinutes does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node State does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node MinimumLifetimeInMinutes does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]               @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The following nodes are not accepted: DoesNotExist, DefaultLength

And elaborated (-Elaborate):

Path                                                                       Value                                                          Valid Issue
----                                                                       -----                                                          ----- -----
NonNodeData                                                                @{AzureAD=@{Authenticat…=…}}                                    [V]  The node or value is of type 'PSMapNode'
NonNodeData.AzureAD                                                        @{Authenticat…=@(…)}                                            [V]  The node or value is of type 'PSMapNode'
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [V]  The node or value is of type 'PSMapNode'
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node Id does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [V]  The child node requirement 'Id' is not met
NonNodeData.AzureAD.AuthenticationMethodPolicy[0].DefaultLength            'string'                                                        [X]  The node 'string' is not of type 'Int'
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node IsUsableOnce does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node Ensure does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node IncludeTargets does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0].DefaultLifetimeInMinutes 10                                                              [V]  The node or value is of type 'Int'
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node MaximumLifetimeInMinutes does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node State does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The node MinimumLifetimeInMinutes does not exist
NonNodeData.AzureAD.AuthenticationMethodPolicy[0]                          @{DefaultLife…=10;DoesNotExist='string';DefaultLeng…='string'}  [X]  The following nodes are not accepted: DoesNotExist, DefaultLength
NonNodeData.AzureAD.AuthenticationMethodPolicy                             @(@{DefaultLife…=…;…})                                          [X]  The following nodes are not accepted: 0
NonNodeData.AzureAD                                                        @{Authenticat…=@(…)}                                            [X]  The following nodes are not accepted: AuthenticationMethodPolicy
NonNodeData                                                                @{AzureAD=@{Authenticat…=…}}                                    [X]  The following nodes are not accepted: AzureAD
                                                                           @{NonNodeData=@{AzureAD=…}}                                     [X]  The following nodes are not accepted: NonNodeData

Background

Below is some background information on the challenges with I would like to document and confirm a few statements in my Pester tests.

The previous version, had basically two possible output modes for each node test:

• Output mode (-not $ValidateOnly), in this mode the results record is immediately released to the pipeline
• Validation mode, in this mode the test is either valid valid or not (-ValidateOnly) when a node is found invalid, the specific node test notifies the parent test which checks other requirements and options and notifies its own parent and so on.
  This Validation mode is also used in the output mode to determine whether a branch in an object graph is valid or not.

Unfortunately as appears from this issue, it isn't enough to test whether a object graph branch is valid or not, it is (sometimes) also required to determine (and feedback) why it (or multiple branches) failed. Hence the result class I have created for the new version. This result class has now an additional 3rd collection mode that is able to collect the results of a branch and pass that to the parent test.

Note here that:

• The pipeline can't be used to return a failing ($false result) or a result collection to the pipeline (as that blur the general pipeline immediate output mode)
• A failed branch doesn't necessarily mean that the parent fails as shown in the following examples:

Example

A specific configuration has some CMO and FMO nodes:

$Data = @{
    Company = @(
        @{ Env = 'CMO'; Id = 1234},
        @{ Env = 'FMO'; Id = 1235},
        @{ Env = 'FMO'; Id = 2345}
    )
}

For a specific FMO application it is important that the company node has at least 1 valid FMO node in the list:

$Schema = @{
    Company = @{
        '@Type' = 'Array'
        '@AllowExtraNodes' = $true
        FMO = @{
            Env = @{ '@Type' = 'String'; '@Like' = 'FMO' }
            Id  = @{ '@Type' = 'Int'; '@Minimum' = 2000 }
        }
    }
}

The example $Data is accepted ($data | Test-Object $schema) even there are some nodes in the company list invalid (these object-graph branches might be inspected but rejected until an acceptable FMO node is found):

For another company wide application you might want to confirm that all the nodes in the company are either CMO - or FMO compliant:

$Schema = @{
   Company = @{
       '@Type' = 'Array'
       '@RequiredNodes' = 'CMO or FMO'
       CMO = @{
           Env = @{ '@Type' = 'String'; '@Like' = 'CMO' }
           Id  = @{ '@Type' = 'Int'; '@Maximum' = 1999 }
       }
       FMO = @{
           Env = @{ '@Type' = 'String'; '@Like' = 'FMO' }
           Id  = @{ '@Type' = 'Int'; '@Minimum' = 2000 }
       }
   }
}

$Data = @{
   Company = @(
       @{ Env = 'CMO'; Id = 1234 },
       @{ Env = 'FMO'; Id = 1235 }, # Id should be >= 2000
       @{ Env = 'FMO'; Id = 2345 }
   )
}

The above $Data configuration will fail because there is an illegal FMO ID:

But might pass if there e.g. only (valid) FMO entries...

The branch that contains this enhancements, can be found here: https://github.com/iRon7/ObjectGraphTools/tree/iRon7-Updates