Observable Datatype's relationship to Observable Objects

[mikeradka]

Originated from ocsf-schema PR ocsf/ocsf-schema#807

I believe there is an important relationship between the observable datatypes and how the observable objects are identified.

For instance, I believe the OCSF translator looks at the datatype, and when the datatype of a given object matches an observable type, it identifies that object as an observable.

Therefore, removal of an observable datatype from an object could be a breaking change.

We should find some way to work this into our documentation (and our process)

[mikeradka]

1. I tested this locally by translating an event with observables using a schema.json where the user object was of datatype username_t:

$ ocsf-cli -p WinEventLog -R rules -r rule-4624-m -S schema-test1.json -o data/4624_0.event | jq -S .observables

...
{
    "name": "user",
    "type": "User",
    "type_id": 21
  },
  {
    "name": "user.name",
    "type": "User Name",
    "type_id": 4,
    "value": "iiwu"
  },
  {
    "name": "logon_process",
    "type": "Process",
    "type_id": 25
  },
...

2. Next, I removed the username_t as the datatype for the user object, and translated the same event with observables using the new schema.json:

$ ocsf-cli -p WinEventLog -R rules -r rule-4624-m -S schema-test2.json -o data/4624_0.event | jq -S .observables

...
{
    "name": "user",
    "type": "User",
    "type_id": 21
  },
  {
    "name": "logon_process",
    "type": "Process",
    "type_id": 25
  },
...

The result: when removing the datatype username_t from the user object, the 'nested' observables (in this case user.name) are not identified.

This leads me to conclude that removing an observable datatype from an object qualifies as a breaking change, as an observable datatype is required for nested observables.

[pagbabian-splunk]

We should add this to the breaking changes restrictions.

[pagbabian-splunk]

I haven't looked at this in a while but as of today, no dictionary attribute is of type username_t and objects are their own type, so the user attribute is of type user. However, the user.name attribute is overridden to be of type username_t and so it will be picked up as Observable type 4.

The user object is observable no. 21.

Since @rmouritzen-splunk enhanced observables so we now have observables by Dictionary Attribute as well as by Dictionary Type we could have a dictionary attribute as an observable but there is no scalar username attribute at the moment.

To your example above, if you change the data type itself of an attribute, including an overridden attribute's data type, it will be breaking. That is separate from removing the observable metadata qualifier for an attribute based observable, but that might also be considered breaking.