schema-strict: Add Compile to JSON Schema

[jarofgreen]

We are looking to basically replicate the functionality of https://compiletojsonschema.readthedocs.io/en/latest/ in this tool.

So that's a command line feature to take a JSON Schema file and process it including:

• Bringing in any $ref's that are there so the output is one JSON Schema file.
• Have the option to preserve title and description when processing refs
• Have the option for codelists - loading enum options from a csv in a given directory
• Have the option to set additional properties false everywhere

Some of the current ocdskit commands modify files in place - the current compiletojsonschema tool writes to standard output and lets the user redirect. One of the common use cases for the compiletojsonschema tool is taking the schema files from a data standard and making one stricter file for CoVE apps, so in this case altering in place would not be a smooth workflow.

We'd want to make sure the $ref processing code is modular, as a later issue will want to go over a schema to build a report ( https://github.com/OpenDataServices/json-schema-auditor ) and this will want to use the $ref processing code.

Noting that we are currently having issues getting urn: refs to work, but we can tackle that later? gazpachoking/jsonref#73

This issue is to discuss approaches, before we start coding.

[jpmckinney]

Features

Bringing in any $ref's that are there so the output is one JSON Schema file.

👍 I think we can just use jsonref's replace_refs.

Have the option to preserve title and description when processing refs

Yes - you should use jsonref's merge_props keyword argument to do that.

Have the option for codelists - loading enum options from a csv in a given directory

Yes - is this already covered by set-closed-codelist-enums?

It can perhaps be merged into the schema-strict command as an option.

Have the option to set additional properties false everywhere

Yes, we can change schema-strict to do nothing by default, and opt-in to different changes.

Some of the current ocdskit commands modify files in place

We can change the default to standard output, and add a flag to write in-place.

Approach

We can maybe move some of the standard-agnostic logic to https://jscc.readthedocs.io/en/latest/ and then work on the schema-strict interface.

What do you mean by "make sure the $ref processing code is modular"?

[jpmckinney]

Regarding:

a later issue will want to go over a schema to build a report ( https://github.com/OpenDataServices/json-schema-auditor )

Do you mean a later issue will be opened to merge that into ocdskit (e.g. schema-report command) and/or jscc? In general, OCDS Kit's schema commands could be moved to jscc. I just haven't because releasing new versions of jscc in order to add new features to OCDS Kit has been unnecessary overhead.

cc @duncandewhurst

[duncandewhurst]

Do you mean a later issue will be opened to merge that into ocdskit (e.g. schema-report command) and/or jscc?

Yep, that's the plan (to merge that functionality into schema-report, wherever it resides). I can open a separate issue about it, since it'll make more sense for me to do that as I wrote the auditor.

[jarofgreen]

We can maybe move some of the standard-agnostic logic to https://jscc.readthedocs.io/en/latest/ and then work on the schema-strict interface.

I wasn't sure of the timeline - if the move is good to do first that makes sense to me and I can wait.

What do you mean by "make sure the $ref processing code is modular"?

I think later chat has covered this - we want to add to the schema-report code, but for the use we have we want to make sure $refs are followed. So it's just making sure that whatever we use for refs can be used here too.

[jpmckinney]

We can maybe move some of the standard-agnostic logic to https://jscc.readthedocs.io/en/latest/ and then work on the schema-strict interface.
I wasn't sure of the timeline - if the move is good to do first that makes sense to me and I can wait.

I am happy to do things in OCDS Kit first. No need to wait :)

If some code is in commands that we want to use as library functions, we can move them into new/existing modules.