Skip to content

Rethink storage of generated artifacts: OWL files and tables #37

@jamesaoverton

Description

@jamesaoverton

As a general rule, version control should only contain code and data for building final artifacts -- everything needed to rebuild completely, but nothing that can simply be rebuilt.

While our ontologies have been small, we've been lazy and included our generated files in version control, e.g. mro.owl and most of the iedb/ directory. Now that the MRO generated files are getting larger than a few megabytes, we should rethink things and do it the right way.

Our code and data files are still small, so I don't see a need for the Large File Storage extension.

I prefer to attach generated files to GitHub Releases. The Release is really just a git tag, so you can find the exact code. GitHub allows each attachment to be up to 2GB:

https://docs.github.com/en/github/managing-large-files/distributing-large-binaries

Things that need to change:

  • write a script to automate releases
  • attach mro.owl to release, stop storing in git
  • update PURL config
  • separate out the generated files in iedb/, attach them as a mro-iedb.zip file to the release OR drop this stuff from this repo and build it somewhere else

Thoughts?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions