Create Python-based Scorecard

[bikegeek]

Replace italics below with details for this issue.

Describe the New Feature

Generate a scorecard, utilizing the following:

• METdataio METreformat code to format the MET .stat output based on linetype
• METcalcpy agg_stat.py module to calculate the bootstrap confidence limits using the reformatted METdataio data as input
• METcalcpy scorecard.py to calculate the p-values and DIFF or DIFF_STAT derived statistics used to determine degrees of statistical significance between models
  • [https://github.com/Make necessary modifications to scorecard module to support a basic scorecard METcalcpy#467](METcalcpy Feature 461 make compare images configurable #467)
• METplotpy scorecard_plot.py to control all aspects of generating the scorecard plot (from MET .stat input to a plot)

Acceptance Testing

Time Estimate

~ 1 1/2 months

Sub-Issues

Consider breaking the new feature down into sub-issues.

• METcalcpy scorecard.py working with YAML config file
• METdataio reformatting of sample NSSL data
• METcalcpy agg_stat.py to calculate bootstrap confidence limits on NSSL sample data
• METcalcpy scorecard.py on output from agg_stat.py to calculate p-values, etc.
• METplotpy generate scorecard plot via scorecard_plot.py
  • YAML config file for plotting
  • Identify how to combine configurations used in METdataio and METcalcpy
  • Consider subsetting an filtering of data early in process to avoid performance degradation for large datasets
• Compare METplotpy with METviewer-generated scorecard to verify that results are consistent
• Provide documentation for NOAA scientists to run using their data

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

• Select engineer(s)
• Select scientist(s) NOAA EMC scientist(s)

Labels

• Review default alert labels
• Select component(s)
• Select priority
• Select requestor(s)

Milestone and Projects###

• Select Milestone as a METplotpy-X.Y.Z version, Consider for Next Release, or Backlog of Development Ideas
• For a METplotpy-X.Y.Z version, select the METplotpy-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

• METplus, MET, METdataio, METviewer, METexpress, METcalcpy, METplotpy

New Feature Checklist

See the METplus Workflow for details.

• Complete the issue definition above, including the Time Estimate and Funding source.
• Fork this repository or create a branch of develop.
  Branch name: feature_<Issue Number>_<Description>
• Complete the development and test your changes.
• Add/update log messages for easier debugging.
• Add/update unit tests.
• Add/update documentation.
• Add any new Python packages to the METplus Components Python Requirements table.
• Push local changes to GitHub.
• Submit a pull request to merge into develop.
  Pull request: feature <Issue Number> <Description>
• Define the pull request metadata, as permissions allow.
  Select: Reviewer(s) and Development issue
  Select: Milestone as the next official version
  Select: METplotpy-X.Y.Z Development project for development toward the next official release
• Iterate until the reviewer(s) accept and merge your changes.
• Delete your fork or branch.
• Close this issue.