readme roadsigns

.gitignore

@@ -117,3 +117,4 @@ dist
 *.whl
 .vscode
 docs/build
+pynumdiff/_version.py

README.md

@@ -49,43 +49,19 @@ For more details, refer to [this paper](https://doi.org/10.1109/ACCESS.2020.3034
 
 ## Structure
 
-    PyNumDiff/
-      |- README.md
-      |- pynumdiff/
-         |- __init__.py
-         |- __version__.py
-         |- finite_difference/
-         |- kalman_smooth/
-         |- linear_model/
-         |- smooth_finite_difference/
-         |- total_variation_regularization/
-         |- utils/
-         |- optimize/
-            |- __init__.py
-            |- __optimize__.py
-            |- finite_difference/
-            |- kalman_smooth/
-            |- linear_model/
-            |- smooth_finite_difference/
-            |- total_variation_regularization/
-         |- tests/
-      |- examples
-         |- 1_basic_tutorial.ipynb
-         |- 2a_optimizing_parameters_with_dxdt_known.ipynb
-         |- 2b_optimizing_parameters_with_dxdt_unknown.ipynb
-      |- docs/
-         |- Makefile
-         |- make.bat
-         |- build/
-         |- source/
-            |- _static
-            |- _summaries
-            |- conf.py
-            |- index.rst
-            |- ...
-      |- .gitignore
-      |- LICENSE.txt
-      |- pyproject.toml
+- `.github/workflows` contains `.yaml` that configures our GitHub Actions continuous integration (CI) runs.
+- `docs/` contains `make` files and `.rst` files to govern the way `sphinx` builds documentation, either locally by navigating to this folder and calling `make html` or in the cloud by `readthedocs.io`.
+- `examples/` contains Jupyter notebooks that demonstrate some usage of the library.
+- `pynumdiff/` contains the source code. For a full list of modules and further navigation help, see the readme in this subfolder.
+- `.editorconfig` ensures tabs are displayed as 4 characters wide.
+- `.gitignore` ensures files generated by local `pip install`s, Jupyter notebook runs, caches from code runs, virtual environments, and more are not picked up by `git` and accidentally added to the repo.
+- `.pylintrc` configures `pylint`, a tool for autochecking code quality.
+- `.readthedocs.yaml` configures `readthedocs` and is necessary for documentation to get auto-rebuilt.
+- `CITATION.cff` is citation information for the Journal of Open-Source Software (JOSS) paper associated with this project.
+- `LICENSE.txt` allows free usage of this project.
+- `README.md` is the text you're reading, hello.
+- `linting.py` is a script to run `pylint`.
+- `pyproject.toml` governs how this package is set up and installed, including dependencies.
 
 ## Citation
 
@@ -121,11 +97,12 @@ See CITATION.cff file as well as the following references.
 
 ### Prerequisite
 
-PyNumDiff requires common packages like `numpy`, `scipy`, `matplotlib`, `pytest` (for unittests), `pylint` 
-(for PEP8 style check). For a full list, you can check the file [pyproject.toml](pyproject.toml)
+PyNumDiff requires common packages like `numpy`, `scipy`, and `matplotlib`. For a full list, you can check the file [pyproject.toml](pyproject.toml)
 
 In addition, it also requires certain additional packages for select functions, though these are not required for a successful install of PyNumDiff:
-* Total Variation Regularization methods: [`cvxpy`](http://www.cvxpy.org/install/index.html)
+- Total Variation Regularization methods: [`cvxpy`](http://www.cvxpy.org/install/index.html)
+- `pytest` for unittests
+- `pylint` for PEP8 style check
 
 When using `cvxpy`, our default solver is set to be `MOSEK` (highly recommended), you would need to download their 
 free academic license from their [website](https://www.mosek.com/products/academic-licenses/). Otherwise, you can also 
@@ -147,7 +124,6 @@ again.
 
 <em>Note: If using the optional MOSEK solver for cvxpy you will also need a [MOSEK license](https://www.mosek.com/products/academic-licenses/), free academic license.</em>
 
-
 ## Usage
 
 **PyNumDiff** uses [Sphinx](http://www.sphinx-doc.org/en/stable/) for code documentation.
@@ -189,25 +165,23 @@ We will frequently update simple examples for demo purposes, and here are curren
 * Parameter Optimization with known ground truth (only for demonstration purpose):  [2a_optimizing_parameters_with_dxdt_known.ipynb](examples/2a_optimizing_parameters_with_dxdt_known.ipynb)
 * Parameter Optimization with unknown ground truth:  [2b_optimizing_parameters_with_dxdt_unknown.ipynb](./examples/2b_optimizing_parameters_with_dxdt_unknown.ipynb)
 
-
 ### Important notes
 
 * Larger values of `tvgamma` produce smoother derivatives
 * The value of `tvgamma` is largely universal across methods, making it easy to compare method results
 * The optimization is not fast. Run it on subsets of your data if you have a lot of data. It will also be much faster with faster differentiation methods, like savgoldiff and butterdiff, and probably too slow for sliding methods like sliding DMD and sliding LTI fit. 
 * The following heuristic works well for choosing `tvgamma`, where `cutoff_frequency` is the highest frequency content of the signal in your data, and `dt` is the timestep: `tvgamma=np.exp(-1.6*np.log(cutoff_frequency)-0.71*np.log(dt)-5.1)`
 
-
 ### Running the tests
 
-We are using Travis CI for continuous intergration testing. You can check out the current status 
-[here](https://travis-ci.com/github/florisvb/PyNumDiff).
+We are using GitHub Actions for continuous intergration testing.
 
 To run tests locally, type:
 ```bash
 > pytest pynumdiff
 ```
 
+Add the flag `--plot` to see plots of the methods against test functions.
 
 ## License
 

linting.py

@@ -1,20 +1,13 @@
 # -*- coding: utf-8 -*-
-import re
-import sys
-import argparse
+import re, sys, argparse
 from pylint import lint
 
+# Call `echo $?` to see the exit code after a run. If the score is over this, the exit
+# code will be 0 (success), and if not will be nonzero (failure).
 THRESHOLD = 8.5
 
 if len(sys.argv) < 2:
     raise argparse.ArgumentError("Module to evaluate needs to be the first argument")
 
 sys.argv[1] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[1])
-run = lint.Run([sys.argv[1]], do_exit=False)
-score = run.linter.stats['global_note']
-
-if score < THRESHOLD:
-    print("Your code doesn't pass the PEP8 style score threshold: %f!" % THRESHOLD)
-    sys.exit(1)
-
-print("Congratulations! Your code has passed the PEP8 style score threshold: %f!" % THRESHOLD)
+run = lint.Run([sys.argv[1], f"--fail-under={THRESHOLD}"])

pynumdiff/README.md

@@ -0,0 +1,11 @@
+- `finite_difference` contains basic first and second order finite differencing methods. The first order method supports iterative application.
+- `kalman_smooth` contains Kalman filtering and smoothing methods, currently constant-derivative methods up to 3rd order (jerk) and a classic linear Kalman Filter based on known dynamics.
+- `linear_model` is a bit of a miscellaneous module, containing methods which work linearly: `lineardiff`, `polydiff`, `savgoldiff`, and `spectraldiff`.
+- `optimize` contains code to find best parameter settings for methods, tuned using Nelder-Mead according to the paper "Numerical differentiation of noisy data: A unifying multi-objective optimization framework"
+- `smooth_finite_difference` contains methods which do a smoothing step followed by simple finite difference.
+- `tests` contains `pytest` unit tests of
+	1. all the differentiation methods, checking their results against a suite of known analytic functions (including an ability to plot if the `--plot` command is passed to `pytest`, see `conftest.py`)
+	2. the optimizer
+	3. utilities, auxiliary functions used throughout the code
+- `total_variation_regularization` contains code to take the derivative based on a finite differencing scheme which is regularized by shrinking changes of value in some derivative (1st, 2nd, or 3rd order)
+- `utils` contains `utility` functions used throughout differentation methods, `evaluate` functions used by the parameter optimizer, and `simulate` examples for demonstrating and testing the methods.