Doc - How to contribute new modes #443
Open
+128
−0
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
keiran-rowell-unsw
added
the
documentation
|
Author
|
@JoseEspinosa happy to edit however or knock back, just would have found this helpful getting up to speed on |
Member
Will take a look tomorrow. Thanks! |
JoseEspinosa
requested changes
Jan 19, 2026
Member
There was a problem hiding this comment.
Just added some suggestions and corrected few "tyops" 😛
Feel free to reject the ones you think are not ok.
This is really awesome @keiran-rowell-unsw ! 🚀
| @@ -0,0 +1,128 @@ | |||
| ## Guidance on how to add a new --mode (i.e. structure prediction software) to ProteinFold | |||
Member
There was a problem hiding this comment.
Suggested change
| ## Guidance on how to add a new --mode (i.e. structure prediction software) to ProteinFold | |
| ## Adding structure prediction modes to nf-core/proteinfold | |
| This section provides guidance on adding new structure prediction modes, implemented via the `--mode` option, to nf-core/proteinfold. |
|
|
||
| ### Contributing | ||
|
|
||
| One of the great advantages of an `nf-core` pipeline is the community can add new protein structure prediction modules as they are released, while still leveraging the workflow infrastructure and reports developed for `proteinfold`. |
Member
There was a problem hiding this comment.
Suggested change
| One of the great advantages of an `nf-core` pipeline is the community can add new protein structure prediction modules as they are released, while still leveraging the workflow infrastructure and reports developed for `proteinfold`. | |
| One of the great advantages of an `nf-core` pipeline is that the community can extend workflows to add new functionalities. In nf-core/proteinfold, this allows adding new protein structure prediction modules as they are released, while still leveraging the existing workflow infrastructure and reporting. |
|
|
||
| One of the great advantages of an `nf-core` pipeline is the community can add new protein structure prediction modules as they are released, while still leveraging the workflow infrastructure and reports developed for `proteinfold`. | ||
|
|
||
| Please consider writing some code to become a [nf-core contributor](https://nf-co.re/contributors) and expand the pipeline! Reach out to a maintainer of contributor for guidance :) |
Member
There was a problem hiding this comment.
Maybe also mention the #proteinfold_dev slack channel? I think it would be easy to people to just write a message on slack and we are all there
| - `"""script block"""`: | ||
| - `program`: the script block calls the program from the Nextflow shell with the programs typical `--flags`, in whatever form (`binary` or `script.py`) the program is distributed from its codebase repository. | ||
| - `extract_metrics.py`: accesses the canonical data output formats from the structure prediction program and returns a core set of plain text `.tsv` metric files. | ||
| - `bin/extract_metrics.py`: a globally accessible program to go from serialised data -> `.tsv` plaintext. Currently runs particular extraction logic functions based upon file format (`.pkl`, `.json`, `.npz`). However, as the commnity adds more `--mode`s to the pipeline, different programs could use the same compressed output format. In which case `extract_metrics.py` should be refactored to match based on the passing the `--mode` to `extract_metrics.py`. |
Member
There was a problem hiding this comment.
Suggested change
| - `bin/extract_metrics.py`: a globally accessible program to go from serialised data -> `.tsv` plaintext. Currently runs particular extraction logic functions based upon file format (`.pkl`, `.json`, `.npz`). However, as the commnity adds more `--mode`s to the pipeline, different programs could use the same compressed output format. In which case `extract_metrics.py` should be refactored to match based on the passing the `--mode` to `extract_metrics.py`. | |
| - `bin/extract_metrics.py`: a globally accessible program to go from serialised data into `.tsv` plaintext. It currently applies format specific extraction logic for `.pkl`, `.json` and `.npz` files. However, as the community adds more `--mode`s to the pipeline, different programs could use the same compressed output format. In which case `extract_metrics.py` should be refactored to match based on the passing the `--mode` to `extract_metrics.py`. |
|
|
||
| ### Process labelling | ||
|
|
||
| At the top of a module's `RUN_[MODE_NAME]`{} process there are a series of labels that allow the `nextflow.config` to pass the job to the approriate resources on the compute cluster. `label 'process_gpu'` is very useful to specify this is the AI inference stage requiring GP-GPU grunt -- whereas other processes can have default labels that request CPU resources and, once finished, will naturally cascade onto GPUs due to Nextflow's dataflow paradigm. |
Member
There was a problem hiding this comment.
Suggested change
| At the top of a module's `RUN_[MODE_NAME]`{} process there are a series of labels that allow the `nextflow.config` to pass the job to the approriate resources on the compute cluster. `label 'process_gpu'` is very useful to specify this is the AI inference stage requiring GP-GPU grunt -- whereas other processes can have default labels that request CPU resources and, once finished, will naturally cascade onto GPUs due to Nextflow's dataflow paradigm. | |
| At the top of a module's `RUN_[MODE_NAME]`{} process, there are a series of labels that allow the `nextflow.config` to pass the job to the appropriate resources on the compute cluster. `label 'process_gpu'` is very useful to specify the AI inference stages requiring GPU-intensive computation. Other processes can use default labels that request CPU resources and, once finished, will naturally cascade onto GPU-enabled steps due to Nextflow's dataflow paradigm. |
|
|
||
| ### Processable structure prediction metrics | ||
|
|
||
| Metrics from AlphaFold-inspired protein strucutre prediction programs are structured in two ways: tabular or as a matrix (PAE values) |
Member
There was a problem hiding this comment.
Suggested change
| Metrics from AlphaFold-inspired protein strucutre prediction programs are structured in two ways: tabular or as a matrix (PAE values) | |
| Metrics from AlphaFold-inspired protein structure prediction programs are structured in two ways: tabular or as a matrix (PAE values) |
|
|
||
| When contributing a new mode to `proteinfold`, functionality should be added to `extract_metrics.py` to access the canonical ouput files of the new program, and extract data into compliant `.tsv` files that can be easily processed by downstream plotting and MultiQC functions. | ||
|
|
||
| Metrics files are **0 indexed**. |
Member
There was a problem hiding this comment.
Suggested change
| Metrics files are **0 indexed**. | |
| > [!WARNING] | |
| > Metrics files are **0 indexed**. |
|
|
||
| #### pLDDT (`{meta.id}_plddt.tsv`) | ||
|
|
||
| Confidence values per residue, rounded to 2 decimal places. Each ranked result gets its own column. [For all-atom modules, atomic token confidences are processed to a naive mean value across the residue] |
Member
There was a problem hiding this comment.
Suggested change
| Confidence values per residue, rounded to 2 decimal places. Each ranked result gets its own column. [For all-atom modules, atomic token confidences are processed to a naive mean value across the residue] | |
| Confidence values per residue, rounded to 2 decimal places. Each ranked result gets its own column (for all-atom modules, atomic token confidences are processed to a naive mean value across the residue). |
|
|
||
| #### (i)pTM (`{meta.id}_[i]ptm.tsv`) | ||
|
|
||
| (i)pTM scores, rounded to 3 decimal places, listed by the rank number. [Currently unsorted] |
Member
There was a problem hiding this comment.
Suggested change
| (i)pTM scores, rounded to 3 decimal places, listed by the rank number. [Currently unsorted] | |
| (i)pTM scores, rounded to 3 decimal places, listed by the rank number (currently unsorted). |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
A Pipeline structure and metrics precis, to guide addition of any new
proteinfold--modes.Designed to quickly orient new contributors of the main places to edit to add a newly release protein structure prediction program, not provide fine-grained implementation details.