-
Notifications
You must be signed in to change notification settings - Fork 261
Draft: Advanced side quests for custom training #756
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
pinin4fjords
wants to merge
111
commits into
master
Choose a base branch
from
custom-training-advanced-topics
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Three new side quests for advanced Nextflow topics: 1. Workflow Output Definitions (45 mins) - New publish/output block syntax - Dynamic paths based on metadata - Index file generation 2. Running on Cloud Executors (30 mins) - Executor concepts and options - AWS Batch configuration patterns - S3 integration and cost management - Conceptual/config-focused (no cloud account needed) 3. Plugin Development (90 mins) - Full hands-on plugin creation - Custom functions with @function annotation - Build, test, and use locally - Complete working nf-greeting plugin solution Also adds Gradle to devcontainer for plugin development support. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
✅ Deploy Preview for nextflow-training ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
Contributor
|
Nextflow linting complete! ❌ 5 files had 22 errors 💡 Tip: Click filename locations to go directly to that code. View all 22 issues
View formatting changes
|
- Align markdown table columns - Expand devcontainer JSON to multi-line format 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Makefiles require tabs for recipe indentation (Make syntax requirement). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add new Section 1 covering plugin usage before development content
- Include examples of installing plugins via plugins {} block
- Show how to import and use plugin functions with include syntax
- Reference nf-schema as practical example (from Hello nf-core)
- Add table of popular community plugins
- Update learning goals to include plugin usage
- Renumber subsequent sections (1→2, 2→3, etc.)
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The gradle-sdkman feature fails with SDKMAN permission errors during build. Plugin projects include a Gradle wrapper (./gradlew) that downloads Gradle automatically, so we only need Java pre-installed (provided by java:1 feature). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add prominent warning that this is an advanced topic - Add info box explaining Java, Groovy, and Gradle for newcomers - Highlight that "Using existing plugins" section is most important - Add detailed breakdown of Groovy syntax with comparisons to Python - Explain what unit tests are and why they matter - Add troubleshooting tips for build failures - Mark "Going further" section as reference material - Add note that Gradle wrapper downloads Gradle automatically These changes make the tutorial more accessible to participants without a software engineering background. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The command requires both plugin name and organization name arguments. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Update tree output to show actual generated files - Change package from nextflow.greeting to training.plugin - Rename classes to NfGreetingExtension, NfGreetingPlugin, etc. - Update all code examples and file paths to match - Update test class names in examples and expected output - Fix build.gradle example to show correct extensionPoints 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Use correct plugin version (0.0.1-alpha4) - Use correct nextflowVersion (24.10.0) - Remove NfGreetingObserver from extensionPoints (not included by default) - Add publishing block with registry configuration 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Show the actual generated template code and the changes needed, making it clearer what participants need to modify. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The generated Makefile uses 'assemble' not 'build' for compilation. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Shows the Gradle download on first run and sets expectations for subsequent faster builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Reassure beginners that deprecation warnings are safe to ignore and explain what each part of the output means. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Clarify that we're creating a new test file (not editing existing) - Use Spock framework syntax to match generated project style - Add note explaining Spock's given/expect syntax - Update test output to show Spock-style test names 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Gradle hides test details on success, which can confuse beginners. Added explanation that BUILD SUCCESSFUL means tests passed, with optional --info flag for verbose output. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The --info flag produces verbose Gradle output, not clean test results. Instead, point users to the HTML test report for detailed results. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Show how to use python http.server to view the Gradle test report in VS Code, with screenshot showing all tests passing. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The starter files already have the plugin configuration. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Shows the starter file and the changes needed to use plugin functions, with highlighted lines and explanation of key changes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The starter config has the plugin commented out. Users need to uncomment it AND add the @0.1.0 version for local plugins. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added note explaining the startup/complete messages come from the NfGreetingObserver included in the plugin template. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Added section 1.6 with a practical exercise using nf-hello plugin. Users create a simple workflow that uses the sayHello function, seeing plugin download and usage in action. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Added 'Why use plugins instead of local functions?' section - Restructured exercise to show local function first, then plugin - Before/After tabs show the code change clearly - Explains the value proposition: versioned, shareable code 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Connect trace observers to the messages users already saw - Add Before/After exercise to customize observer messages - Add table of available observer hooks - Mark operators as reference (more complex) - Update publishing section with clearer steps 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ion types - Add progress table showing coverage of extension types from table 2.1 - Add TaskCounterObserver creation exercise (new file, not editing existing) - Add custom operator (shoutAll) with full Before/After convention - Add configurable decorator exercise with Before/After - Add conceptual coverage of Factories, Executors, and Filesystems - Update publishing section with complete instructions - Clean up solution main.nf formatting Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix duplicate 5.4 numbering (install locally → 5.5) - Split monolithic section 7 into focused sections: - Section 7: Trace observers - Section 8: Custom operators and factories - Section 9: Configuration - Section 10: Publishing your plugin - Add Takeaway/What's next transitions between new sections Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Replace implicit 'it' with explicit parameter in summary example - Update section references from 3-10 to 3-11 throughout - Add operators and trace observers to development checklist - Change "publish" to "distribute" in learning goals Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix hl_lines to highlight correct lines (24-29 for imports, 62-73 for operator) - Add line-by-line breakdown explaining DataflowHelper, callbacks, and Channel.STOP Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix malformed hl_lines (24-16 -> 24-26) - Add key changes explanation for TaskCounterObserver (verbose flag, constructor, conditional print) - Add explanation for NfGreetingFactory config reading Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…-io/training into custom-training-advanced-topics
- Convert section 0's #### headings to numbered ### subsections (0.1-0.6) to match the style used throughout the rest of the document - Use consistent dot notation for taskCounter.verbose in section 9.2 to match section 9.3's style Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Improved the explanation to acknowledge that publishDir can be configured via processes or config selectors (withName/withLabel), and explained what a manifest is for readers unfamiliar with the term. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Explains the fundamental shift: with publishDir, processes handle their own publishing; with workflow outputs, publishing moves to the workflow level. This provides context before diving into syntax details. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- CLAUDE.md: Add detailed explanation that hl_lines is snippet-relative, with worked example showing how to count lines (including blanks) - new-lesson.md: Add concise instruction for setting hl_lines correctly - New check-highlights skill: Verification tool for reviewing highlight line numbers in existing documentation Key points reinforced: count from line 1 of snippet, blank lines count, hl_lines is independent of linenums. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Section 1 now starts with processes that already have metadata tuples (greeting and language), so adding tuples isn't conflated with the workflow output syntax - Section 2 now only teaches removing publishDir and adding the workflow output syntax - a single focused concept - Updated starter files (main.nf, modules/greetings.nf) to match docs - Fixed hl_lines values that were off-by-one Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
When a workflow uses a publish: section, the workflow content must be placed in a main: block. Updated the After example and explanation. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Shows the complete main.nf file with and without the output block, making the change clearer and more consistent with other sections. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The training environment uses an older Nextflow version that requires this flag. Added note explaining this is not needed in 25.10+. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The solution was missing the required main: block when using publish: Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The nextflow.preview.output flag must be in the script file, not nextflow.config. Updated documentation and solution to add the flag at the top of main.nf when the output block is added. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Include nextflow.preview.output in the starter main.nf so users don't need to add it manually. Updated all code blocks in documentation to include the flag consistently. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- CSV output has no header row (tuples don't have field names) - Paths are absolute, not relative - Clarified tip about using maps for named fields Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Refactored to use nf-core community standard [meta, file] tuples - meta map contains id and language fields - Updated all code examples to use meta.language for dynamic paths - Changed index format from CSV to JSON (works better with meta maps) - JSON properly expands meta maps to objects with named fields - Removed outdated tip about using maps (now using them from the start) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- JSON index shows all 5 greetings with proper /workspaces paths - File listing sorted alphabetically to match real output Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Line 25 was the process declaration, not the second publishDir which is on line 27. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Based on feedback from @bentsherman: custom operators/factories are in a precarious position for future type checking support, and nearly every example would have been better as a custom function. Changes: - Remove Section 8 (Custom operators and factories) entirely - Remove Section 10.1 (Channel factories) - Renumber sections 9-11 to 8-10 - Update all references to operators/factories throughout - Update code examples to use functions instead of shoutAll operator - Keep focus on functions and trace observers as the key extension types Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove shoutAll operator from main.nf solution, use reverseGreeting function instead to match the updated tutorial content. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Change "functions, factories, etc." to "functions, observers, etc." to avoid confusion with channel factories (which were removed). Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR contains custom training content for specific teaching sessions. It is not intended to be merged into the main training materials at this time.
Summary
Three new side quests covering advanced Nextflow topics:
1. Workflow Output Definitions (45 mins)
publish:andoutput {}block syntax (replacingpublishDir)2. Running on Cloud Executors (30 mins)
3. Plugin Development (90 mins)
@Functionannotationnf-greetingplugin solution includedAdditional Changes
Testing
Notes
Keep on branch for custom training sessions. May be adapted for general availability later.
🤖 Generated with Claude Code