Studio: add recipe and config json to outputs for downloading #435

rugeli · 2025-11-25T22:06:31Z

Problem

Solution

added recipe and config data to output folder so users can download them from UI. they will appear as recipe.json and config.json

Note: we send in the full packing_config_data to output folder as a json, regardless of whether the pack used default configs or a config file

Type of change

New feature (non-breaking change which adds functionality)

github-actions · 2025-11-25T22:07:19Z

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Ingredient name	Encapsulating radius	Average number packed
ext_A	25	236.0

Packing image

Distance analysis

Expected minimum distance: 50.00
Actual minimum distance: 50.01

Ingredient key	Pairwise distance distribution
ext_A

cellpack/autopack/DBRecipeHandler.py

…in path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

cellpack/autopack/DBRecipeHandler.py

…ture/add-files-to-output

github-actions · 2025-12-01T21:30:52Z

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Ingredient name	Encapsulating radius	Average number packed
ext_A	25	236.0

Packing image

Distance analysis

Expected minimum distance: 50.00
Actual minimum distance: 50.01

Ingredient key	Pairwise distance distribution
ext_A

rugeli · 2025-12-01T22:13:12Z

.github/workflows/analyze.yml

+                  EXISTING_COMMENT_ID=$(gh pr view ${{ github.event.pull_request.number }} --json comments --jq ".comments[] | select(.body | contains(\"${MARKER}\")) | .id" | head -1 | tr -d '[:space:]')
+                  if [ -n "$EXISTING_COMMENT_ID" ] && [ "$EXISTING_COMMENT_ID" -eq "$EXISTING_COMMENT_ID" ] 2>/dev/null; then


updated the workflow to only delete redundant comment when a valid comment id found

rugeli · 2025-12-01T22:17:00Z

cellpack/autopack/DBRecipeHandler.py

+
+        output_path = Path(output_folder)
+
+        # Normalize the recipe path relative to the recipes base dir


CodeQL was failing due to an unsecured user-provided path, so I accepted the suggested fix in this spot

rugeli · 2025-12-04T20:11:35Z

@ascibisz testing for this pr is a bit funky:

temporarily set a fake environment id in pack.py Line 81,
job_id = os.environ.get("AWS_BATCH_JOB_ID", None)
change None to something like test123, you may hit aws credential issue, switch to batch role by following all the "unset AWS_ACCESS_KEY_ID ..." steps from our server planning google doc
pack a recipe with a config file and the docker flag
pack -r examples/recipes/v2/spheres_on_a_plane.json -c examples/packing-configs/debug.json -d
you should see a log like2025-12-04 11:58:08 | root | INFO | FirebaseHandler:108 | set_doc() | successfully uploaded to path: job_status/test123, then go to staging firebase to confirm the outputs directory got uploaded
go to the outputs directory, verify the recipe(full recipe data) and config(complete settings) are in the folder as two json files(recipe.json and config.json)

ascibisz · 2025-12-09T00:58:57Z

I tried running this locally and it works as expected for local config and recipe files, but it doesn't work for firebase config and recipe files (it just tries to read the firebase path as a local file path). Maybe we can discuss potential solutions at pair programming?

ERROR | DBRecipeHandler:650 | upload_packing_results_workflow() | [Errno 2] No such file or directory: '/Users/alli.scibisz/cellpack/firebase:recipes/one_sphere_v_1.0.0'

rugeli · 2025-12-09T17:27:07Z

I tried running this locally and it works as expected for local config and recipe files, but it doesn't work for firebase config and recipe files (it just tries to read the firebase path as a local file path). Maybe we can discuss potential solutions at pair programming?

darn I keep forgetting about the firebase path handling, thanks for catching it! I'll get that fixed in this pr before tomorrow.

github-actions · 2025-12-09T22:02:39Z

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Ingredient name	Encapsulating radius	Average number packed
ext_A	25	236.0

Packing image

Distance analysis

Expected minimum distance: 50.00
Actual minimum distance: 50.01

Ingredient key	Pairwise distance distribution
ext_A

cellpack/autopack/DBRecipeHandler.py

rugeli · 2025-12-09T22:11:10Z

I tried running this locally and it works as expected for local config and recipe files, but it doesn't work for firebase config and recipe files (it just tries to read the firebase path as a local file path). Maybe we can discuss potential solutions at pair programming?
ERROR | DBRecipeHandler:650 | upload_packing_results_workflow() | [Errno 2] No such file or directory: '/Users/alli.scibisz/cellpack/firebase:recipes/one_sphere_v_1.0.0'

@ascibisz Just pushed a fix for firebase recipes! It now checks for firebase recipes and I added a serialized copy of the recipe so we can json dump human readable version instead of having something like "partners": "<cellpack.autopack.interface_objects.partners.Partners object at 0x136618810>" in the json. Let me know if you like this approach, or we can chat about alternatives tomorrow!

github-actions · 2025-12-11T19:57:08Z

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Ingredient name	Encapsulating radius	Average number packed
ext_A	25	236.0

Packing image

Distance analysis

Expected minimum distance: 50.00
Actual minimum distance: 50.01

Ingredient key	Pairwise distance distribution
ext_A

github-actions · 2025-12-11T20:22:54Z

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Ingredient name	Encapsulating radius	Average number packed
ext_A	25	236.0

Packing image

Distance analysis

Expected minimum distance: 50.00
Actual minimum distance: 50.01

Ingredient key	Pairwise distance distribution
ext_A

github-actions · 2025-12-11T21:01:58Z

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Ingredient name	Encapsulating radius	Average number packed
ext_A	25	236.0

Packing image

Distance analysis

Expected minimum distance: 50.00
Actual minimum distance: 50.01

Ingredient key	Pairwise distance distribution
ext_A

rugeli · 2025-12-11T21:17:30Z

@ascibisz ready for another round of review! I removed path checks and simplified the logic so we now save recipe data(both local and firebase) directly to the output folder, no re-reading the paths. happy to walk through the changes if you'd prefer, no rush at all!

Sorry for the flood of duplicate reports, this is fixed in a different pr!

ascibisz

Looks good!! And worked well for me locally

…ture/add-files-to-output

rugeli added 2 commits November 25, 2025 13:28

add recipe and config to output

617f512

refactor

16e102f

github-advanced-security bot found potential problems Nov 25, 2025

View reviewed changes

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

rugeli requested a review from ascibisz November 26, 2025 00:04

path guard

a62c3f8

github-advanced-security bot found potential problems Dec 1, 2025

View reviewed changes

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

Potential fix for code scanning alert no. 21: Uncontrolled data used …

0051c65

…in path expression Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>

github-advanced-security bot found potential problems Dec 1, 2025

View reviewed changes

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

rugeli added 3 commits December 1, 2025 12:49

Merge branch 'main' of https://github.com/mesoscope/cellpack into fea…

7d618fe

…ture/add-files-to-output

format

7432714

fix 404 comment

6695a41

rugeli commented Dec 1, 2025

View reviewed changes

handle firebase recipe, serialize it before passing to s3

21202a6

github-advanced-security bot found potential problems Dec 9, 2025

View reviewed changes

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

cellpack/autopack/DBRecipeHandler.py Fixed Show fixed Hide fixed

save recipe data directly to outputs

586576b

remove unused import

414e493

one more unused import

b30f139

ascibisz approved these changes Dec 17, 2025

View reviewed changes

Merge branch 'main' of https://github.com/mesoscope/cellpack into fea…

5e1a617

…ture/add-files-to-output

rugeli merged commit f60a1d8 into main Dec 17, 2025
10 checks passed

rugeli deleted the feature/add-files-to-output branch December 17, 2025 20:29

		EXISTING_COMMENT_ID=$(gh pr view ${{ github.event.pull_request.number }} --json comments --jq ".comments[] \| select(.body \| contains(\"${MARKER}\")) \| .id" \| head -1 \| tr -d '[:space:]')
		if [ -n "$EXISTING_COMMENT_ID" ] && [ "$EXISTING_COMMENT_ID" -eq "$EXISTING_COMMENT_ID" ] 2>/dev/null; then


		output_path = Path(output_folder)

		# Normalize the recipe path relative to the recipes base dir

Studio: add recipe and config json to outputs for downloading #435

Studio: add recipe and config json to outputs for downloading #435

Uh oh!

Conversation

rugeli commented Nov 25, 2025

Problem

Solution

Type of change

Uh oh!

github-actions bot commented Nov 25, 2025

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Packing image

Distance analysis

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Dec 1, 2025

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Packing image

Distance analysis

Uh oh!

rugeli Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

rugeli Dec 1, 2025

Choose a reason for hiding this comment

Uh oh!

rugeli commented Dec 4, 2025

Uh oh!

ascibisz commented Dec 9, 2025

Uh oh!

rugeli commented Dec 9, 2025

Uh oh!

github-actions bot commented Dec 9, 2025

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Packing image

Distance analysis

Uh oh!

Uh oh!

Uh oh!

rugeli commented Dec 9, 2025

Uh oh!

github-actions bot commented Dec 11, 2025

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Packing image

Distance analysis

Uh oh!

github-actions bot commented Dec 11, 2025

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Packing image

Distance analysis

Uh oh!

github-actions bot commented Dec 11, 2025

Packing analysis report

Analysis for packing results located at cellpack/tests/outputs/test_spheres/spheresSST

Packing image

Distance analysis

Uh oh!

rugeli commented Dec 11, 2025

Uh oh!

ascibisz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants