Add semantic segmentation tutorial#136
Add semantic segmentation tutorial#136pepisg wants to merge 2 commits intoros-navigation:rollingfrom
Conversation
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
pepisg
force-pushed
the
semantic-segmentation
branch
from
6def6a3 to
07bba12
Compare
SteveMacenski
left a comment
SteveMacenski
left a comment
There was a problem hiding this comment.
Really just small stuff - thanks for this :-)
| <name>nav2_semantic_segmentation_demo</name> | ||
| <version>0.0.0</version> | ||
| <description>Metapackage for semantic segmentation demo packages</description> | ||
| <maintainer email="user@example.com">User</maintainer> |
nav2_semantic_segmentation_demo/nav2_semantic_segmentation_demo/CMakeLists.txt
Show resolved
Hide resolved
| <name>semantic_segmentation_sim</name> | ||
| <version>0.0.0</version> | ||
| <description>Launch files and Gazebo worlds for semantic segmentation layer testing</description> | ||
| <maintainer email="user@example.com">User</maintainer> |
|
|
||
| ### Training Your Own Model | ||
|
|
||
| 1. Capture training images from Gazebo with varying lighting and environmental conditions |
There was a problem hiding this comment.
Does it need to be GZ or also real data?
There was a problem hiding this comment.
Can you call out any changes from the default configs? It'll make it easier to update in the future if we know what changed
| # Each entry maps a class name to its text prompt for GroundingDINO | ||
| classes: | ||
| - name: sidewalk | ||
| color: [255, 0, 0] # BGR format: Blue |
There was a problem hiding this comment.
Is this for parsing from the detection or just for visualization? This would be good to mention
|
|
||
|
|
||
| class SegmentationNode(Node): | ||
| """ROS2 node that performs semantic segmentation using ONNX Runtime.""" |
There was a problem hiding this comment.
Question: is this dependent on platform / TRT versions / anything or a general purpose format? I just wanted to make sure its generalized (ish)
| providers = ['CUDAExecutionProvider', 'CPUExecutionProvider'] | ||
| self.session = ort.InferenceSession(str(model_path), providers=providers) | ||
|
|
||
| # Get model device | ||
| provider = self.session.get_providers()[0] | ||
| self.get_logger().info(f'Using provider: {provider}') |
There was a problem hiding this comment.
Not used from the node's configuration YAML? I thought I saw a cpu/gpu param
|
Question: I recently opened github.com/ros-navigation/navigation2.ai. Would parts of this fall into that? The costmap layer I know we've talked about (maybe submodule). But is the node sufficiently generic? It seems so. I might have a few more scrutiny comments on it for efficiency if adding to github.com/ros-navigation/navigation2.ai, but if its general (i.e. any ONNX file works) then I think that would be a good not-just-tutorial contribution :-) |
2 participants
goes with ros-navigation/docs.nav2.org#876