Add semantic segmentation tutorial#876
Open
pepisg wants to merge 4 commits intoros-navigation:masterfrom
Open
Conversation
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
pepisg
force-pushed
the
semantic-segmentation
branch
from
feac7fb to
bdcfb78
Compare
Signed-off-by: pepisg <pedro.gonzalez@eia.edu.co>
Member
SteveMacenski
left a comment
SteveMacenski
left a comment
There was a problem hiding this comment.
Really love this, thank you! ❤️
| .. _navigation2_with_semantic_segmentation: | ||
|
|
||
| Navigating with Semantic Segmentation | ||
| ************************************** |
Member
There was a problem hiding this comment.
Suggested change
| ************************************** | |
| ************************************* |
| ======== | ||
|
|
||
| This tutorial demonstrates how to use semantic segmentation in costmaps with nav2, using a custom `semantic_segmentation_layer plugin <https://github.com/ros-navigation/navigation2_tutorials/tree/master/nav2_semantic_segmentation_layer>`_ and a pre-trained segmentation model that works on gazebo's Baylands world. It was written by Pedro Gonzalez at `robot.com <https://robot.com/>`_. | ||
|
|
Member
There was a problem hiding this comment.
Immediately right here put on of your impressive gifs :-)
| :alt: Example of semantic segmentation showing original image and segmented mask | ||
|
|
||
| Once trained, the output of a semantic segmentation model is typically an image with the same size as the input, where each pixel holds the probability of that pixel belonging to each class. | ||
| For instance, the model provided in this tutorial has 3 classes: sidewalk, grass, and background; hence its raw output is a 3-channel image, where each channel corresponds to the probability of the pixel belonging to that class. |
Member
There was a problem hiding this comment.
What if this was a general purpose segementation algorithm with many classes?
Might it be better to have the node have the probability handling so that the output image was 1 channel where it corresponded to the thresholded value (or unknown)?
| :alt: Gazebo Baylands world | ||
|
|
||
| 1- Setup Semantic Segmentation Inference Node | ||
| ----------------------------------------------- |
Member
There was a problem hiding this comment.
Suggested change
| ----------------------------------------------- | |
| --------------------------------------------- |
| segmentation_topic: "/segmentation/mask" | ||
| confidence_topic: "/segmentation/confidence" | ||
| labels_topic: "/segmentation/label_info" | ||
| pointcloud_topic: "/rgbd_camera/depth/points" |
Member
There was a problem hiding this comment.
Question for me + the tutorial to clarify: Does this pointcloud topic need to be from the same frame as the image is from, or can it be from another source?
| The tutorial provides a pre-configured `nav2_params.yaml <https://github.com/ros-navigation/navigation2_tutorials/blob/master/nav2_semantic_segmentation_demo/semantic_segmentation_sim/config/nav2_params.yaml>`_ file in the semantic_segmentation_sim package. You can use it to configure the Nav2 costmaps for your own application. | ||
|
|
||
| 3- Run everything together | ||
| ---------------------------- |
Member
There was a problem hiding this comment.
Suggested change
| ---------------------------- | |
| -------------------------- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goes together with ros-navigation/navigation2_tutorials#136