Add Task Sorting and Corresponding Example #371
Conversation
isaaclab_arena/tasks/terminations.py
Outdated
| """ | ||
| condition_met = torch.ones((env.num_envs), device=env.device) | ||
| for object_cfg, contact_sensor_cfg in zip(object_cfg_list, contact_sensor_cfg_list): | ||
| object: RigidObject = env.scene[object_cfg.name] |
There was a problem hiding this comment.
Could we reuse object_on_destination() in here to avoid duplicating its code?
There was a problem hiding this comment.
Updated to reuse ready-made functions.
| name = "red_cube" | ||
| tags = ["object"] | ||
|
|
||
| # usd_path =f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/red_block.usd" # not support, rigid body attribute need to be bind to root xform. |
There was a problem hiding this comment.
This is left here because there is a known bug that rigid body attributes can only bind to the root layer. So I adjusted the original assets from ISAAC_NUCLEUS_DIR and uploaded adjusted one to ISAAC_NUCLEUS_STAGING_DIR. One the bug is solved, we can use original assets.
Do you think if we should remove this comments?
| name = "green_cube" | ||
| tags = ["object"] | ||
|
|
||
| # usd_path = f"{ISAAC_NUCLEUS_DIR}/Props/Blocks/green_block.usd" # not support, rigid body attribute need to be bind to root xform. |
| from isaaclab_arena.utils.cameras import get_viewer_cfg_look_at_object | ||
|
|
||
|
|
||
| class SortMultiObjectTask(TaskBase): |
There was a problem hiding this comment.
I feel like we could use a Sequential task composed of multiple atomic PickAndPlace tasks for that behaviour. @peterd-NV what do you think?
We would have an example how to compose a subtask in /isaac_arena/isaaclab_arena_environments/franka_put_and_close_door_environment.py
There was a problem hiding this comment.
Might wanna note that if the ordering of how the objects get sorted doesn't matter, then the sequential task might not be the best fit since it assumes the subtasks to be completed in a specific order. For this task where the success depends on object_on_destination I think it would still work since any out of order sorted object will eventually have it's success triggered once all objects are placed, but this does go against the idea of a sequential task and mean the intermediate subtask success states won't have meaning.
There was a problem hiding this comment.
Yes, the order in which we place objects into containers doesn’t matter. So maybe we can use a more intuitive example — for instance, pulling out a drawer, putting something inside, and then closing it.
| }, | ||
| ) | ||
| object_dropped = TerminationTermCfg( | ||
| func=root_height_below_minimum_multi_objects, |
There was a problem hiding this comment.
If we compose this sorting_task as a sequential task we could reuse the single mdp_isaac_lab.root_height_below_minimum of each subtask (PickAndPlaceTask) individually
There was a problem hiding this comment.
As we are still discussing whether to use sequential tasks, I’ve temporarily updated the implementation to reuse the existing mdp_isaac_lab.root_height_below_minimum function.
Nyquist0
force-pushed
the
lance/feature/sorting
branch
from
8511ba1 to
4625e63
Compare
Summary
Object sorting is a classic task in robotic manipulation, requiring the robot to pick multiple objects and place them at their corresponding destinations. This PR introduces a
SortMultiObjectTaskas a reusable task component and provides a concrete example environment (TableTopSortCubesEnvironment) that demonstrates how to use it.Detailed description
What was the reason for the change?
The arena framework needed a fundamental task type for multi-object sorting that can be used as a building block for more complex manipulation tasks. Many robotic manipulation scenarios require moving multiple objects to their respective target positions (e.g., sorting by color), so having a standardized task definition helps maintain consistency and reusability.
What has been changed?
New Task:
SortMultiObjectTaskinisaaclab_arena/tasks/sorting_task.py— a configurable multi-object sorting task that supports:New Termination Logic:
objects_on_destinationsfunction inisaaclab_arena/tasks/terminations.pythat evaluates whether ALL objects in the list have reached their respective destinationsroot_height_below_minimum_multi_objectsfunction for detecting failure conditions when any object falls below minimum heightNew Assets:
RedCube: A red cube blockGreenCube: A green cube blockRedContainer: A red containerGreenContainer: A green containerNew Example Environment:
TableTopSortCubesEnvironmentinisaaclab_arena_environments/sorting_environment.py— a complete example showing how to configure and use the task with a Franka robot sorting colored cubes into their matching colored containersTests:
isaaclab_arena/tests/test_sorting_task.py:What is the impact of this change?
SortMultiObjectTaskcan be reused for various object sorting and classification scenarios