Migrates Pink IK to Newton branch #4514
Open
+6,976
−61
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…s with more accurate actuator parameters. Passing Pink IK test on G1.
…weight back to root link.
michaellin6
requested review from
Mayankm96,
jtigue-bdai,
kellyguo11 and
pascal-roth
as code owners
Contributor
Greptile OverviewGreptile SummaryThis PR successfully integrates the Pink IK controller into IsaacLab's Newton branch, enabling inverse kinematics control for humanoid robots (G1 and GR1T2). The implementation includes comprehensive test coverage and introduces two important infrastructure features. Major Changes:
Known Limitations (acknowledged in PR):
Technical Quality:
Confidence Score: 4/5
Important Files Changed
Sequence DiagramsequenceDiagram
participant User
participant Env as ManagerBasedEnv
participant Action as PinkInverseKinematicsAction
participant IK as PinkIKController
participant Pink as Pink Solver
participant Art as Articulation
participant Sim as Newton Simulator
User->>Env: step(action)
Env->>Action: process_actions(action)
Action->>Action: Parse action into task targets
loop For each environment
Action->>IK: Set task targets (poses/joints)
Action->>Art: Get current joint positions
Art-->>Action: Joint positions
Action->>IK: compute(curr_joint_pos, dt)
IK->>IK: Update Pink configuration
IK->>Pink: solve_ik(tasks, dt)
Pink-->>IK: Joint velocities
IK->>IK: Convert velocities to positions
IK-->>Action: Target joint positions
end
Action->>Art: set_joint_position_target()
Env->>Art: write_data_to_sim()
Art->>Art: Apply gravity compensation
Art->>Art: Apply actuator models
Art->>Sim: Write joint commands
Env->>Sim: step()
Sim-->>Sim: Physics simulation
Env->>Art: update(dt)
Art->>Sim: Read state data
Sim-->>Art: Updated positions/velocities
Art-->>Env: Updated state
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+54
to
+60
| def set_target(self, transform_target_to_base: pin.SE3) -> None: | ||
| """Set task target pose in the world frame. | ||
|
|
||
| Args: | ||
| transform_target_to_world: Transform from the task target frame to | ||
| the world frame. | ||
| """ |
Contributor
There was a problem hiding this comment.
Docstring parameter name doesn't match function parameter
Suggested change
| def set_target(self, transform_target_to_base: pin.SE3) -> None: | |
| """Set task target pose in the world frame. | |
| Args: | |
| transform_target_to_world: Transform from the task target frame to | |
| the world frame. | |
| """ | |
| def set_target(self, transform_target_to_base: pin.SE3) -> None: | |
| """Set task target pose in the base frame. | |
| Args: | |
| transform_target_to_base: Transform from the task target frame to | |
| the base link frame. | |
| """ |
Comment on lines
+2096
to
+2134
| def _apply_gravity_compensation(self): | ||
| """Apply gravity compensation to the articulation. | ||
|
|
||
| This method uses a two-kernel approach to avoid race conditions: | ||
| 1. First kernel accumulates compensation forces per-body and atomically accumulates | ||
| the negative compensation to the root link accumulator. | ||
| 2. Second kernel applies the accumulated force to the root link. | ||
| """ | ||
| if self._has_gravity_compensation: | ||
| gravity = NewtonManager._gravity_vector # Already a tuple (x, y, z) | ||
|
|
||
| # Zero the accumulator before use | ||
| self._root_comp_force.zero_() | ||
|
|
||
| # First kernel: apply per-body compensation and atomically accumulate root link force | ||
| wp.launch( | ||
| apply_gravity_compensation_force, | ||
| dim=(self.num_instances, self.num_bodies), | ||
| inputs=[ | ||
| self._data.body_mass, | ||
| self._gravity_compensation_factor, | ||
| wp.vec3f(gravity[0], gravity[1], gravity[2]), | ||
| self._data._sim_bind_body_external_wrench, | ||
| self._root_link_index, | ||
| self._root_comp_force, | ||
| ], | ||
| device=self.device, | ||
| ) | ||
|
|
||
| # Second kernel: apply the accumulated compensation force to the root link | ||
| wp.launch( | ||
| apply_accumulated_root_force, | ||
| dim=self.num_instances, | ||
| inputs=[ | ||
| self._root_comp_force, | ||
| self._data._sim_bind_body_external_wrench, | ||
| self._root_link_index, | ||
| ], | ||
| device=self.device, |
Contributor
There was a problem hiding this comment.
The PR description acknowledges that the current gravity compensation approach is not ideal. It applies anti-gravity forces to compensated bodies and adds their weight to the root link. The proper approach would be to apply joint torques. Consider requesting this feature from the Newton team as noted in the PR description.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adding Pink IK controller and a subset of Mimic environments. Pink IK unit tests and controller tests on G1 env are passing.
Features added:
Pending issues:
RuntimeError: Unable to copy file: 'https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1/Isaac/Props/PackingTable/materials/textures/t_corrugatedboxes_b01_trim_Albedo.png'. Is the Nucleus Server running?RuntimeError: Unable to copy file: 'https://omniverse-content-production.s3-us-west-2.amazonaws.com/Assets/Isaac/5.1/Isaac/Robots/FourierIntelligence/GR-1/GR1T2_fourier_hand_6dof/textures/texture/GR1T2FourierIntelligence_GR1T2_roughness.<UDIM>.png'. Is the Nucleus Server running?Type of change
Checklist
pre-commitchecks with./isaaclab.sh --formatconfig/extension.tomlfileCONTRIBUTORS.mdor my name already exists there