Issue with Running Training Command for Hover Task in SimpleFlight

[atahirkarasahin]

Hi,
Firstly, thank you for your contribution. I want to develop a policy for the hovering and aim to evaluate in real-world experiment with Crazyflie 2.1. I installed your SimpleFlight according to the instructions, but I ran below command:

python train.py headless=true wandb.mode=disabled total_frames=50000 task=Hover

and received error is as follow:

After I ran tried different command:

python train.py headless=true wandb.mode=disabled total_frames=50000 task=Hover task.action_transform=rate algo=ppo

Still I received error is as follow:

I would be grateful if you could help me with this. I look forward to your response. Thanks.

[jiayu-ch15]

Thank you for your interest in our project !

The issue you are encountering may be due to a version mismatch between the third-party libraries.

Please ensure that the versions of tensordict and torchrl match those used in our implementation. I have updated the README.md file for your reference, which includes the required package versions. Additionally, I have attached a screenshot of the system running normally and a list of the pip packages used in our environment for further clarification.

Also we recently fixed some bugs in the hover.py script, please re-pull our code!

I hope my answer solves your problem.

[atahirkarasahin]

Thank you for your response. Where can find this torchrl 0.1.1+e39e701 version? Could you please share with me?

[jiayu-ch15]

You can follow the instructions in README.md. We provide versions of tensordict and torchrl available for running.

# at SimpleFlight/
git submodule update --init --recursive

For tensordict 0.1.2+5e6205c,

cd third_party/tensordict
git checkout 5e6205c
pip install -e .

For torchrl 0.1.1+e39e701

cd third_party/torchrl
git checkout e39e701
pip install -e .

[atahirkarasahin]

Thank you for your response. I installed these third part library. I ran successful training code with below command:

python train.py headless=true wandb.mode=disabled total_frames=50_000 task=Track

But I try below command for the hovering and received below error:

python train.py headless=true wandb.mode=disabled total_frames=50_000 task=Hover

The error messages are as follow:
[1]

[2]

I think the hover.py code still has a few bugs.

I do not get any error when I use algo:mappo, but I would like to point out that I also get the following error when I issue a command as algo:ppo.

python train.py headless=true wandb.mode=disabled total_frames=50_000 algo=ppo

[3]

I wonder which using the PPO policy is on your paper?

[jiayu-ch15]

Maybe you need to pull our branch again.

hover.py did have a bug in the initial release.

The problem is that we do not use asymmetric actor and critical inputs in the initial version of hover.py.

But the default setting of "critic_input" is "state" in mappo.yaml, which is consistent with the setting in our paper. The "state" adds 4 dimensions to the time vector compared to "obs", so the dimensions don't line up.

The problem is now fixed!

[atahirkarasahin]

I pulled your repo again and according to the initial tests, the problem is fixed. I completed a training related to the hovering task. I wanted to reply after finished training. I aim to evaluate this policy in the real-world experiment in a few weeks. I will also share with you the results of the real-world experiment. I really appreciate to share all the details as a transparent and thank you for your contribution to the same field researchers to create a baseline to create a new progress.