[Question] Support for running trainer with --num-envs > 1 on headless Linux builds

[danorel]

Hi!

I would like to ask whether it is possible to run the trainer on the default environment with --num-envs > 1.

I have forked the repository and implemented a unified setup for MacOS and Linux builds. I am trying to train the mouse environment on a server (GCP) using GPU acceleration. You can see the relevant implementation in my repository: train.py.

Here is the command I use to launch training:

cmd = [
    "mlagents-learn",
    config_path,
    "--env", str(Path(env_path) / "LinuxHeadless.x86_64"),
    "--run-id", current,
    "--num-envs", "1",
    "--time-scale", "40",
    "--force",
    "--debug",
    "--env-args", "--screen-width=155", "--screen-height=86",
]

Currently, I can run training successfully with --num-envs=1. However, when I try to increase --num-envs > 1, the environment crashes with the following error:

Loading in SingleInstance mode
Loading in SingleInstance mode
Player is already running 
(Filename:  Line: 457)

2025-08-21 17:45:55 INFO [environment.py:113] Connected to Unity environment with package version 2.0.1 and communication version 1.5.0
2025-08-21 17:45:57 INFO [environment.py:297] Connected new brain: My Behavior?team=0
2025-08-21 17:45:58 DEBUG [subprocess_env_manager.py:223] UnityEnvironment worker 1: environment stopping.
2025-08-21 17:45:58 DEBUG [subprocess_env_manager.py:237] UnityEnvironment worker 1 closing.
2025-08-21 17:45:58 DEBUG [subprocess_env_manager.py:240] UnityEnvironment worker 1 done.
2025-08-21 17:45:58 INFO [trainer_controller.py:188] Learning was interrupted. Please wait while the graph is generated.
2025-08-21 17:45:58 DEBUG [trainer_controller.py:81] Saved Model
2025-08-21 17:45:58 DEBUG [subprocess_env_manager.py:482] SubprocessEnvManager closing.
2025-08-21 17:45:58 DEBUG [subprocess_env_manager.py:110] UnityEnvWorker 1 got exception trying to close.
2025-08-21 17:45:58 DEBUG [subprocess_env_manager.py:237] UnityEnvironment worker 0 closing.
2025-08-21 17:45:59 DEBUG [environment.py:444] Environment shut down with return code 0.
2025-08-21 17:45:59 DEBUG [subprocess_env_manager.py:240] UnityEnvironment worker 0 done.
mlagents_envs.exception.UnityEnvironmentException: Environment shut down with return code 0

It seems related to the SingleInstance mode of the LinuxHeadless build. I understand that the default headless build enforces a single instance, but I want to train a single model using multiple environment instances on GPU for faster parallel training.

My questions are:

• Is it possible to run multiple environments with a single LinuxHeadless build, or do we need to rebuild the environment differently?
• If rebuilding is required, what is the recommended way to generate a LinuxHeadless build that supports multiple concurrent instances?
• Are there any workarounds to train on multiple environments on GPU without rebuilding from the Unity Editor?

Any guidance or suggestions would be greatly appreciated!

Thank you in advance!