Issue While Running the Web App

[chkmook]

Hello. Thank you for sharing your research.

I encountered a few issues while running the Web App and would like to ask for your guidance:

• Issue with providing a depth map in “.exr” format
  I wasn’t able to use a depth map in the “.exr” format as input for the [diffhandles] app.
  If I want to run the [diffhandles] app independently, what format should the depth map be in?

• Error during Step 3 in [diffhandles_pipeline] app
  When executing “Perform Edit” in Step 3, the following error occurs:
  transform_foreground_job.outputs() returns an empty list ([]).
  

  DiffusionHandles/webapp/webapps/diffhandles_pipeline_webapp.py

  Line 610 in ae6230a

  transform_foreground_job = GradioJob(job=self.diffhandles_client.submit(

  
  This leads to an IndexError at the following line:
  

  DiffusionHandles/webapp/webapps/diffhandles_pipeline_webapp.py

  Line 604 in ae6230a

  edited_image_path, debug_images_path = jobs[0].outputs()[0]

  
  I have confirmed that the inputs are being passed correctly.
  Could you help identify what might be causing this issue?

Thank you for your assistance!

[paulguerrero]

Sorry for the late reply (I did not receive notifications before).
If you want to create a custom pipeline, I would honestly recommend not using the webapp and instead directly calling the code in
diffhandles/diffusion_handles.py, which has much cleaner interfaces.

If you still want to use the webapp though, the gradio interface to pass exr images is through the custom gradio component HDRImage (since gradio does not support HDR images natively). If you want to encode the depth map into a regular image, you would need to replace the HDRImage component with the regular Image component when the gradio app is constructed.
The current code assumes depth is given as floating-point metric depth, so you would need to decode the depth map before using it, and encode it before returning it (as needed), in each of the four functions set_input_image, set_foreground, transform_foreground, and run_diffhandles.

For the error you get, it seems like transform_foreground in diffhandles_webapp.py does not return any values. Make sure you have the diffhandles_webapp.py running and check what output transform_foreground returns and if you get any error messages for that app on the terminal. Also make sure you use the latest gradio version, version 4.44.1 works for me, for example.

Hope that helps!