ControlNet Annotators to ONNX

[jdp8]

Is your feature request related to a problem? Please describe.
At the moment I am able to use Canny Edge Detection, Pose Estimation and Semantic Segmentation (all in JavaScript) in diffusers.js for the annotation (pre-processing) of images that are used as input for various ControlNets. These are the only ones that I've managed to implement so far but there are more Annotator models such as Scribble, HED, M-LSD, etc that I have not found a JavaScript implementation for. It would be useful to have these models in ONNX format so they can be called the same way as an UNET, VAE or other model.

Describe the solution you'd like.
A script that can convert the Annotator models to ONNX format or the models directly in ONNX format so they can be used in diffusers.js. I believe the original implementation is controlnet_aux by @patrickvonplaten.

Describe alternatives you've considered.
I've tried using JavaScript to annotate images and so far I have Canny Edge Detection, Pose Estimation and Semantic Segmentation thanks to OpenCV.js.

Additional context.
N/A.

[sayakpaul]

It's outside the scope of diffusers. I will let @xenova chime in here.

[xenova]

Although this might be of interest to the Optimum team (regarding conversions to ONNX; cc @fxmarty), I see that you've already opened up a new request here, and I believe they will be able to provide you with the assistance you need. 😇

I will close the issue for now (since this is unplanned and out of scope), but feel free to respond with questions (or tag us in the other thread).