moveMouseToPosition() doesn't move mouse to correct position

[zkghost]

I've got a setup with 3 monitors where my primary monitor is in the middle. I have a loop that's something like:

def get_mouse_position():
    pos = autohotpy.getMousePosition()
    return {'x': pos[0], 'y': pos[1]}

while True:
    time.sleep(0.01)
    mouse = get_mouse_position()
    message = "\rMouse is at x: {}, y: {}".format(mouse['x'], mouse['y']) + ' ' * 10
    print(message, end='', flush=True)
    sys.stdout.flush()

This shows that on my primary monitor, the mouse coordinates range from:
x:0 -> x:1919
y:0 -> y:1199
Which makes sense, considering the monitor is 1920 x 1200.

If I go to my left-most monitor, the x values returned are negative, from -1 to -3840 which makes sense because it is a 4k monitor. The y values only go to 864 though, which is odd.. Not sure if that is due to UI scaling?

Anyways, given all this, if I do:

autohotpy.moveMouseToPosition(500, 500)

I would expect it to land in the same spot where my loop above would show 500,500.

What actually happens is the mouse ends up on my leftmost monitor at -3040, 360.

Do you have any idea why this is happening?

This is on a Windows 10 environment.

[zkghost]

Digging a bit, I think I might see the issue:

#InceptionWrapper.py
class InterceptionMouseFlag(object):
    """
    If INTERCEPTION_MOUSE_MOVE_ABSOLUTE value is specified, dx and dy contain
    normalized absolute coordinates between 0 and 65,535.
    The event procedure maps these coordinates onto the display surface.
    Coordinate (0,0) maps onto the upper-left corner of the display surface;
    coordinate (65535,65535) maps onto the lower-right corner.
    In a multimonitor system, the coordinates map to the primary monitor. 
    http://msdn.microsoft.com/en-us/library/windows/desktop/ms646273%28v=vs.85%29.aspx
    """

But even then, I would expect moveMouseToPosition(65535/2, 65535/2) to put the mouse in the relative middle of all my monitors. This just puts it in the top left corner of my left most monitor. So I am still confused!

[zkghost]

Ah, ok this is interesting:

So I have 1 4k monitor and 2 1920x1200 monitors, so half of the total pixels in the x direction should be taken up by the 4k monitor, and the remaining two monitors make up the other half.

I notice if I do moveMouseToPosition(960, 0) this moves the mouse to the top left of the middle monitor (my main display) (actually, I need like 960.25~).

>>> 960 * (65535.0/float(1920))
32767.5
>>> 32767.5*2
65535.0

This is just the same scaling done in moveMouseToPosition(). But that does seem to give me the halfway point along the X axis.

I am still confused though, is this the intended behavior? Having to renormalize all this myself seems annoying. And it's not clear how I'm supposed to handle this in a generic way so that I can reliably target pixels on any monitor, or a monitor with a specific window or application open.

[zkghost]

It does seem to actually move to the correct coordinates when I only have one monitor. That's a relief at least; is this a limitation of the library then that it doesn't support accurate mouse movement with more than one monitor?

[dc740]

Hi, This behavior seems to be related exclusively to the Interception driver.
http://www.oblita.com/interception.html

Your best bet is to search for documentation in the original C library, or you may want to get in touch with the author, but it looks like you already discovered you need to normalize the values to a from your monitor coordinates all the time.

[zkghost]

Ah, I see! I will follow that thread then, thanks! Also thanks for quick reply :)

[zkghost]

After some digging, I found an additional pitfall I thought I'd note for anyone else who hits this issue.

After following up with oblitum, he confirmed the normalization is required. But I discovered that normalization is different depending on the arrangement of the monitors.

For example, with my setup like

                              main
+----------------------+ +-------------+ +-------------+
|    3840 x 2160       | |             | |             |
|                      | |  1920x1200  | |  1920x1200  |
|                      | |             | |             |
|                      | +-------------+ +-------------+
|                      |
|                      |
+----------------------+

When moving along the x axis I can use the math mentioned in the ticket above to accurately move the mouse. When moving along the y axis though, I've discovered it actually requires your normalization constant to be 65535/(<max_monitor_height> - 1). So in my case, I need to multiply my final output by 1200/2159 because I have a 4k monitor. This undoes the 65535.0/1200 that AutoHotPy.moveToMousePosition() applies and replaces it with 65535.0/2159.

Anyways, if your monitors are stacked vertically, this problem flips and you have to do this for the X axis I believe. And if you have one monitor to the left and one above, I bet you have to do both. I tested it a little bit, forget the details.

Basically, if you really want to be sure you are moving the mouse to the right spot on a multi-monitor system, you're probably best off writing some code to calibrate and make sure the mouse is going where you expect it.

[dc740]

Ah I see now. I remembered this was related to the driver, but I didn't remember that I wrote a function to hide this from users. In that case, the function needs to be fixed to support multiple monitors.
Looks like you are currently working on this already. Feel free to modify AutoHotPy and send a PR, I'll walk you through the final steps if needed and we can discuss different alternatives. Ideally we don't want to break current code for single monitors, but it'd be great if the function would also map a huge screen so we can ignore it's made of different monitors, like some window managers do with dead space on Xinerama ( check the bottom of the page here: https://en.wikipedia.org/wiki/Xinerama ).

[zkghost]

Yeah, I will try to put out a PR. I will be a bit unavailable over the next week, but I'll reach ping out again when I have the PR together

[Elkis]

I found, if change in AutoHotpy from:
def moveMouseToPosition(self, x, y):
width_constant = 65535.0/float(self.user32.GetSystemMetrics(0))
height_constant = 65535.0/float(self.user32.GetSystemMetrics (1))
to this:
def moveMouseToPosition(self, x, y):
width_constant = 65535.0/float(self.user32.GetSystemMetrics(78))
height_constant = 65535.0/float(self.user32.GetSystemMetrics (79))
it actually start to work, GetSystemMetrics start to show multimonitor resolution, and then you try move to center of object on a screen, it stop shift on axis