JNA / WinAPI。模拟鼠标点击会移动鼠标光标，但不会 return 它回到起始位置
JNA / WinAPI. Simulating mouse click moves mouse cursor and doesn't return it back to the start position
在我的 Java 代码中，我想在按下 Q 键时模拟鼠标在特定屏幕坐标 (x, y) 上的不可见点击。人眼应该不可见:鼠标移动->鼠标点击->鼠标后退
为此，我正在使用 JNA 5.6.0 和 jna-platform 5.6.0，它们使用本机 WinAPI 函数。我在 LowLevelKeyboardProc() {callback()} 中实现了低级键盘挂钩，用于拦截击键。对于鼠标点击模拟，我使用 SendInput().
我预期鼠标点击应该在某些全局屏幕坐标 (x, y) 上完成。在我的代码示例中，坐标是 40, 40.
但是我得到的结果不是预期的：
- 鼠标移动和鼠标点击，但没有返回到点击前光标所在的起始位置。
- 鼠标移动并点击当前鼠标位置的坐标 40、40，而不是全局屏幕的坐标。
演示此行为的最佳方法，运行 下面提供的代码，打开画图，select 画笔工具并按 'Q' 按钮。
这是它的样子：
如您所见，第一次按下 Q 按钮后光标没有移回起始位置。下一次按下 Q 按钮时，单击点与当前鼠标位置之间的距离会增加，并且此距离在第一次按下 Q 按钮后会有所不同。
import com.sun.jna.Pointer;
import com.sun.jna.platform.win32.*;
import static com.sun.jna.platform.win32.WinUser.*;
public class TestExample {
public static final int MOUSEEVENTF_MOVE = 1;
public static final int MOUSEEVENTF_LEFTDOWN = 2;
public static final int MOUSEEVENTF_LEFTUP = 4;
private static WinUser.HHOOK hHook;
static final User32 user32Library = User32.INSTANCE;
static WinDef.HMODULE hMod = Kernel32.INSTANCE.GetModuleHandle(null);
public static void main(String[] args) {
keyReMapOn();
}
public static void keyReMapOn() {
WinUser.LowLevelKeyboardProc keyboardHook = new WinUser.LowLevelKeyboardProc() {
@Override
public WinDef.LRESULT callback(int nCode, WinDef.WPARAM wParam, WinUser.KBDLLHOOKSTRUCT kbDllHookStruct) {
if (nCode >= 0) {
if (wParam.intValue() == WM_KEYDOWN) {
if (kbDllHookStruct.vkCode == 81) { // Q button
clickByCord(40, 40);
return new WinDef.LRESULT(1);
}
}
}
Pointer ptr = kbDllHookStruct.getPointer();
long peer = Pointer.nativeValue(ptr);
return user32Library.CallNextHookEx(hHook, nCode, wParam, new WinDef.LPARAM(peer));
}
};
hHook = user32Library.SetWindowsHookEx(WH_KEYBOARD_LL, keyboardHook, hMod, 0);
int result;
WinUser.MSG msg = new WinUser.MSG();
while ((result = user32Library.GetMessage(msg, null, 0, 0)) != 0) {
if (result == -1) {
break;
} else {
user32Library.TranslateMessage(msg);
user32Library.DispatchMessage(msg);
}
}
}
public static void clickByCord(int x, int y) {
mouseMove(x, y);
mouseLeftClick(x, y);
}
static void mouseMove(int x, int y) {
mouseAction(x, y, MOUSEEVENTF_MOVE);
}
public static void mouseLeftClick(int x, int y) {
mouseAction(x, y, MOUSEEVENTF_LEFTDOWN);
mouseAction(x, y, MOUSEEVENTF_LEFTUP);
}
public static void mouseAction(int x, int y, int flags) {
INPUT input = new INPUT();
input.type = new DWORD(INPUT.INPUT_MOUSE);
input.input.setType("mi");
if (x != -1) {
input.input.mi.dx = new LONG(x);
}
if (y != -1) {
input.input.mi.dy = new LONG(y);
}
input.input.mi.time = new DWORD(0);
input.input.mi.dwExtraInfo = new BaseTSD.ULONG_PTR(0);
input.input.mi.dwFlags = new DWORD(flags);
User32.INSTANCE.SendInput(new DWORD(1), new INPUT[]{input}, input.size());
}
}
我在某处的代码中出错了。
谁能告诉我如何在全局屏幕坐标上实现隐形鼠标左键点击？
下一次按下 Q 键时点击点和当前鼠标位置之间的距离会变大，这个距离在第一次按下 Q 键后会有所不同。
documentation已经解释过了：
Relative mouse motion is subject to the effects of the mouse speed and
the two-mouse threshold values. A user sets these three values with
the Pointer Speed slider of the Control Panel's Mouse Properties
sheet. You can obtain and set these values using the
SystemParametersInfo function.
The system applies two tests to the specified relative mouse movement.
If the specified distance along either the x or y axis is greater than
the first mouse threshold value, and the mouse speed is not zero, the
system doubles the distance. If the specified distance along either
the x or y axis is greater than the second mouse threshold value, and
the mouse speed is equal to two, the system doubles the distance that
resulted from applying the first threshold test. It is thus possible
for the system to multiply specified relative mouse movement along the
x or y axis by up to four times.
您可以使用绝对坐标来完成项目。
If MOUSEEVENTF_ABSOLUTE value is specified, dx and dy contain
normalized absolute coordinates between 0 and 65,535. The event
procedure maps these coordinates onto the display surface. Coordinate
(0,0) maps onto the upper-left corner of the display surface;
coordinate (65535,65535) maps onto the lower-right corner. In a
multimonitor system, the coordinates map to the primary monitor.
部分代码：
UINT mouseAction(int x, int y, int flags)
{
INPUT input;
POINT pos;
GetCursorPos(&pos);
input.type = INPUT_MOUSE;
input.mi.dwFlags = flags;
input.mi.time = NULL;
input.mi.mouseData = NULL;
input.mi.dx = (pos.x + x)*(65536.0f / GetSystemMetrics(SM_CXSCREEN));
input.mi.dy = (pos.y + y)*(65536.0f / GetSystemMetrics(SM_CYSCREEN));
input.mi.dwExtraInfo = GetMessageExtraInfo();
return SendInput(1, &input, sizeof(INPUT));
}
void mouseMove(int x, int y)
{
mouseAction(x, y, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
代码是C++，我对JAVA代码不熟悉，不过我觉得你改成Java.
也很容易
补充：可以用GetAsyncKeyState检测Q键是否被按下。依赖的hook过多会影响运行性能
如您所见，第一次按下 Q 键后光标没有移回起始位置。
可以保存起始坐标。然后在需要的时候return到起始位置
Gif 演示：
更新：
为了进一步帮助您解决问题，我将演示一些操作供您参考。
我们用GetCursorPos得到ABC的坐标
A(267,337) B(508,334) C(714,329)
然后将它们的坐标传递给输入结构体，参考如下代码：
#include <Windows.h>
#include <iostream>
UINT mouseAction(int x, int y, int flags)
{
INPUT input;
POINT pos;
int x1 = GetSystemMetrics(SM_CXSCREEN);
int y1 = GetSystemMetrics(SM_CYSCREEN);
input.type = INPUT_MOUSE;
input.mi.dwFlags = flags;
input.mi.time = NULL;
input.mi.mouseData = NULL;
input.mi.dx = x* (65536.0f / GetSystemMetrics(SM_CXSCREEN));
input.mi.dy = y* (65536.0f / GetSystemMetrics(SM_CYSCREEN));
input.mi.dwExtraInfo = GetMessageExtraInfo();
return SendInput(1, &input, sizeof(INPUT));
}
int main()
{
while(1)
{
if (GetAsyncKeyState(0x51) & 0x0001)
{
mouseAction(267, 337, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
if (GetAsyncKeyState(0x57) & 0x0001)
{
mouseAction(508, 334, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
if (GetAsyncKeyState(0x45) & 0x0001)
{
mouseAction(714, 329, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
// mouseAction(267-87, 337-65, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
}
return 0;
}
虽然是C++代码，但是基本逻辑和winapi是一样的。按Q键，鼠标会移动到A坐标，按W键移动到B坐标，按E键移动到C坐标。
然后我们可以根据A和D的距离计算出D的坐标，这样可以帮你解决移动两个相邻坐标的问题
另外，我们可以根据实际情况微调距离
添加以下代码，
mouseAction(267-87, 337-65, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE); // D(267-87, 337-65)
在我的 Java 代码中，我想在按下 Q 键时模拟鼠标在特定屏幕坐标 (x, y) 上的不可见点击。人眼应该不可见:鼠标移动->鼠标点击->鼠标后退
为此，我正在使用 JNA 5.6.0 和 jna-platform 5.6.0，它们使用本机 WinAPI 函数。我在 LowLevelKeyboardProc() {callback()} 中实现了低级键盘挂钩，用于拦截击键。对于鼠标点击模拟，我使用 SendInput().
我预期鼠标点击应该在某些全局屏幕坐标 (x, y) 上完成。在我的代码示例中，坐标是 40, 40.
但是我得到的结果不是预期的：
- 鼠标移动和鼠标点击，但没有返回到点击前光标所在的起始位置。
- 鼠标移动并点击当前鼠标位置的坐标 40、40，而不是全局屏幕的坐标。
演示此行为的最佳方法，运行 下面提供的代码，打开画图，select 画笔工具并按 'Q' 按钮。
这是它的样子：
如您所见，第一次按下 Q 按钮后光标没有移回起始位置。下一次按下 Q 按钮时，单击点与当前鼠标位置之间的距离会增加，并且此距离在第一次按下 Q 按钮后会有所不同。
import com.sun.jna.Pointer;
import com.sun.jna.platform.win32.*;
import static com.sun.jna.platform.win32.WinUser.*;
public class TestExample {
public static final int MOUSEEVENTF_MOVE = 1;
public static final int MOUSEEVENTF_LEFTDOWN = 2;
public static final int MOUSEEVENTF_LEFTUP = 4;
private static WinUser.HHOOK hHook;
static final User32 user32Library = User32.INSTANCE;
static WinDef.HMODULE hMod = Kernel32.INSTANCE.GetModuleHandle(null);
public static void main(String[] args) {
keyReMapOn();
}
public static void keyReMapOn() {
WinUser.LowLevelKeyboardProc keyboardHook = new WinUser.LowLevelKeyboardProc() {
@Override
public WinDef.LRESULT callback(int nCode, WinDef.WPARAM wParam, WinUser.KBDLLHOOKSTRUCT kbDllHookStruct) {
if (nCode >= 0) {
if (wParam.intValue() == WM_KEYDOWN) {
if (kbDllHookStruct.vkCode == 81) { // Q button
clickByCord(40, 40);
return new WinDef.LRESULT(1);
}
}
}
Pointer ptr = kbDllHookStruct.getPointer();
long peer = Pointer.nativeValue(ptr);
return user32Library.CallNextHookEx(hHook, nCode, wParam, new WinDef.LPARAM(peer));
}
};
hHook = user32Library.SetWindowsHookEx(WH_KEYBOARD_LL, keyboardHook, hMod, 0);
int result;
WinUser.MSG msg = new WinUser.MSG();
while ((result = user32Library.GetMessage(msg, null, 0, 0)) != 0) {
if (result == -1) {
break;
} else {
user32Library.TranslateMessage(msg);
user32Library.DispatchMessage(msg);
}
}
}
public static void clickByCord(int x, int y) {
mouseMove(x, y);
mouseLeftClick(x, y);
}
static void mouseMove(int x, int y) {
mouseAction(x, y, MOUSEEVENTF_MOVE);
}
public static void mouseLeftClick(int x, int y) {
mouseAction(x, y, MOUSEEVENTF_LEFTDOWN);
mouseAction(x, y, MOUSEEVENTF_LEFTUP);
}
public static void mouseAction(int x, int y, int flags) {
INPUT input = new INPUT();
input.type = new DWORD(INPUT.INPUT_MOUSE);
input.input.setType("mi");
if (x != -1) {
input.input.mi.dx = new LONG(x);
}
if (y != -1) {
input.input.mi.dy = new LONG(y);
}
input.input.mi.time = new DWORD(0);
input.input.mi.dwExtraInfo = new BaseTSD.ULONG_PTR(0);
input.input.mi.dwFlags = new DWORD(flags);
User32.INSTANCE.SendInput(new DWORD(1), new INPUT[]{input}, input.size());
}
}
我在某处的代码中出错了。
谁能告诉我如何在全局屏幕坐标上实现隐形鼠标左键点击？
下一次按下 Q 键时点击点和当前鼠标位置之间的距离会变大，这个距离在第一次按下 Q 键后会有所不同。
documentation已经解释过了：
Relative mouse motion is subject to the effects of the mouse speed and the two-mouse threshold values. A user sets these three values with the Pointer Speed slider of the Control Panel's Mouse Properties sheet. You can obtain and set these values using the SystemParametersInfo function.
The system applies two tests to the specified relative mouse movement. If the specified distance along either the x or y axis is greater than the first mouse threshold value, and the mouse speed is not zero, the system doubles the distance. If the specified distance along either the x or y axis is greater than the second mouse threshold value, and the mouse speed is equal to two, the system doubles the distance that resulted from applying the first threshold test. It is thus possible for the system to multiply specified relative mouse movement along the x or y axis by up to four times.
您可以使用绝对坐标来完成项目。
If MOUSEEVENTF_ABSOLUTE value is specified, dx and dy contain normalized absolute coordinates between 0 and 65,535. The event procedure maps these coordinates onto the display surface. Coordinate (0,0) maps onto the upper-left corner of the display surface; coordinate (65535,65535) maps onto the lower-right corner. In a multimonitor system, the coordinates map to the primary monitor.
部分代码：
UINT mouseAction(int x, int y, int flags)
{
INPUT input;
POINT pos;
GetCursorPos(&pos);
input.type = INPUT_MOUSE;
input.mi.dwFlags = flags;
input.mi.time = NULL;
input.mi.mouseData = NULL;
input.mi.dx = (pos.x + x)*(65536.0f / GetSystemMetrics(SM_CXSCREEN));
input.mi.dy = (pos.y + y)*(65536.0f / GetSystemMetrics(SM_CYSCREEN));
input.mi.dwExtraInfo = GetMessageExtraInfo();
return SendInput(1, &input, sizeof(INPUT));
}
void mouseMove(int x, int y)
{
mouseAction(x, y, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
代码是C++，我对JAVA代码不熟悉，不过我觉得你改成Java.
也很容易补充：可以用GetAsyncKeyState检测Q键是否被按下。依赖的hook过多会影响运行性能
如您所见，第一次按下 Q 键后光标没有移回起始位置。
可以保存起始坐标。然后在需要的时候return到起始位置
Gif 演示：
更新：
为了进一步帮助您解决问题，我将演示一些操作供您参考。
我们用GetCursorPos得到ABC的坐标
A(267,337) B(508,334) C(714,329)
然后将它们的坐标传递给输入结构体，参考如下代码：
#include <Windows.h>
#include <iostream>
UINT mouseAction(int x, int y, int flags)
{
INPUT input;
POINT pos;
int x1 = GetSystemMetrics(SM_CXSCREEN);
int y1 = GetSystemMetrics(SM_CYSCREEN);
input.type = INPUT_MOUSE;
input.mi.dwFlags = flags;
input.mi.time = NULL;
input.mi.mouseData = NULL;
input.mi.dx = x* (65536.0f / GetSystemMetrics(SM_CXSCREEN));
input.mi.dy = y* (65536.0f / GetSystemMetrics(SM_CYSCREEN));
input.mi.dwExtraInfo = GetMessageExtraInfo();
return SendInput(1, &input, sizeof(INPUT));
}
int main()
{
while(1)
{
if (GetAsyncKeyState(0x51) & 0x0001)
{
mouseAction(267, 337, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
if (GetAsyncKeyState(0x57) & 0x0001)
{
mouseAction(508, 334, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
if (GetAsyncKeyState(0x45) & 0x0001)
{
mouseAction(714, 329, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
// mouseAction(267-87, 337-65, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE);
}
}
return 0;
}
虽然是C++代码，但是基本逻辑和winapi是一样的。按Q键，鼠标会移动到A坐标，按W键移动到B坐标，按E键移动到C坐标。
然后我们可以根据A和D的距离计算出D的坐标，这样可以帮你解决移动两个相邻坐标的问题
另外，我们可以根据实际情况微调距离
添加以下代码，
mouseAction(267-87, 337-65, MOUSEEVENTF_MOVE | MOUSEEVENTF_ABSOLUTE); // D(267-87, 337-65)