如何使用AccessibilityService在Android上进行拖动（基于X，Y鼠标坐标？）>

Question

我想知道如何在基于X，Y鼠标坐标的android上执行拖动？考虑两个简单的示例，Team Viewer / QuickSupport分别在远程智能手机和Windows Paint Pen上绘制“密码模式”：

我所能做的就是simulate touch（带有dispatchGesture()和AccessibilityNodeInfo.ACTION_CLICK）。

我找到了这些相关链接，但不知道它们是否有用：

• Perform swipe on screen using AccessibilityService
• Example 1
• Continued gestures

下面是我的工作代码，用于将鼠标坐标（在PictureBox控件内部）发送到远程电话并模拟触摸。

Windows窗体应用程序：

private void pictureBox1_MouseDown(object sender, MouseEventArgs e)
{
    foreach (ListViewItem item in lvConnections.SelectedItems)
    {
        // Remote screen resolution
        string[] tokens = item.SubItems[5].Text.Split('x'); // Ex: 1080x1920

        int xClick = (e.X * int.Parse(tokens[0].ToString())) / (pictureBox1.Size.Width);
        int yClick = (e.Y * int.Parse(tokens[1].ToString())) / (pictureBox1.Size.Height);

        Client client = (Client)item.Tag;

        if (e.Button == MouseButtons.Left)
            client.sock.Send(Encoding.UTF8.GetBytes("TOUCH" + xClick + "<|>" + yClick + Environment.NewLine));
    }
}
编辑：

我最后一次尝试是分别使用鼠标坐标（C＃Windows Forms Application）和自定义android例程（参考上面链接的“ swipe screen”的代码）进行“ swipe screen”：

private Point mdownPoint = new Point();

private void pictureBox1_MouseDown(object sender, MouseEventArgs e)
{
    foreach (ListViewItem item in lvConnections.SelectedItems)
    {
        // Remote screen resolution
        string[] tokens = item.SubItems[5].Text.Split('x'); // Ex: 1080x1920

        Client client = (Client)item.Tag;

        if (e.Button == MouseButtons.Left)
        {
            xClick = (e.X * int.Parse(tokens[0].ToString())) / (pictureBox1.Size.Width); 
            yClick = (e.Y * int.Parse(tokens[1].ToString())) / (pictureBox1.Size.Height);

            // Saving start position:

            mdownPoint.X = xClick; 
            mdownPoint.Y = yClick; 

            client.sock.Send(Encoding.UTF8.GetBytes("TOUCH" + xClick + "<|>" + yClick + Environment.NewLine));
        }
    }
}

private void PictureBox1_MouseMove(object sender, MouseEventArgs e)
{
    foreach (ListViewItem item in lvConnections.SelectedItems)
    {
        // Remote screen resolution
        string[] tokens = item.SubItems[5].Text.Split('x'); // Ex: 1080x1920

        Client client = (Client)item.Tag;

        if (e.Button == MouseButtons.Left)
        {
            xClick = (e.X * int.Parse(tokens[0].ToString())) / (pictureBox1.Size.Width);
            yClick = (e.Y * int.Parse(tokens[1].ToString())) / (pictureBox1.Size.Height);

            client.sock.Send(Encoding.UTF8.GetBytes("MOUSESWIPESCREEN" + mdownPoint.X + "<|>" + mdownPoint.Y + "<|>" + xClick + "<|>" + yClick + Environment.NewLine));
        }
    }
}
android AccessibilityService

：

public void Swipe(int x1, int y1, int x2, int y2, int time) {

if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.N) {
    System.out.println(" ======= Swipe =======");

    GestureDescription.Builder gestureBuilder = new GestureDescription.Builder();
    Path path = new Path();
    path.moveTo(x1, y1);
    path.lineTo(x2, y2);

    gestureBuilder.addStroke(new GestureDescription.StrokeDescription(path, 100, time));
    dispatchGesture(gestureBuilder.build(), new GestureResultCallback() {
        @Override
        public void onCompleted(GestureDescription gestureDescription) {
            System.out.println("SWIPE Gesture Completed :D");
            super.onCompleted(gestureDescription);
        }
    }, null);
}

}
会产生以下结果（例如，仍然无法绘制“图案密码”，例如TeamViewer）。但是就像下面的评论中所说，我认为使用类似的方法可以使用Continued gestures来实现。欢迎对此方向提出任何建议。

---

编辑2：

当然，解决方案是continued gestures，如先前的Edit

所述。

• Simulating joystick movement using AccessibilityService
• Why the continueStroke function is not work

下面是我发现的here =>]假定的固定代码>

android AccessibilityService：

// Simulates an L-shaped drag path: 200 pixels right, then 200 pixels down.
Path path = new Path();
path.moveTo(200,200);
path.lineTo(400,200);

final GestureDescription.StrokeDescription sd = new GestureDescription.StrokeDescription(path, 0, 500, true);

// The starting point of the second path must match
// the ending point of the first path.
Path path2 = new Path();
path2.moveTo(400,200);
path2.lineTo(400,400);

final GestureDescription.StrokeDescription sd2 = sd.continueStroke(path2, 0, 500, false); // 0.5 second

HongBaoService.mService.dispatchGesture(new GestureDescription.Builder().addStroke(sd).build(), new AccessibilityService.GestureResultCallback(){

@Override
public void onCompleted(GestureDescription gestureDescription){
super.onCompleted(gestureDescription);
HongBaoService.mService.dispatchGesture(new GestureDescription.Builder().addStroke(sd2).build(),null,null);
}

@Override
public void onCancelled(GestureDescription gestureDescription){
super.onCancelled(gestureDescription);
}
},null);
然后，我的疑问是：如何为上面的代码正确发送鼠标坐标，从而可以向任意方向执行拖动？
有什么想法？

我想知道如何在基于X，Y鼠标坐标的android上执行拖动？考虑两个简单的示例，Team Viewer / QuickSupport在远程智能手机和Pen上绘制“密码模式” ...

Answer 1

您是否尝试过使用AutoIt脚本？

您可以在特定的窗口/屏幕中保存坐标。您可以在绘制图案时按住鼠标单击。