如何使用AccessibilityService在Android上执行拖放操作（基于X，Y鼠标坐标？

Question

此bounty已结束。该问题的答案有资格获得+ 400声望赏金。赏金宽限期在15小时内结束。BrowJr正在寻找规范答案。

我想知道如何基于X，Y鼠标坐标在android上模拟拖放？考虑两个简单的示例，Team Viewer / QuickSupport分别在远程智能手机和Windows Paint Pen上绘制“密码模式”：

我所能做的就是simulate touch（带有dispatchGesture()和AccessibilityNodeInfo.ACTION_CLICK）。

我找到了这些相关链接，但不知道它们是否有用：

• Perform swipe on screen using AccessibilityService
• Example 1
• Continued gestures

下面是我的工作代码，用于将鼠标坐标（在PictureBox控件内部）发送到远程电话并模拟触摸。

Windows窗体应用程序：

private void pictureBox1_MouseDown(object sender, MouseEventArgs e)
{
    foreach (ListViewItem item in lvConnections.SelectedItems)
    {
        // Remote screen resolution
        string[] tokens = item.SubItems[5].Text.Split('x'); // Ex: 1080x1920

        int xClick = (e.X * int.Parse(tokens[0].ToString())) / (pictureBox1.Size.Width);
        int yClick = (e.Y * int.Parse(tokens[1].ToString())) / (pictureBox1.Size.Height);

        Client client = (Client)item.Tag;

        if (e.Button == MouseButtons.Left)
            client.sock.Send(Encoding.UTF8.GetBytes("TOUCH" + xClick + "<|>" + yClick + Environment.NewLine));
    }
}

---

编辑：

我最后一次尝试是分别使用鼠标坐标（C＃Windows Forms Application）和自定义android例程（参考上面链接的“ swipe screen”的代码）进行“ swipe screen”：

private Point mdownPoint = new Point();

private void pictureBox1_MouseDown(object sender, MouseEventArgs e)
{
    foreach (ListViewItem item in lvConnections.SelectedItems)
    {
        // Remote screen resolution
        string[] tokens = item.SubItems[5].Text.Split('x'); // Ex: 1080x1920

        Client client = (Client)item.Tag;

        if (e.Button == MouseButtons.Left)
        {
            xClick = (e.X * int.Parse(tokens[0].ToString())) / (pictureBox1.Size.Width); 
            yClick = (e.Y * int.Parse(tokens[1].ToString())) / (pictureBox1.Size.Height);

            // Saving start position:

            mdownPoint.X = xClick; 
            mdownPoint.Y = yClick; 

            client.sock.Send(Encoding.UTF8.GetBytes("TOUCH" + xClick + "<|>" + yClick + Environment.NewLine));
        }
    }
}

private void PictureBox1_MouseMove(object sender, MouseEventArgs e)
{
    foreach (ListViewItem item in lvConnections.SelectedItems)
    {
        // Remote screen resolution
        string[] tokens = item.SubItems[5].Text.Split('x'); // Ex: 1080x1920

        Client client = (Client)item.Tag;

        if (e.Button == MouseButtons.Left)
        {
            xClick = (e.X * int.Parse(tokens[0].ToString())) / (pictureBox1.Size.Width);
            yClick = (e.Y * int.Parse(tokens[1].ToString())) / (pictureBox1.Size.Height);

            client.sock.Send(Encoding.UTF8.GetBytes("MOUSESWIPESCREEN" + mdownPoint.X + "<|>" + mdownPoint.Y + "<|>" + xClick + "<|>" + yClick + Environment.NewLine));
        }
    }
}

android AccessibilityService：

public void Swipe(int x1, int y1, int x2, int y2, int time) {

if (android.os.Build.VERSION.SDK_INT >= android.os.Build.VERSION_CODES.N) {
    System.out.println(" ======= Swipe =======");

    GestureDescription.Builder gestureBuilder = new GestureDescription.Builder();
    Path path = new Path();
    path.moveTo(x1, y1);
    path.lineTo(x2, y2);

    gestureBuilder.addStroke(new GestureDescription.StrokeDescription(path, 100, time));
    dispatchGesture(gestureBuilder.build(), new GestureResultCallback() {
        @Override
        public void onCompleted(GestureDescription gestureDescription) {
            System.out.println("SWIPE Gesture Completed :D");
            super.onCompleted(gestureDescription);
        }
    }, null);
}

}

会产生以下结果（但仍无法像TeamViewer一样绘制“模式密码”）。但是就像下面的评论中所说，我认为使用类似的方法可以使用Continued gestures来实现。欢迎对此方向提出任何建议。

Answer 1

您是否尝试过使用AutoIt脚本？

您可以在特定的窗口/屏幕中保存坐标。您可以在绘制图案时按住鼠标单击。

如果需要，我也为您提供了一些示例代码/脚本！