Android 无法在少数使用无障碍服务的设备上阅读 window 内容

Question

我的要求：从特定应用程序的弹出窗口、对话框等读取文本。

我已经实施了无障碍服务，并且正在根据我的要求接收适当的事件和数据。然而，在测试时我意识到，在某些设备上，他们没有使用 AlertDialog 或 Dialog，而是使用了 activity（主题为对话框）。因此，在我的可访问性事件中，我只收到 activity 标题，有没有办法找到这个特定弹出窗口显示的文本 activity?

我已经很努力地搜索了，但在这个主题上找不到太多帮助，文档也没有解决这个问题。无障碍服务的代码不多，如果你还需要我稍后再post

谢谢

Answer 1

这是我使用的代码，它适用于我：

public class USSDService extends AccessibilityService {

public static String TAG = USSDService.class.getSimpleName();

@Override
public void onAccessibilityEvent(AccessibilityEvent event) {
    Log.d(TAG, "onAccessibilityEvent");

    AccessibilityNodeInfo source = event.getSource();
    if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED && !String.valueOf(event.getClassName()).contains("AlertDialog")) {
        return;
    }
    if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED && (source == null || !source.getClassName().equals("android.widget.TextView"))) {
        return;
    }
    if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED && TextUtils.isEmpty(source.getText())) {
        return;
    }

    List<CharSequence> eventText;

    if(event.getEventType() == AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED) {
        eventText = event.getText();
    } else {
        eventText = Collections.singletonList(source.getText());
    }

    String text = processUSSDText(eventText);

    if( TextUtils.isEmpty(text) ) return;

    // Close dialog
    performGlobalAction(GLOBAL_ACTION_BACK); // This works on 4.1+ only

    Log.d(TAG, text);
    // Handle USSD response here

}

private String processUSSDText(List<CharSequence> eventText) {
    for (CharSequence s : eventText) {
        String text = String.valueOf(s);
        // Return text if text is the expected ussd response
        if( true ) {
            return text;
        }
    }
    return null;
}

@Override
public void onInterrupt() {
}

@Override
protected void onServiceConnected() {
    super.onServiceConnected();
    Log.d(TAG, "onServiceConnected");
    AccessibilityServiceInfo info = new AccessibilityServiceInfo();
    info.flags = AccessibilityServiceInfo.DEFAULT;
    info.packageNames = new String[]{"com.android.phone"};
    info.eventTypes = AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED | AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED;
    info.feedbackType = AccessibilityServiceInfo.FEEDBACK_GENERIC;
    setServiceInfo(info);
}
}

在Android清单中声明

<service android:name=".USSDService"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
    <action android:name="android.accessibilityservice.AccessibilityService" />
</intent-filter>
<meta-data android:name="android.accessibilityservice"
    android:resource="@xml/ussd_service" />

创建一个 xml 文件来描述名为 ussd_service

的无障碍服务

<?xml version="1.0" encoding="utf-8"?>
<accessibility-service xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeWindowStateChanged|typeWindowContentChanged"
android:accessibilityFeedbackType="feedbackGeneric"
android:accessibilityFlags="flagDefault"
android:canRetrieveWindowContent="true"
android:description="@string/accessibility_service_description"
android:notificationTimeout="0"
android:packageNames="com.android.phone" />

Answer 2

即使在 phone 正在 returning 的情况下，也可以使用 accessiblityNodeInfo 获取信息，它会获取 ussd 响应，当有输入多个选项的选项时，它也会关闭对话框。

首先，在 [pohne] 事件的情况下 class 名称被 return 编辑为 ussdalertactivty，所以我只使用 "alert" 来识别 ussd 响应的警报对话框。

public void onAccessibilityEvent(AccessibilityEvent event) {
    if (event.getPackageName().toString().equals("com.android.phone")
                        && event.getClassName().toString().toLowerCase()
                                .contains("alert")) {

                    AccessibilityNodeInfo source = event.getSource();

                    if (source != null) {
                    String pcnResponse = fetchResponse(source);
                    }
    }

现在我创建了一个名为 fetchResponse 的函数，它将 return 来自 pcn 的响应作为字符串并关闭对话框，因此需要执行 performGlobalAction(GLOBAL_ACTION_BACK).

private String fetchResponse(AccessibilityNodeInfo accessibilityNodeInfo) {

        String fetchedResponse = "";
        if (accessibilityNodeInfo != null) {
            for (int i = 0; i < accessibilityNodeInfo.getChildCount(); i++) {
                AccessibilityNodeInfo child = accessibilityNodeInfo.getChild(i);
                if (child != null) {

                    CharSequence text = child.getText();

                    if (text != null
                            && child.getClassName().equals(
                                    Button.class.getName())) {

                        // dismiss dialog by performing action click in normal
                        // cases
                        if((text.toString().toLowerCase().equals("ok") || text
                                        .toString().toLowerCase()
                                        .equals("cancel"))) {

                            child.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                            return fetchedResponse;

                        }

                    } else if (text != null
                            && child.getClassName().equals(
                                    TextView.class.getName())) {

                        // response of normal cases
                        if (text.toString().length() > 10) {
                            fetchedResponse = text.toString();
                        }

                    } else if (child.getClassName().equals(
                            ScrollView.class.getName())) {

                        // when response comes as phone then response can be
                        // retrived from subchild
                        for (int j = 0; j < child.getChildCount(); j++) {

                            AccessibilityNodeInfo subChild = child.getChild(j);
                            CharSequence subText = subChild.getText();

                            if (subText != null
                                    && subChild.getClassName().equals(
                                            TextView.class.getName())) {

                                // response of different cases
                                if (subText.toString().length() > 10) {
                                    fetchedResponse = subText.toString();
                                }

                            }

                            else if (subText != null
                                    && subChild.getClassName().equals(
                                            Button.class.getName())) {

                                // dismiss dialog by performing action click in
                                // different
                                // cases
                                if ((subText.toString().toLowerCase()
                                                .equals("ok") || subText
                                                .toString().toLowerCase()
                                                .equals("cancel"))) {
                                    subChild.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                                    return fetchedResponse;
                                }

                            }
                        }
                    }
                }
            }
        }
        return fetchedResponse;
    }

希望这能解决所有设备的问题。

Answer 3

你应该检查是否有子节点的子节点。

private void clickPerform(AccessibilityNodeInfo nodeInfo)
 {
   if(nodeInfo != null)
 {

            for (int i = 0; i < nodeInfo.getChildCount(); i++) {
                AccessibilityNodeInfo childNode = nodeInfo.getChild(i);
                Log.e("test", "clickPerform: "+childNode );
                if (childNode != null) {
                    for (int j = 0; j <= childNode.getChildCount(); j++) {
                        AccessibilityNodeInfo subChild = childNode.getChild(i);

                        if (String.valueOf(subChild.getText()).toLowerCase().equals("ok")) {
                            subChild.performAction(AccessibilityNodeInfo.ACTION_CLICK);
                        } else {
                            Log.e("t2", "clickPerform: ");
                        }
                    }
                }

                }
            }
}

Android 无法在少数使用无障碍服务的设备上阅读 window 内容

Android unable read window content on few devices using accessibility service

android

accessibilityservice

accessibility-api