"Taking Action for Users" 如何使用无障碍服务？
How to use Accessibility Services for "Taking Action for Users"?
背景
回一个few years ago，我问TeamViewer如何让用户在不与设备正常交互的情况下控制设备。有人告诉我这是一个特殊的 "backdoor" 制造商专门允许此应用程序，并且只能对其他应用程序使用 root 权限。
看到像 "Airplane Mode Shortcut" 这样的应用程序允许切换飞行模式，通过自动导航到它的屏幕并切换开关，这让我意识到这种情况已经改变了。
问题
the docs中说：
Starting with Android 4.0 (API Level 14), accessibility services can
act on behalf of users, including changing the input focus and
selecting (activating) user interface elements. In Android 4.1 (API
Level 16) the range of actions has been expanded to include scrolling
lists and interacting with text fields. Accessibility services can
also take global actions, such as navigating to the Home screen,
pressing the Back button, opening the notifications screen and recent
applications list. Android 4.1 also includes a new type of focus,
Accessibilty Focus, which makes all visible elements selectable by an
accessibility service.
These new capabilities make it possible for developers of
accessibility services to create alternative navigation modes such as
gesture navigation, and give users with disabilities improved control
of their Android devices.
但是没有更多关于如何使用它的信息。
只有我找到的示例位于底部，但这些示例非常旧并且是 apiDemos 包的一部分。
问题
如何制作可以查询、聚焦、单击、输入文本以及执行其他 UI 相关操作的服务？
通过实施 AccessibilityService (https://developer.android.com/training/accessibility/service.html)，您可以访问该功能。
您可以检查或对用户最后交互的元素执行操作，或者检查当前处于活动状态的整个应用程序。
通过实现 onAccessibilityEvent(AccessibilityEvent event) 拦截用户事件，在这里你可以用 event.getSource() 检索虚拟视图（代表原始视图），然后用 getClassName() 或 getText() 检查它或您在文档中找到的任何内容。
通过调用 getRootInActiveWindow() 检查整个应用程序并使用 getRootInActiveWindow().getChild(index) 遍历虚拟视图树。
getRootInActiveWindow() 和 event.getSource() return AccessibilityNodeInfo，您可以在其上调用 performAction(action) 并执行类似 Click, 设置文本, 等等
示例：Play 商店
打开 Play 商店应用后，搜索 'facebook' 应用并在 Play 商店中打开它的页面。
@Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if(suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
编辑：完整代码如下：
MyAccessibilityService
public class MyAccessibilityService extends AccessibilityService {
@Override
public void onCreate() {
super.onCreate();
Log.d("MyAccessibilityService", "onCreate");
}
@Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
Log.d("MyAccessibilityService", "onAccessibilityEvent");
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if (suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
@Override
public void onInterrupt() {
}
}
AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.findfacebookapp">
<application
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:supportsRtl="true"
android:theme="@style/AppTheme">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
<service
android:name=".MyAccessibilityService"
android:label="@string/accessibility_service_label"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService"/>
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="@xml/accessibility_service_config"/>
</service>
</application>
</manifest>
res/xml/accessibility_service_config.xml
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackAllMask"
android:accessibilityFlags="flagDefault"
android:canRequestEnhancedWebAccessibility="true"
android:canRetrieveWindowContent="true"
android:description="@string/app_name"
android:notificationTimeout="100"/>
MainActivity
public class MainActivity extends AppCompatActivity {
public void onEnableAccClick(View view) {
startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
}
}
背景
回一个few years ago，我问TeamViewer如何让用户在不与设备正常交互的情况下控制设备。有人告诉我这是一个特殊的 "backdoor" 制造商专门允许此应用程序，并且只能对其他应用程序使用 root 权限。
看到像 "Airplane Mode Shortcut" 这样的应用程序允许切换飞行模式，通过自动导航到它的屏幕并切换开关，这让我意识到这种情况已经改变了。
问题
the docs中说：
Starting with Android 4.0 (API Level 14), accessibility services can act on behalf of users, including changing the input focus and selecting (activating) user interface elements. In Android 4.1 (API Level 16) the range of actions has been expanded to include scrolling lists and interacting with text fields. Accessibility services can also take global actions, such as navigating to the Home screen, pressing the Back button, opening the notifications screen and recent applications list. Android 4.1 also includes a new type of focus, Accessibilty Focus, which makes all visible elements selectable by an accessibility service.
These new capabilities make it possible for developers of accessibility services to create alternative navigation modes such as gesture navigation, and give users with disabilities improved control of their Android devices.
但是没有更多关于如何使用它的信息。 只有我找到的示例位于底部，但这些示例非常旧并且是 apiDemos 包的一部分。
问题
如何制作可以查询、聚焦、单击、输入文本以及执行其他 UI 相关操作的服务？
通过实施 AccessibilityService (https://developer.android.com/training/accessibility/service.html)，您可以访问该功能。
您可以检查或对用户最后交互的元素执行操作，或者检查当前处于活动状态的整个应用程序。
通过实现 onAccessibilityEvent(AccessibilityEvent event) 拦截用户事件，在这里你可以用 event.getSource() 检索虚拟视图（代表原始视图），然后用 getClassName() 或 getText() 检查它或您在文档中找到的任何内容。
通过调用 getRootInActiveWindow() 检查整个应用程序并使用 getRootInActiveWindow().getChild(index) 遍历虚拟视图树。
getRootInActiveWindow() 和 event.getSource() return AccessibilityNodeInfo，您可以在其上调用 performAction(action) 并执行类似 Click, 设置文本, 等等
示例：Play 商店
打开 Play 商店应用后，搜索 'facebook' 应用并在 Play 商店中打开它的页面。
@Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if(suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
编辑：完整代码如下：
MyAccessibilityService
public class MyAccessibilityService extends AccessibilityService {
@Override
public void onCreate() {
super.onCreate();
Log.d("MyAccessibilityService", "onCreate");
}
@Override
public void onAccessibilityEvent(final AccessibilityEvent event) {
Log.d("MyAccessibilityService", "onAccessibilityEvent");
AccessibilityNodeInfo rootInActiveWindow = getRootInActiveWindow();
//Inspect app elements if ready
if (rootInActiveWindow != null) {
//Search bar is covered with textview which need to be clicked
List<AccessibilityNodeInfo> searchBarIdle = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_idle_text");
if (searchBarIdle.size() > 0) {
AccessibilityNodeInfo searchBar = searchBarIdle.get(0);
searchBar.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
//Check is search bar is visible
List<AccessibilityNodeInfo> searchBars = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/search_box_text_input");
if (searchBars.size() > 0) {
AccessibilityNodeInfo searchBar = searchBars.get(0);
//Check is searchbar have the required text, if not set the text
if (searchBar.getText() == null || !searchBar.getText().toString().equalsIgnoreCase("facebook")) {
Bundle args = new Bundle();
args.putString(AccessibilityNodeInfo.ACTION_ARGUMENT_SET_TEXT_CHARSEQUENCE, "facebook");
searchBar.performAction(AccessibilityNodeInfo.ACTION_SET_TEXT, args);
} else {
//There is no way to press Enter to perform search, so find corresponding suggestion and click
List<AccessibilityNodeInfo> searchSuggestions = rootInActiveWindow.findAccessibilityNodeInfosByViewId("com.android.vending:id/suggest_text");
for (AccessibilityNodeInfo suggestion : searchSuggestions) {
if (suggestion.getText().toString().equals("Facebook")) {
//We found textview, but its not clickable, so we should perform the click on the parent
AccessibilityNodeInfo clickableParent = suggestion.getParent();
clickableParent.performAction(AccessibilityNodeInfo.ACTION_CLICK);
}
}
}
}
}
}
@Override
public void onInterrupt() {
}
}
AndroidManifest.xml
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
package="com.example.findfacebookapp">
<application
android:allowBackup="true"
android:icon="@mipmap/ic_launcher"
android:label="@string/app_name"
android:supportsRtl="true"
android:theme="@style/AppTheme">
<activity android:name=".MainActivity">
<intent-filter>
<action android:name="android.intent.action.MAIN"/>
<category android:name="android.intent.category.LAUNCHER"/>
</intent-filter>
</activity>
<service
android:name=".MyAccessibilityService"
android:label="@string/accessibility_service_label"
android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE">
<intent-filter>
<action android:name="android.accessibilityservice.AccessibilityService"/>
</intent-filter>
<meta-data
android:name="android.accessibilityservice"
android:resource="@xml/accessibility_service_config"/>
</service>
</application>
</manifest>
res/xml/accessibility_service_config.xml
<?xml version="1.0" encoding="utf-8"?>
<accessibility-service
xmlns:android="http://schemas.android.com/apk/res/android"
android:accessibilityEventTypes="typeAllMask"
android:accessibilityFeedbackType="feedbackAllMask"
android:accessibilityFlags="flagDefault"
android:canRequestEnhancedWebAccessibility="true"
android:canRetrieveWindowContent="true"
android:description="@string/app_name"
android:notificationTimeout="100"/>
MainActivity
public class MainActivity extends AppCompatActivity {
public void onEnableAccClick(View view) {
startActivityForResult(new Intent(Settings.ACTION_ACCESSIBILITY_SETTINGS), 1);
}
}