SwitchAccess 兼容的虚拟视图节点的深树层次结构
SwitchAccess compatible virtual View nodes' deep tree hierarchy
我正在尝试在 Android 中创建可访问的自定义 View。
View 由一个虚拟节点的树层次结构组成，在我的例子中可以是多层深度。
树不一定是二叉树：每一层上的每个节点都可以有用户想要的任意数量的子节点。
具体来说，我正在尝试使其与 TalkBack 的 SwitchAccess 辅助功能服务兼容。
最终目标是兼容TalkBack的所有无障碍服务，但至少要兼容SwitchAccess。
在这个 post 中，我正在尝试单开关访问。
事先不知道树的结构，也不知道它的扫描顺序。它们都是在运行时定义的。
我正在使用 class ExploreByTouchHelper，它是 AccessibilityDelegate.
的便利包装器
以下类似于事件序列的图像以及虚拟树：
示例扫描序列由节点内的数字表示。
从用户的角度来看，他们将叶子视为具有某种随机颜色的矩形。
这是什么才画到风俗View。
描绘的扫描顺序如下：
- 首先用户看到所有节点一起被扫描（有根被扫描）。他们按下开关并选择根，将扫描程序移至树的第二层。
- 然后扫描第一个内部节点，用户忽略它，所以扫描移动到第二个根的子节点，用户点击开关，将扫描过程移动到树的第三层，即单击节点的子节点。
- 依此类推，直到用户 clicks/selects 一个叶节点（在名为
7. Click! 的步骤），在这种情况下会发生自定义操作。
命名法：
- 什么是 SwitchAccess？这项服务允许用户通过将一个（或几个）硬件开关连接到他们的手机来键入 keys/letters。
你可以把它想象成一个为残障人士设计的物理键盘，它有一个（或几个）开关，而不是每个字母一个开关。
用户在称为 scanning 的过程的协助下键入字母，其中字母表中的每个字母（或任何类型的键）都被 scanned 一个接一个当用户单击他们的单个开关时，就会键入相应的字母。
这就像点击一个开关，点击的频率被该服务转换为字母。
- 什么是 TalkBack？ Android 用户可用的一组无障碍服务（包括 SwitchAccess）。
这是一个普通的 Android 应用程序（Android Accessibility Suite），如果您的系统中尚未包含，可以从 here 下载。
我尝试过（但失败了）的方法：
- 正在试验每个
AccessibilityNodeInfo 的焦点状态、选中状态和辅助功能焦点状态。
- 每次点击仅报告我感兴趣的子树部分。这意味着每次点击都会报告不同的树（通过
getVisibleVirtualViews）。
- 试验
AccessibilityNodeInfo.CollectionInfo 和 AccessibilityNodeInfo.CollectionItemInfo，灵感来自 GridView 的实现。
- 仅当用户单击叶时发送
AccessibilityEvents（而不是内部节点），以便在单击内部节点时继续扫描。
- 仅报告叶组而不是内部节点。
- Reinstalling/Changing 每个用户点击整个
AccessibilityNodeProvider and/or AccessibilityDelegate。
到目前为止我的最大努力（这是我上面的一些努力的组合），我们也可以就此进行讨论：
import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.graphics.Point;
import android.graphics.Rect;
import android.graphics.RectF;
import android.os.Bundle;
import android.view.KeyEvent;
import android.view.MotionEvent;
import android.view.View;
import android.view.ViewGroup;
import android.view.accessibility.AccessibilityEvent;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.view.ViewCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
import androidx.customview.widget.ExploreByTouchHelper;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Locale;
import java.util.Random;
public class MainActivity extends AppCompatActivity {
public static int randomColor(final int alpha, final Random rand) {
final byte[] components = new byte[3];
rand.nextBytes(components);
return Color.argb(alpha, components[0] & 0xFF, components[1] & 0xFF, components[2] & 0xFF);
}
public static class Node {
public Node parent = null;
public final ArrayList<Node> children = new ArrayList<>();
public final Point index = new Point(), //Location of this Node in its parent. Root Node does not use this.
size = new Point(); //Number of children this Node has per dimension (x == columns, y == rows).
public String text = null; //The text associated with each Node.
public int id = -1, //It is initialized at Tree construction.
color = 0; //It is only used for leaves, but for simplicity included in every Node.
/**
* Used as a default way to create the children of this Node.
* @param title Some value used to construct the text of each children.
* @param sizeX Number of columns each children will be initialized to have.
* @param sizeY Number of rows each children will be initialized to have.
* @param rand Used for producing each child's color.
*/
public void addChildren(final String title, final int sizeX, final int sizeY, final Random rand) {
for (int row = 0; row < size.y; ++row) {
for (int col = 0; col < size.x; ++col) {
final Node child = new Node();
child.parent = this;
children.add(child);
child.index.set(col, row);
child.size.set(sizeX, sizeY);
child.text = String.format(Locale.ENGLISH /*Just use ENGLISH for only the demonstration purposes.*/, "%s|%s:%d,%d", text, title, row, col);
child.color = randomColor(255, rand);
}
}
}
/** @param bounds Serves as input (initialized with the root Node's bounds) and as output (giving the bounds relative to root for the calling Node). */
public void updateBounds(final RectF bounds) {
if (parent != null) {
parent.updateBounds(bounds);
//Adjust parent bounds to locate the current node:
final float cellWidth = bounds.width() / parent.size.x, cellHeight = bounds.height() / parent.size.y;
bounds.left += (cellWidth * index.x);
bounds.top += (cellHeight * index.y);
bounds.right -= (cellWidth * (parent.size.x - index.x - 1));
bounds.bottom -= (cellHeight * (parent.size.y - index.y - 1));
}
}
}
/**
* Gets a subtree (starting from the given Node) of nodes into the given lists.
* @param node the root of the subtree we are interested in.
* @param allNodes all nodes of the subtree will go in here.
* @param leavesOnly only the leaves of the subtree will go in here.
*/
public static void getNodes(final Node node, final ArrayList<Node> allNodes, final ArrayList<Node> leavesOnly) {
allNodes.add(node);
if (node.children.isEmpty())
leavesOnly.add(node);
else
for (final Node child: node.children)
getNodes(child, allNodes, leavesOnly);
}
/** Sacrificing memory for speed: this is essentially a huge cache. */
public static class Tree {
public final Node root;
public final List<Node> nodes, //All nodes of the tree.
leaves; //Only leaves of the tree (which will exist in both 'nodes' property and in 'leaves' property).
public Tree(final Node root) {
this.root = root;
final ArrayList<Node> nodesList = new ArrayList<>();
final ArrayList<Node> leavesList = new ArrayList<>();
getNodes(root, nodesList, leavesList);
nodes = Collections.unmodifiableList(nodesList);
leaves = Collections.unmodifiableList(leavesList);
final int sz = nodes.size();
for (int i = 0; i < sz; ++i)
nodes.get(i).id = i; //As you can see the id corresponds exactly to the index of the Node in the list (so as to have easier+faster retrieval of Node by its id).
}
}
/** @return a Tree for testing. */
public static Tree buildTestTree() {
final Random rand = new Random();
final Node root = new Node();
root.size.set(2, 1); //2 columns, 1 row.
root.text = "Root";
root.addChildren("Inner", 2, 1, rand); //2 columns, 1 row.
for (final Node rootChild: root.children) {
rootChild.addChildren("Inner", 2, 1, rand); //2 columns, 1 row.
for (final Node rootInnerChild: rootChild.children)
rootInnerChild.addChildren("Leaf", 1, 1, rand); //1 column, 1 row. Basically a leaf.
}
return new Tree(root);
}
/** @return a value conforming to the measureSpec, while being as close as possible to the preferredSizeInPixels. */
public static int getViewSize(final int preferredSizeInPixels,
final int measureSpec) {
int result = preferredSizeInPixels;
final int specMode = View.MeasureSpec.getMode(measureSpec);
final int specSize = View.MeasureSpec.getSize(measureSpec);
switch (specMode) {
case View.MeasureSpec.UNSPECIFIED: result = preferredSizeInPixels; break;
case View.MeasureSpec.AT_MOST: result = Math.min(preferredSizeInPixels, specSize); break;
case View.MeasureSpec.EXACTLY: result = specSize; break;
}
return result;
}
/** The custom View which maintains the tree hierarchy of virtual views. */
public static class HierarchyView extends View {
private final MyAccessibilityDelegate delegate; //The ExploreByTouchHelper implementation.
public final Tree tree; //The tree of virtual views.
public Node selected; //The last 'clicked' node from all the nodes in the tree.
private final int preferredWidth, preferredHeight; //The preferred size of this View.
private final Paint tmpPaint; //Used for drawing.
private final RectF tmpBounds; //Used for drawing.
public HierarchyView(final Context context) {
super(context);
tmpPaint = new Paint();
tmpBounds = new RectF();
selected = null;
//Hardcoded magic numbers for the dimensions of this View, only in order to keep things simple in this demonstration:
preferredWidth = 600;
preferredHeight = 300;
tree = buildTestTree();
super.setContentDescription("Hierarchy");
ViewCompat.setImportantForAccessibility(this, ViewCompat.IMPORTANT_FOR_ACCESSIBILITY_YES);
ViewCompat.setAccessibilityDelegate(this, delegate = new MyAccessibilityDelegate(this));
}
@Override
protected void onMeasure(final int widthMeasureSpec, final int heightMeasureSpec) {
setMeasuredDimension(getViewSize(preferredWidth, widthMeasureSpec), getViewSize(preferredHeight, heightMeasureSpec));
}
/**
* Use this method instead of {@link Node#updateBounds(RectF)}, which (this method) will properly initialize the root Node's bounds.
* @param bounds The output bounds for the given Node.
* @param node The input Node to get the bounds for.
*/
public void updateBounds(final RectF bounds, final Node node) {
bounds.left = bounds.top = 0;
bounds.right = getWidth();
bounds.bottom = getHeight();
node.updateBounds(bounds);
}
@Override
protected void onDraw(final Canvas canvas) {
for (final Node leaf: tree.leaves) {
tmpPaint.setColor(leaf.color);
tmpPaint.setAlpha(selected == leaf? 255: 64);
updateBounds(tmpBounds, leaf); //Not the most efficient (needs logN), but remember this is just a demo.
canvas.drawRect(tmpBounds, tmpPaint);
}
}
@Override
public boolean dispatchHoverEvent(final MotionEvent event) {
//This is required by ExploreByTouchHelper's docs:
return delegate.dispatchHoverEvent(event) || super.dispatchHoverEvent(event);
}
@Override
public boolean dispatchKeyEvent(final KeyEvent event) {
//This is required by ExploreByTouchHelper's docs:
return delegate.dispatchKeyEvent(event) || super.dispatchKeyEvent(event);
}
@Override
protected void onFocusChanged(final boolean gainFocus, final int direction, final @Nullable Rect previouslyFocusedRect) {
super.onFocusChanged(gainFocus, direction, previouslyFocusedRect);
//This is required by ExploreByTouchHelper's docs:
delegate.onFocusChanged(gainFocus, direction, previouslyFocusedRect);
}
/*
@Override
public boolean onTouchEvent(final MotionEvent event) {
final int virtualViewId = delegate.getVirtualViewAt(event.getX(), event.getY());
if (virtualViewId != ExploreByTouchHelper.INVALID_ID)
selected = tree.nodes.get(virtualViewId);
invalidate();
return super.onTouchEvent(event);
}
*/
}
public static class MyAccessibilityDelegate extends ExploreByTouchHelper {
private final HierarchyView host;
/**
* This is used as the <b>parent</b> of each node that should be interactive. If null, then the
* root should be interactive, otherwise if not null, then the its children should be interactive.
*/
private Node last;
public MyAccessibilityDelegate(final @NonNull HierarchyView host) {
super(host);
this.host = host;
last = null; //Start with root.
}
/** Helper method to retrieve a read-only Iterable of the nodes that should be interactive. */
private Iterable<Node> readVisibleNodes() {
return last == null? Collections.singletonList(host.tree.root) : Collections.unmodifiableList(last.children);
}
@Override
protected int getVirtualViewAt(final float x, final float y) {
final RectF bounds = new RectF();
for (final Node node: readVisibleNodes()) {
host.updateBounds(bounds, node);
if (bounds.contains(x, y))
return node.id;
}
return INVALID_ID;
}
@Override
protected void getVisibleVirtualViews(final List<Integer> virtualViewIds) {
for (final Node node: readVisibleNodes())
virtualViewIds.add(node.id);
}
@Override
protected void onPopulateNodeForVirtualView(final int virtualViewId, final @NonNull AccessibilityNodeInfoCompat info) {
final Node node = host.tree.nodes.get(virtualViewId);
//Just set all text to node#text for simplicity:
info.setText(node.text);
info.setHintText(node.text);
info.setContentDescription(node.text);
//Get the node's bounds:
final RectF bounds = new RectF();
host.updateBounds(bounds, node);
/*Although deprecated, setBoundsInParent is actually what ExploreByTouchHelper requires, and itself
then computes the bounds in screen. So lets just setBoundsInParent, instead of setBoundsInScreen...*/
if (node.parent == null) { //If node is the root:
info.setParent(host); //The View itself is the parent of it (or maybe not, I am not sure).
info.setBoundsInParent(new Rect(Math.round(bounds.left), Math.round(bounds.top), Math.round(bounds.right), Math.round(bounds.bottom)));
}
else {
/*To get the bounds of any node which is not the root, I simply subtract the parent's bounds
with the current node's bounds. I know... not the most efficient, but it's just a demo now.*/
info.setParent(host, node.parent.id);
final RectF parentBounds = new RectF();
host.updateBounds(parentBounds, node.parent);
info.setBoundsInParent(new Rect(Math.round(bounds.left - parentBounds.left), Math.round(bounds.top - parentBounds.top), Math.round(bounds.right - parentBounds.left), Math.round(bounds.bottom - parentBounds.top)));
}
//As I have found out, those calls are absolutely necessary for the virtual views:
info.setEnabled(true);
info.setFocusable(true);
//These calls seem to not be absolutely necessary, but I am not sure:
info.setVisibleToUser(true);
info.setImportantForAccessibility(true);
// info.setContentInvalid(false);
// info.setAccessibilityFocused(last == node);
// info.setFocused(last == node);
// info.setChecked(last == node);
// info.setSelected(last == node);
if (node.parent == last) { //This is the way I am testing if the current node should be interactive.
info.setClickable(true);
info.setCheckable(true);
//info.setCanOpenPopup(true);
//info.setContextClickable(true);
//info.addAction(AccessibilityNodeInfoCompat.ACTION_SELECT);
info.addAction(AccessibilityNodeInfoCompat.ACTION_CLICK);
}
if (!node.children.isEmpty()) {
info.setCollectionInfo(AccessibilityNodeInfoCompat.CollectionInfoCompat.obtain(node.size.y, node.size.x, true, AccessibilityNodeInfoCompat.CollectionInfoCompat.SELECTION_MODE_SINGLE));
for (final Node child: node.children)
info.addChild(host, child.id);
}
if (node.parent != null)
info.setCollectionItemInfo(AccessibilityNodeInfoCompat.CollectionItemInfoCompat.obtain(node.index.y, 1, node.index.x, 1, false, false));
}
@Override
protected boolean onPerformActionForVirtualView(final int virtualViewId, final int action, final @Nullable Bundle arguments) {
if (action == AccessibilityNodeInfoCompat.ACTION_CLICK) {
host.selected = host.tree.nodes.get(virtualViewId);
last = host.selected.children.isEmpty()? null: host.selected;
// if (host.selected.children.isEmpty()) {
invalidateVirtualView(virtualViewId); //, AccessibilityEventCompat.CONTENT_CHANGE_TYPE_SUBTREE);
sendEventForVirtualView(virtualViewId, AccessibilityEvent.TYPE_VIEW_CLICKED);
host.invalidate(); //To redraw the UI.
// }
// else
// invalidateRoot();
// invalidateVirtualView(virtualViewId);
// host.sendAccessibilityEvent(AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED);
// host.sendAccessibilityEvent(AccessibilityEvent.TYPE_VIEW_CLICKED);
// host.sendAccessibilityEvent(AccessibilityEventCompat.TYPE_VIEW_CONTEXT_CLICKED);
// sendEventForVirtualView(virtualViewId, AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED);
// invalidateVirtualView(virtualViewId, AccessibilityEventCompat.CONTENT_CHANGE_TYPE_SUBTREE);
return true;
}
return false;
}
}
@Override
protected void onCreate(final Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(new HierarchyView(this), new ViewGroup.LayoutParams(ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT));
}
}
这导致扫描在点击时冻结（但至少它在树中更深一层），而实际需要的行为是让扫描继续，直到点击叶子，扫描应该结束。
几乎就像扫描软件键盘一样（即首先扫描一组键，然后在单击时继续扫描每个键，依此类推）。
请注意，由给定代码构建的树与图像的不完全相同。
图像作为视觉示例，而代码作为讨论的基础。
我主要是想找出 SwitchAccess 中的哪些规则将使我的应用程序与其兼容。
理想情况下，我希望用户看到正在扫描的叶子组，而不是像提供的代码中那样每次都看到单个节点，但我想这是另一回事了。
我想我问的是可能的，因为否则 setParent 之类的方法将不会包含在 AccessibilityNodeInfo class.
中
我也在考虑用 drawing order 进行实验，但我不知道它是否相关。
我正在使用最低 SDK 版本 14，如果有的话。
网上有一些关于单层深度虚拟树的例子，但我就是想不通如何让它们成为多层深度。
请注意，已采取几个步骤来缩短代码，因此它不会遵循有关面向对象编程的最佳实践，也不会遵循时间复杂度、内存使用等，因为它只是作为演示。
部分资源：
- The corresponding Google I/O 2013 video（从
ExploreByTouchHelper 用例的介绍开始）。
- TalkBack's source code。我看了很多次，但我仍然不知道如何解决我的问题。
- How to use SwitchAccess on your phone（用户视角）。
我不相信你想要的是可能的。或者至少，我不相信你想要的有支持它的“显式 APIs”。让我们谈谈“开关控制”将关注的重点。基本上，它会关注任何
- 未明确标记为对辅助功能不重要
- 有某种与之相关的动作
- 点击
- 点击并按住
- 自定义操作
- 等等
开关控制对事物进行分组的方式不响应任何特定的 API，而是根据与标准用户体验相关的现有信息并基于开关控制配置计算得出的。您可能会根据行列扫描与组选择的不同情况来设置它。一般来说，重要的事情是：
- 视图层次结构中的顺序
- 屏幕上的位置
为“精确分组”而操纵这些将非常困难。每个版本的开关控制都可以为所欲为。没有记录 API 说“这是分组”。 Switch Access 只是尽力理解标准 Android APIs.
从用户的角度准确表达您所掌握的信息。
- 将成组的事物放入虚拟布局中。
- 确保所有可以交互的东西都被标记为这样。
- 在您的虚拟层次结构中以合理的顺序放置您的视图
这就是您真正能做的。事实上，尝试对其进行更多操作可能会使用户感到困惑。
我正在尝试在 Android 中创建可访问的自定义 View。
View 由一个虚拟节点的树层次结构组成，在我的例子中可以是多层深度。
树不一定是二叉树：每一层上的每个节点都可以有用户想要的任意数量的子节点。
具体来说，我正在尝试使其与 TalkBack 的 SwitchAccess 辅助功能服务兼容。
最终目标是兼容TalkBack的所有无障碍服务，但至少要兼容SwitchAccess。
在这个 post 中，我正在尝试单开关访问。
事先不知道树的结构，也不知道它的扫描顺序。它们都是在运行时定义的。
我正在使用 class ExploreByTouchHelper，它是 AccessibilityDelegate.
以下类似于事件序列的图像以及虚拟树：
示例扫描序列由节点内的数字表示。
从用户的角度来看，他们将叶子视为具有某种随机颜色的矩形。
这是什么才画到风俗View。
描绘的扫描顺序如下：
- 首先用户看到所有节点一起被扫描（有根被扫描）。他们按下开关并选择根，将扫描程序移至树的第二层。
- 然后扫描第一个内部节点，用户忽略它，所以扫描移动到第二个根的子节点，用户点击开关，将扫描过程移动到树的第三层，即单击节点的子节点。
- 依此类推，直到用户 clicks/selects 一个叶节点（在名为
7. Click!的步骤），在这种情况下会发生自定义操作。
命名法：
- 什么是 SwitchAccess？这项服务允许用户通过将一个（或几个）硬件开关连接到他们的手机来键入 keys/letters。 你可以把它想象成一个为残障人士设计的物理键盘，它有一个（或几个）开关，而不是每个字母一个开关。 用户在称为 scanning 的过程的协助下键入字母，其中字母表中的每个字母（或任何类型的键）都被 scanned 一个接一个当用户单击他们的单个开关时，就会键入相应的字母。 这就像点击一个开关，点击的频率被该服务转换为字母。
- 什么是 TalkBack？ Android 用户可用的一组无障碍服务（包括 SwitchAccess）。 这是一个普通的 Android 应用程序（Android Accessibility Suite），如果您的系统中尚未包含，可以从 here 下载。
我尝试过（但失败了）的方法：
- 正在试验每个
AccessibilityNodeInfo的焦点状态、选中状态和辅助功能焦点状态。 - 每次点击仅报告我感兴趣的子树部分。这意味着每次点击都会报告不同的树（通过
getVisibleVirtualViews）。 - 试验
AccessibilityNodeInfo.CollectionInfo和AccessibilityNodeInfo.CollectionItemInfo，灵感来自GridView的实现。 - 仅当用户单击叶时发送
AccessibilityEvents（而不是内部节点），以便在单击内部节点时继续扫描。 - 仅报告叶组而不是内部节点。
- Reinstalling/Changing 每个用户点击整个
AccessibilityNodeProviderand/orAccessibilityDelegate。
到目前为止我的最大努力（这是我上面的一些努力的组合），我们也可以就此进行讨论：
import android.content.Context;
import android.graphics.Canvas;
import android.graphics.Color;
import android.graphics.Paint;
import android.graphics.Point;
import android.graphics.Rect;
import android.graphics.RectF;
import android.os.Bundle;
import android.view.KeyEvent;
import android.view.MotionEvent;
import android.view.View;
import android.view.ViewGroup;
import android.view.accessibility.AccessibilityEvent;
import androidx.annotation.NonNull;
import androidx.annotation.Nullable;
import androidx.appcompat.app.AppCompatActivity;
import androidx.core.view.ViewCompat;
import androidx.core.view.accessibility.AccessibilityNodeInfoCompat;
import androidx.customview.widget.ExploreByTouchHelper;
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
import java.util.Locale;
import java.util.Random;
public class MainActivity extends AppCompatActivity {
public static int randomColor(final int alpha, final Random rand) {
final byte[] components = new byte[3];
rand.nextBytes(components);
return Color.argb(alpha, components[0] & 0xFF, components[1] & 0xFF, components[2] & 0xFF);
}
public static class Node {
public Node parent = null;
public final ArrayList<Node> children = new ArrayList<>();
public final Point index = new Point(), //Location of this Node in its parent. Root Node does not use this.
size = new Point(); //Number of children this Node has per dimension (x == columns, y == rows).
public String text = null; //The text associated with each Node.
public int id = -1, //It is initialized at Tree construction.
color = 0; //It is only used for leaves, but for simplicity included in every Node.
/**
* Used as a default way to create the children of this Node.
* @param title Some value used to construct the text of each children.
* @param sizeX Number of columns each children will be initialized to have.
* @param sizeY Number of rows each children will be initialized to have.
* @param rand Used for producing each child's color.
*/
public void addChildren(final String title, final int sizeX, final int sizeY, final Random rand) {
for (int row = 0; row < size.y; ++row) {
for (int col = 0; col < size.x; ++col) {
final Node child = new Node();
child.parent = this;
children.add(child);
child.index.set(col, row);
child.size.set(sizeX, sizeY);
child.text = String.format(Locale.ENGLISH /*Just use ENGLISH for only the demonstration purposes.*/, "%s|%s:%d,%d", text, title, row, col);
child.color = randomColor(255, rand);
}
}
}
/** @param bounds Serves as input (initialized with the root Node's bounds) and as output (giving the bounds relative to root for the calling Node). */
public void updateBounds(final RectF bounds) {
if (parent != null) {
parent.updateBounds(bounds);
//Adjust parent bounds to locate the current node:
final float cellWidth = bounds.width() / parent.size.x, cellHeight = bounds.height() / parent.size.y;
bounds.left += (cellWidth * index.x);
bounds.top += (cellHeight * index.y);
bounds.right -= (cellWidth * (parent.size.x - index.x - 1));
bounds.bottom -= (cellHeight * (parent.size.y - index.y - 1));
}
}
}
/**
* Gets a subtree (starting from the given Node) of nodes into the given lists.
* @param node the root of the subtree we are interested in.
* @param allNodes all nodes of the subtree will go in here.
* @param leavesOnly only the leaves of the subtree will go in here.
*/
public static void getNodes(final Node node, final ArrayList<Node> allNodes, final ArrayList<Node> leavesOnly) {
allNodes.add(node);
if (node.children.isEmpty())
leavesOnly.add(node);
else
for (final Node child: node.children)
getNodes(child, allNodes, leavesOnly);
}
/** Sacrificing memory for speed: this is essentially a huge cache. */
public static class Tree {
public final Node root;
public final List<Node> nodes, //All nodes of the tree.
leaves; //Only leaves of the tree (which will exist in both 'nodes' property and in 'leaves' property).
public Tree(final Node root) {
this.root = root;
final ArrayList<Node> nodesList = new ArrayList<>();
final ArrayList<Node> leavesList = new ArrayList<>();
getNodes(root, nodesList, leavesList);
nodes = Collections.unmodifiableList(nodesList);
leaves = Collections.unmodifiableList(leavesList);
final int sz = nodes.size();
for (int i = 0; i < sz; ++i)
nodes.get(i).id = i; //As you can see the id corresponds exactly to the index of the Node in the list (so as to have easier+faster retrieval of Node by its id).
}
}
/** @return a Tree for testing. */
public static Tree buildTestTree() {
final Random rand = new Random();
final Node root = new Node();
root.size.set(2, 1); //2 columns, 1 row.
root.text = "Root";
root.addChildren("Inner", 2, 1, rand); //2 columns, 1 row.
for (final Node rootChild: root.children) {
rootChild.addChildren("Inner", 2, 1, rand); //2 columns, 1 row.
for (final Node rootInnerChild: rootChild.children)
rootInnerChild.addChildren("Leaf", 1, 1, rand); //1 column, 1 row. Basically a leaf.
}
return new Tree(root);
}
/** @return a value conforming to the measureSpec, while being as close as possible to the preferredSizeInPixels. */
public static int getViewSize(final int preferredSizeInPixels,
final int measureSpec) {
int result = preferredSizeInPixels;
final int specMode = View.MeasureSpec.getMode(measureSpec);
final int specSize = View.MeasureSpec.getSize(measureSpec);
switch (specMode) {
case View.MeasureSpec.UNSPECIFIED: result = preferredSizeInPixels; break;
case View.MeasureSpec.AT_MOST: result = Math.min(preferredSizeInPixels, specSize); break;
case View.MeasureSpec.EXACTLY: result = specSize; break;
}
return result;
}
/** The custom View which maintains the tree hierarchy of virtual views. */
public static class HierarchyView extends View {
private final MyAccessibilityDelegate delegate; //The ExploreByTouchHelper implementation.
public final Tree tree; //The tree of virtual views.
public Node selected; //The last 'clicked' node from all the nodes in the tree.
private final int preferredWidth, preferredHeight; //The preferred size of this View.
private final Paint tmpPaint; //Used for drawing.
private final RectF tmpBounds; //Used for drawing.
public HierarchyView(final Context context) {
super(context);
tmpPaint = new Paint();
tmpBounds = new RectF();
selected = null;
//Hardcoded magic numbers for the dimensions of this View, only in order to keep things simple in this demonstration:
preferredWidth = 600;
preferredHeight = 300;
tree = buildTestTree();
super.setContentDescription("Hierarchy");
ViewCompat.setImportantForAccessibility(this, ViewCompat.IMPORTANT_FOR_ACCESSIBILITY_YES);
ViewCompat.setAccessibilityDelegate(this, delegate = new MyAccessibilityDelegate(this));
}
@Override
protected void onMeasure(final int widthMeasureSpec, final int heightMeasureSpec) {
setMeasuredDimension(getViewSize(preferredWidth, widthMeasureSpec), getViewSize(preferredHeight, heightMeasureSpec));
}
/**
* Use this method instead of {@link Node#updateBounds(RectF)}, which (this method) will properly initialize the root Node's bounds.
* @param bounds The output bounds for the given Node.
* @param node The input Node to get the bounds for.
*/
public void updateBounds(final RectF bounds, final Node node) {
bounds.left = bounds.top = 0;
bounds.right = getWidth();
bounds.bottom = getHeight();
node.updateBounds(bounds);
}
@Override
protected void onDraw(final Canvas canvas) {
for (final Node leaf: tree.leaves) {
tmpPaint.setColor(leaf.color);
tmpPaint.setAlpha(selected == leaf? 255: 64);
updateBounds(tmpBounds, leaf); //Not the most efficient (needs logN), but remember this is just a demo.
canvas.drawRect(tmpBounds, tmpPaint);
}
}
@Override
public boolean dispatchHoverEvent(final MotionEvent event) {
//This is required by ExploreByTouchHelper's docs:
return delegate.dispatchHoverEvent(event) || super.dispatchHoverEvent(event);
}
@Override
public boolean dispatchKeyEvent(final KeyEvent event) {
//This is required by ExploreByTouchHelper's docs:
return delegate.dispatchKeyEvent(event) || super.dispatchKeyEvent(event);
}
@Override
protected void onFocusChanged(final boolean gainFocus, final int direction, final @Nullable Rect previouslyFocusedRect) {
super.onFocusChanged(gainFocus, direction, previouslyFocusedRect);
//This is required by ExploreByTouchHelper's docs:
delegate.onFocusChanged(gainFocus, direction, previouslyFocusedRect);
}
/*
@Override
public boolean onTouchEvent(final MotionEvent event) {
final int virtualViewId = delegate.getVirtualViewAt(event.getX(), event.getY());
if (virtualViewId != ExploreByTouchHelper.INVALID_ID)
selected = tree.nodes.get(virtualViewId);
invalidate();
return super.onTouchEvent(event);
}
*/
}
public static class MyAccessibilityDelegate extends ExploreByTouchHelper {
private final HierarchyView host;
/**
* This is used as the <b>parent</b> of each node that should be interactive. If null, then the
* root should be interactive, otherwise if not null, then the its children should be interactive.
*/
private Node last;
public MyAccessibilityDelegate(final @NonNull HierarchyView host) {
super(host);
this.host = host;
last = null; //Start with root.
}
/** Helper method to retrieve a read-only Iterable of the nodes that should be interactive. */
private Iterable<Node> readVisibleNodes() {
return last == null? Collections.singletonList(host.tree.root) : Collections.unmodifiableList(last.children);
}
@Override
protected int getVirtualViewAt(final float x, final float y) {
final RectF bounds = new RectF();
for (final Node node: readVisibleNodes()) {
host.updateBounds(bounds, node);
if (bounds.contains(x, y))
return node.id;
}
return INVALID_ID;
}
@Override
protected void getVisibleVirtualViews(final List<Integer> virtualViewIds) {
for (final Node node: readVisibleNodes())
virtualViewIds.add(node.id);
}
@Override
protected void onPopulateNodeForVirtualView(final int virtualViewId, final @NonNull AccessibilityNodeInfoCompat info) {
final Node node = host.tree.nodes.get(virtualViewId);
//Just set all text to node#text for simplicity:
info.setText(node.text);
info.setHintText(node.text);
info.setContentDescription(node.text);
//Get the node's bounds:
final RectF bounds = new RectF();
host.updateBounds(bounds, node);
/*Although deprecated, setBoundsInParent is actually what ExploreByTouchHelper requires, and itself
then computes the bounds in screen. So lets just setBoundsInParent, instead of setBoundsInScreen...*/
if (node.parent == null) { //If node is the root:
info.setParent(host); //The View itself is the parent of it (or maybe not, I am not sure).
info.setBoundsInParent(new Rect(Math.round(bounds.left), Math.round(bounds.top), Math.round(bounds.right), Math.round(bounds.bottom)));
}
else {
/*To get the bounds of any node which is not the root, I simply subtract the parent's bounds
with the current node's bounds. I know... not the most efficient, but it's just a demo now.*/
info.setParent(host, node.parent.id);
final RectF parentBounds = new RectF();
host.updateBounds(parentBounds, node.parent);
info.setBoundsInParent(new Rect(Math.round(bounds.left - parentBounds.left), Math.round(bounds.top - parentBounds.top), Math.round(bounds.right - parentBounds.left), Math.round(bounds.bottom - parentBounds.top)));
}
//As I have found out, those calls are absolutely necessary for the virtual views:
info.setEnabled(true);
info.setFocusable(true);
//These calls seem to not be absolutely necessary, but I am not sure:
info.setVisibleToUser(true);
info.setImportantForAccessibility(true);
// info.setContentInvalid(false);
// info.setAccessibilityFocused(last == node);
// info.setFocused(last == node);
// info.setChecked(last == node);
// info.setSelected(last == node);
if (node.parent == last) { //This is the way I am testing if the current node should be interactive.
info.setClickable(true);
info.setCheckable(true);
//info.setCanOpenPopup(true);
//info.setContextClickable(true);
//info.addAction(AccessibilityNodeInfoCompat.ACTION_SELECT);
info.addAction(AccessibilityNodeInfoCompat.ACTION_CLICK);
}
if (!node.children.isEmpty()) {
info.setCollectionInfo(AccessibilityNodeInfoCompat.CollectionInfoCompat.obtain(node.size.y, node.size.x, true, AccessibilityNodeInfoCompat.CollectionInfoCompat.SELECTION_MODE_SINGLE));
for (final Node child: node.children)
info.addChild(host, child.id);
}
if (node.parent != null)
info.setCollectionItemInfo(AccessibilityNodeInfoCompat.CollectionItemInfoCompat.obtain(node.index.y, 1, node.index.x, 1, false, false));
}
@Override
protected boolean onPerformActionForVirtualView(final int virtualViewId, final int action, final @Nullable Bundle arguments) {
if (action == AccessibilityNodeInfoCompat.ACTION_CLICK) {
host.selected = host.tree.nodes.get(virtualViewId);
last = host.selected.children.isEmpty()? null: host.selected;
// if (host.selected.children.isEmpty()) {
invalidateVirtualView(virtualViewId); //, AccessibilityEventCompat.CONTENT_CHANGE_TYPE_SUBTREE);
sendEventForVirtualView(virtualViewId, AccessibilityEvent.TYPE_VIEW_CLICKED);
host.invalidate(); //To redraw the UI.
// }
// else
// invalidateRoot();
// invalidateVirtualView(virtualViewId);
// host.sendAccessibilityEvent(AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED);
// host.sendAccessibilityEvent(AccessibilityEvent.TYPE_VIEW_CLICKED);
// host.sendAccessibilityEvent(AccessibilityEventCompat.TYPE_VIEW_CONTEXT_CLICKED);
// sendEventForVirtualView(virtualViewId, AccessibilityEvent.TYPE_WINDOW_STATE_CHANGED);
// invalidateVirtualView(virtualViewId, AccessibilityEventCompat.CONTENT_CHANGE_TYPE_SUBTREE);
return true;
}
return false;
}
}
@Override
protected void onCreate(final Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(new HierarchyView(this), new ViewGroup.LayoutParams(ViewGroup.LayoutParams.WRAP_CONTENT, ViewGroup.LayoutParams.WRAP_CONTENT));
}
}
这导致扫描在点击时冻结（但至少它在树中更深一层），而实际需要的行为是让扫描继续，直到点击叶子，扫描应该结束。 几乎就像扫描软件键盘一样（即首先扫描一组键，然后在单击时继续扫描每个键，依此类推）。
请注意，由给定代码构建的树与图像的不完全相同。 图像作为视觉示例，而代码作为讨论的基础。
我主要是想找出 SwitchAccess 中的哪些规则将使我的应用程序与其兼容。 理想情况下，我希望用户看到正在扫描的叶子组，而不是像提供的代码中那样每次都看到单个节点，但我想这是另一回事了。
我想我问的是可能的，因为否则 setParent 之类的方法将不会包含在 AccessibilityNodeInfo class.
我也在考虑用 drawing order 进行实验，但我不知道它是否相关。
我正在使用最低 SDK 版本 14，如果有的话。
网上有一些关于单层深度虚拟树的例子，但我就是想不通如何让它们成为多层深度。
请注意，已采取几个步骤来缩短代码，因此它不会遵循有关面向对象编程的最佳实践，也不会遵循时间复杂度、内存使用等，因为它只是作为演示。
部分资源：
- The corresponding Google I/O 2013 video（从
ExploreByTouchHelper用例的介绍开始）。 - TalkBack's source code。我看了很多次，但我仍然不知道如何解决我的问题。
- How to use SwitchAccess on your phone（用户视角）。
我不相信你想要的是可能的。或者至少，我不相信你想要的有支持它的“显式 APIs”。让我们谈谈“开关控制”将关注的重点。基本上，它会关注任何
- 未明确标记为对辅助功能不重要
- 有某种与之相关的动作
- 点击
- 点击并按住
- 自定义操作
- 等等
开关控制对事物进行分组的方式不响应任何特定的 API，而是根据与标准用户体验相关的现有信息并基于开关控制配置计算得出的。您可能会根据行列扫描与组选择的不同情况来设置它。一般来说，重要的事情是：
- 视图层次结构中的顺序
- 屏幕上的位置
为“精确分组”而操纵这些将非常困难。每个版本的开关控制都可以为所欲为。没有记录 API 说“这是分组”。 Switch Access 只是尽力理解标准 Android APIs.
从用户的角度准确表达您所掌握的信息。
- 将成组的事物放入虚拟布局中。
- 确保所有可以交互的东西都被标记为这样。
- 在您的虚拟层次结构中以合理的顺序放置您的视图
这就是您真正能做的。事实上，尝试对其进行更多操作可能会使用户感到困惑。