Google 现在的自定义命令

Question

我正在尝试 Google 现在接受自定义命令并在进行特定查询时向我的应用程序发送 Intent。

我使用 Tasker 和 Autovoice 成功地做到了这一点，但我想在不使用这些应用程序的情况下做同样的事情。

我在文档中找到了这个 link。我在哪里可以处理未完成任务的常见意图。

我也尝试了Google提供的语音交互API，这几乎是一回事，但这并没有帮助。

这里有没有人在不使用 Commander、Autovoice 或 Tasker 等其他应用程序的情况下实现这一目标的？

Answer 1

不是你想听到的，但是API的当前版本不允许自定义语音命令：

来自 https://developers.google.com/voice-actions/custom-actions

Note: We are not accepting requests for Custom Voice Actions. Stay tuned to Voice Actions - Google Developers and +GoogleDevelopers for product updates.

Answer 2

Google 目前没有 'accept' 自定义命令。您详细说明的应用程序使用 AcccessibilityService 'hack' to intercept the voice command, or for rooted devices, the xposed framework.

他们要么对它们采取行动，同时杀死 Google，要么忽略它们并允许 Google 照常显示其结果。

出于多种原因，这是一个坏主意：

Google 将找到一种方法来防止这种类型的交互，如果它变得司空见惯，因为他们显然不希望他们的 Now 服务受到负面影响。
它使用硬编码常量，与 classes Google 用于显示语音命令的视图相关。这当然会随着每个版本的变化而变化。
黑客破解！

免责声明完成！使用风险自负....

您需要在 Manifest 中注册一个 AccessibilityService:

    <service
        android:name="com.something.MyAccessibilityService"
        android:enabled="true"
        android:label="@string/label"
        android:permission="android.permission.BIND_ACCESSIBILITY_SERVICE" >
        <intent-filter>
            <action android:name="android.accessibilityservice.AccessibilityService" />
        </intent-filter>

        <meta-data
            android:name="android.accessibilityservice"
            android:resource="@xml/accessibilityconfig" />
    </service>

并将配置文件添加到 res/xml:

<accessibility-service
    xmlns:android="http://schemas.android.com/apk/res/android"
    android:accessibilityEventTypes="typeWindowContentChanged"
    android:accessibilityFeedbackType="feedbackGeneric"
    android:accessibilityFlags="flagIncludeNotImportantViews"
    android:canRetrieveWindowContent="true"
    android:description="@string/accessibility_description"
    android:notificationTimeout="100"
    android:settingsActivity="SettingsActivity"/>

您可以选择添加：

    android:packageNames="xxxxxx"

或通过添加更多事件类型来扩展功能：

    android:accessibilityEventTypes="typeViewTextSelectionChanged|typeWindowContentChanged|typeNotificationStateChanged"

包括以下 AccessibilityService 示例 class:

/*
 * Copyright (c) 2016 Ben Randall
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 *   http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

package com.your.package;

import android.accessibilityservice.AccessibilityService;
import android.support.annotation.NonNull;
import android.support.annotation.Nullable;
import android.util.Log;
import android.view.accessibility.AccessibilityEvent;
import android.view.accessibility.AccessibilityNodeInfo;


/**
 * @author benrandall76 AT gmail DOT com
 */

public class MyAccessibilityService extends AccessibilityService {

    private final boolean DEBUG = true;
    private final String CLS_NAME = MyAccessibilityService.class.getSimpleName();

    private static final String GOOGLE_VOICE_SEARCH_PACKAGE_NAME = "com.google.android.googlequicksearchbox";
    private static final String GOOGLE_VOICE_SEARCH_INTERIM_FIELD = "com.google.android.apps.gsa.searchplate.widget.StreamingTextView";
    private static final String GOOGLE_VOICE_SEARCH_FINAL_FIELD = "com.google.android.apps.gsa.searchplate.SearchPlate";

    private static final long COMMAND_UPDATE_DELAY = 1000L;

    private long previousCommandTime;
    private String previousCommand = null;

    private final boolean EXTRA_VERBOSE = false;

    @Override
    protected void onServiceConnected() {
        super.onServiceConnected();
        if (DEBUG) {
            Log.i(CLS_NAME, "onServiceConnected");
        }
    }

    @Override
    public void onAccessibilityEvent(final AccessibilityEvent event) {
        if (DEBUG) {
            Log.i(CLS_NAME, "onAccessibilityEvent");
        }

        if (event != null) {

            switch (event.getEventType()) {

                case AccessibilityEvent.TYPE_WINDOW_CONTENT_CHANGED:
                    if (DEBUG) {
                        Log.i(CLS_NAME, "onAccessibilityEvent: checking for google");
                    }

                    if (event.getPackageName() != null && event.getPackageName().toString().matches(
                            GOOGLE_VOICE_SEARCH_PACKAGE_NAME)) {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "onAccessibilityEvent: checking for google: true");
                            Log.i(CLS_NAME, "onAccessibilityEvent: event.getPackageName: " + event.getPackageName());
                            Log.i(CLS_NAME, "onAccessibilityEvent: event.getClassName: " + event.getClassName());
                        }

                        final AccessibilityNodeInfo source = event.getSource();

                        if (source != null && source.getClassName() != null) {

                            if (source.getClassName().toString().matches(
                                    GOOGLE_VOICE_SEARCH_INTERIM_FIELD)) {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: className interim: true");
                                    Log.i(CLS_NAME, "onAccessibilityEvent: source.getClassName: " + source.getClassName());
                                }

                                if (source.getText() != null) {

                                    final String text = source.getText().toString();
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: interim text: " + text);
                                    }

                                    if (interimMatch(text)) {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: child: interim match: true");
                                        }

                                        if (commandDelaySufficient(event.getEventTime())) {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: true");
                                            }

                                            if (!commandPreviousMatches(text)) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: false");
                                                }

                                                previousCommandTime = event.getEventTime();
                                                previousCommand = text;

                                                killGoogle();

                                                if (DEBUG) {
                                                    Log.e(CLS_NAME, "onAccessibilityEvent: INTERIM PROCESSING: " + text);
                                                }

                                            } else {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: true");
                                                }
                                            }
                                        } else {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: false");
                                            }
                                        }
                                        break;
                                    } else {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: child: interim match: false");
                                        }
                                    }
                                } else {
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: interim text: null");
                                    }
                                }
                            } else if (source.getClassName().toString().matches(
                                    GOOGLE_VOICE_SEARCH_FINAL_FIELD)) {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: className final: true");
                                    Log.i(CLS_NAME, "onAccessibilityEvent: source.getClassName: " + source.getClassName());
                                }

                                final int childCount = source.getChildCount();
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: childCount: " + childCount);
                                }

                                if (childCount > 0) {
                                    for (int i = 0; i < childCount; i++) {

                                        final String text = examineChild(source.getChild(i));

                                        if (text != null) {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: child text: " + text);
                                            }

                                            if (finalMatch(text)) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: child: final match: true");
                                                }

                                                if (commandDelaySufficient(event.getEventTime())) {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: true");
                                                    }

                                                    if (!commandPreviousMatches(text)) {
                                                        if (DEBUG) {
                                                            Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: false");
                                                        }

                                                        previousCommandTime = event.getEventTime();
                                                        previousCommand = text;

                                                        killGoogle();

                                                        if (DEBUG) {
                                                            Log.e(CLS_NAME, "onAccessibilityEvent: FINAL PROCESSING: " + text);
                                                        }

                                                    } else {
                                                        if (DEBUG) {
                                                            Log.i(CLS_NAME, "onAccessibilityEvent: commandPreviousMatches: true");
                                                        }
                                                    }
                                                } else {
                                                    if (DEBUG) {
                                                        Log.i(CLS_NAME, "onAccessibilityEvent: commandDelaySufficient: false");
                                                    }
                                                }
                                                break;
                                            } else {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: child: final match: false");
                                                }
                                            }
                                        } else {
                                            if (DEBUG) {
                                                Log.i(CLS_NAME, "onAccessibilityEvent: child text: null");
                                            }
                                        }
                                    }
                                }
                            } else {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "onAccessibilityEvent: className: unwanted " + source.getClassName());
                                }

                                if (EXTRA_VERBOSE) {

                                    if (source.getText() != null) {

                                        final String text = source.getText().toString();
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: unwanted text: " + text);
                                        }
                                    } else {
                                        if (DEBUG) {
                                            Log.i(CLS_NAME, "onAccessibilityEvent: unwanted text: null");
                                        }
                                    }

                                    final int childCount = source.getChildCount();
                                    if (DEBUG) {
                                        Log.i(CLS_NAME, "onAccessibilityEvent: unwanted childCount: " + childCount);
                                    }

                                    if (childCount > 0) {

                                        for (int i = 0; i < childCount; i++) {

                                            final String text = examineChild(source.getChild(i));

                                            if (text != null) {
                                                if (DEBUG) {
                                                    Log.i(CLS_NAME, "onAccessibilityEvent: unwanted child text: " + text);
                                                }
                                            }
                                        }
                                    }
                                }
                            }
                        } else {
                            if (DEBUG) {
                                Log.i(CLS_NAME, "onAccessibilityEvent: source null");
                            }
                        }
                    } else {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "onAccessibilityEvent: checking for google: false");
                        }
                    }
                    break;
                default:
                    if (DEBUG) {
                        Log.i(CLS_NAME, "onAccessibilityEvent: not interested in type");
                    }
                    break;
            }
        } else {
            if (DEBUG) {
                Log.i(CLS_NAME, "onAccessibilityEvent: event null");
            }
        }
    }

    /**
     * Check if the previous command was actioned within the {@link #COMMAND_UPDATE_DELAY}
     *
     * @param currentTime the time of the current {@link AccessibilityEvent}
     * @return true if the delay is sufficient to proceed, false otherwise
     */
    private boolean commandDelaySufficient(final long currentTime) {
        if (DEBUG) {
            Log.i(CLS_NAME, "commandDelaySufficient");
        }

        final long delay = (currentTime - COMMAND_UPDATE_DELAY);

        if (DEBUG) {
            Log.i(CLS_NAME, "commandDelaySufficient: delay: " + delay);
            Log.i(CLS_NAME, "commandDelaySufficient: previousCommandTime: " + previousCommandTime);
        }

        return delay > previousCommandTime;
    }

    /**
     * Check if the previous command/text matches the current text we are considering processing
     *
     * @param text the current text
     * @return true if the text matches the previous text we processed, false otherwise.
     */
    private boolean commandPreviousMatches(@NonNull final String text) {
        if (DEBUG) {
            Log.i(CLS_NAME, "commandPreviousMatches");
        }

        return previousCommand != null && previousCommand.matches(text);
    }

    /**
     * Check if the interim text matches a command we want to intercept
     *
     * @param text the intercepted text
     * @return true if the text matches a command false otherwise
     */
    private boolean interimMatch(@NonNull final String text) {
        if (DEBUG) {
            Log.i(CLS_NAME, "interimMatch");
        }
        return text.matches("do interim results work");
    }

    /**
     * Check if the final text matches a command we want to intercept
     *
     * @param text the intercepted text
     * @return true if the text matches a command false otherwise
     */
    private boolean finalMatch(@NonNull final String text) {
        if (DEBUG) {
            Log.i(CLS_NAME, "finalMatch");
        }

        return text.matches("do final results work");
    }

    /**
     * Recursively examine the {@link AccessibilityNodeInfo} object
     *
     * @param parent the {@link AccessibilityNodeInfo} parent object
     * @return the extracted text or null if no text was contained in the child objects
     */
    private String examineChild(@Nullable final AccessibilityNodeInfo parent) {
        if (DEBUG) {
            Log.i(CLS_NAME, "examineChild");
        }

        if (parent != null) {

            for (int i = 0; i < parent.getChildCount(); i++) {

                final AccessibilityNodeInfo nodeInfo = parent.getChild(i);

                if (nodeInfo != null) {
                    if (DEBUG) {
                        Log.i(CLS_NAME, "examineChild: nodeInfo: getClassName: " + nodeInfo.getClassName());
                    }

                    if (nodeInfo.getText() != null) {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: have text: returning: " + nodeInfo.getText().toString());
                        }
                        return nodeInfo.getText().toString();
                    } else {
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: text: null: recurse");
                        }

                        final int childCount = nodeInfo.getChildCount();
                        if (DEBUG) {
                            Log.i(CLS_NAME, "examineChild: childCount: " + childCount);
                        }

                        if (childCount > 0) {

                            final String text = examineChild(nodeInfo);

                            if (text != null) {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "examineChild: have recursive text: returning: " + text);
                                }
                                return text;
                            } else {
                                if (DEBUG) {
                                    Log.i(CLS_NAME, "examineChild: recursive text: null");
                                }
                            }
                        }
                    }
                } else {
                    if (DEBUG) {
                        Log.i(CLS_NAME, "examineChild: nodeInfo null");
                    }
                }
            }
        } else {
            if (DEBUG) {
                Log.i(CLS_NAME, "examineChild: parent null");
            }
        }

        return null;
    }

    /**
     * Kill or reset Google
     */
    private void killGoogle() {
        if (DEBUG) {
            Log.i(CLS_NAME, "killGoogle");
        }

        // TODO - Either kill the Google process or send an empty intent to clear current search process
    }

    @Override
    public void onInterrupt() {
        if (DEBUG) {
            Log.i(CLS_NAME, "onInterrupt");
        }
    }

    @Override
    public void onDestroy() {
        super.onDestroy();
        if (DEBUG) {
            Log.i(CLS_NAME, "onDestroy");
        }
    }
}

我把 class 写得尽可能冗长和缩进，希望它更容易理解。

它执行以下操作：

检查事件类型是否正确
检查包裹是否是Google的'Now'
检查硬编码 class 类型的节点信息
检查加载到视图中的临时语音命令
检查加载到视图中的最终语音命令
递归检查语音命令的视图
检查事件之间的时间差
检查语音命令是否与之前检测到的相同

测试：

在Android辅助功能设置

Service

您的应用程序可能需要重新启动才能正确注册服务
开始 Google 语音识别并说“做中期结果工作”
退出Google现在
开始识别并说“做最终结果工作”

以上内容将演示从两个硬编码视图中提取的 text/command。如果不重新启动 Google 现在，该命令仍将被检测为临时命令。

使用提取的语音命令，需要进行自己的语言匹配，判断这是否是你感兴趣的命令，如果是，需要阻止Google说出或显示结果.这是通过立即终止 Google 或向其发送一个空的语音搜索意图来实现的，其中包含应该 clear/reset task.

的标志

这样做你会处于竞争状态，所以你的语言处理需要非常聪明，或者非常基本....

希望对您有所帮助。

编辑：

对于那些提问的人，'kill' Google 现在，您要么需要有终止进程的权限，要么发送一个空的 ("")清除当前搜索的搜索意图：

public static final String PACKAGE_NAME_GOOGLE_NOW = "com.google.android.googlequicksearchbox";
public static final String ACTIVITY_GOOGLE_NOW_SEARCH = ".SearchActivity";

/**
 * Launch Google Now with a specific search term to resolve
 *
 * @param ctx        the application context
 * @param searchTerm the search term to resolve
 * @return true if the search term was handled correctly, false otherwise
 */
public static boolean googleNow(@NonNull final Context ctx, @NonNull final String searchTerm) {
    if (DEBUG) {
        Log.i(CLS_NAME, "googleNow");
    }

    final Intent intent = new Intent(Intent.ACTION_WEB_SEARCH);
    intent.setComponent(new ComponentName(PACKAGE_NAME_GOOGLE_NOW,
            PACKAGE_NAME_GOOGLE_NOW + ACTIVITY_GOOGLE_NOW_SEARCH));

    intent.putExtra(SearchManager.QUERY, searchTerm);
    intent.setFlags(Intent.FLAG_ACTIVITY_NEW_TASK | Intent.FLAG_ACTIVITY_CLEAR_TOP
            | Intent.FLAG_ACTIVITY_RESET_TASK_IF_NEEDED);

    try {
        ctx.startActivity(intent);
        return true;
    } catch (final ActivityNotFoundException e) {
        if (DEBUG) {
            Log.e(CLS_NAME, "googleNow: ActivityNotFoundException");
            e.printStackTrace();
        }
    } catch (final Exception e) {
        if (DEBUG) {
            Log.e(CLS_NAME, "googleNow: Exception");
            e.printStackTrace();
        }
    }

    return false;

}

Google 现在的自定义命令

Custom commands for Google Now

java

android

google-voice

google-now

google-voice-search