使用 HTMLunit 登录重定向到谷歌验证码

Question

我正在尝试使用 HTMLunit 2.31 登录 https://www.jvzoo.com/。但它显示 google 验证码。但如果我尝试通过浏览器登录，它工作正常。我已经添加了请求所需的所有参数。

代码片段：

   public static void main(String[] args) {

    String url = "https://www.jvzoo.com/auth/login";
    WebClient webClient = new WebClient(BrowserVersion.CHROME);
    webClient.getOptions().setJavaScriptEnabled(false);
    webClient.getOptions().setCssEnabled(false);
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    try {
        WebRequest webRequest = new WebRequest(new URL(url), HttpMethod.POST);
             webRequest.setAdditionalHeader("Accept", "ext/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
        webRequest.setAdditionalHeader("Accept-Encoding", "gzip, deflate, br");
        webRequest.setAdditionalHeader("Accept-Language", "en-US,en;q=0.5");

        webRequest.setAdditionalHeader("Connection", "keep-alive");
        webRequest.setAdditionalHeader("Content-Type", "application/x-www-form-urlencoded");

        webRequest.setAdditionalHeader("Host", "www.jvzoo.com");
        webRequest.setAdditionalHeader("Referer", "https://www.jvzoo.com/auth/login");
        webRequest.setAdditionalHeader("userAgent", "Mozilla/5.0 (Windows NT 6.2; r…) Gecko/20100101 Firefox/61.0");
        webRequest.setAdditionalHeader("Upgrade-Insecure-Requests", "1");

        Date date = new Date();
        SimpleDateFormat smt = new SimpleDateFormat("yyyy-M-d HH:mm:ss");
        String format = smt.format(date);

        webRequest.setAdditionalHeader("Cookie", "__cfduid=d042d9103fb79ed922e5ac17dac50f45a1532320289;_ga=GA1.2.1331177163.1532322753"
                + ";_gid=GA1.2.810982780.1532322753;announcement_cookie=" + format + ";cookieconsent_status=allow"
                + ";csrf_protected=64c6852ecf1bc1720e3e83cafc1b4618;"
                + "jvzsession=6lAUaU8wUPAbdXF9alRljftFv3aytU05vTVRm22yNrWvZNsjxsyYNfGEQF2xx4irKm/DALWU/pQZ1fRgjv7HwQ==");
        List<NameValuePair> params = new ArrayList<NameValuePair>();
        params.add(new NameValuePair("csrf_protected", "64c6852ecf1bc1720e3e83cafc1b4618"));
        params.add(new NameValuePair("forward", ""));
        params.add(new NameValuePair("login", username));
        params.add(new NameValuePair("password", password));
        params.add(new NameValuePair("remember", "1"));
        webRequest.setRequestParameters(params);

        Page page = webClient.getPage(webRequest);
        WebResponse webResponse = page.getWebResponse();

        HtmlPage htmlpage = HTMLParser.parseHtml(webResponse, webClient.getCurrentWindow());
        System.out.println("doc:" + htmlpage.asXml());
        System.out.println("Getting dashboard..");
        //HtmlPage page1 = webClient.getPage("https://www.jvzoo.com/dashboard");
        //System.out.println("doc:" + page1.asXml());

    } catch (IOException ex) {
        Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
    } catch (FailingHttpStatusCodeException ex) {
        Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
    }
}

如果有人能帮助我，我会很高兴。

Answer 1

恐怕您完全没有理解 HtmlUnit API 的要点。请查看 HtmlUnit Getting Started 页面上的第二个示例（提交表单）。通常不需要构建自己的网络请求，也绝对不需要自己解析响应。

作为第二个建议，请不要禁用 javascript；仅当您真的确定不需要它时才这样做。

如果这没有帮助，您可以尝试通过邮件向我发送测试凭据，我会看看。

使用 HTMLunit 登录重定向到谷歌验证码

Login using HTMLunit redirects to goolge capcha

java

login

web-scraping

htmlunit