Java HtmlUnit Ajax 调用永远不会在 HtmlPage 中呈现

Question

我正在尝试使用 HtmlUnit 2.31 扫描页面。我只是通过 URL 获得了一个 HtmlPage。但是，页面似乎启动了一些 AJAX 调用（无用户交互）。我需要等待调用完成并成为 "rendered"，这样我才能看到这些值。这是我的代码：

    public static void main( String[] args ) throws IOException, InterruptedException {
        WebClient webClient = new WebClient(BrowserVersion.CHROME);
        webClient.getOptions().setJavaScriptEnabled(true);
        webClient.getOptions().setThrowExceptionOnScriptError(false);
        webClient.getOptions().setCssEnabled(false);
        webClient.setAjaxController(new NicelyResynchronizingAjaxController());
        webClient.setJavaScriptTimeout(10000);

        String itemsForSalePageURL = "https://miami-dade.realtaxdeed.com/index.cfm?zaction=AUCTION&Zmethod=PREVIEW&AUCTIONDATE=07/24/2018";
        HtmlPage previewItemsForSalePage = webClient.getPage(itemsForSalePageURL);

        final JavaScriptJobManager tmpJobManager = previewItemsForSalePage.getEnclosingWindow().getJobManager();
        while(tmpJobManager.getJobCount()>0){
            System.out.println("javascript is still running...");
        }

        boolean doneLoading = false;
        while(!doneLoading){
            System.out.println("checking if page is done loading...");
            List<HtmlDivision> loadingElements = previewItemsForSalePage.getByXPath("//div[@class = 'Loading']");
            if(loadingElements.isEmpty()){
                doneLoading = true;
            }
        }

        String contents = previewItemsForSalePage.asXml();

        System.out.println( "Run complete." );
    }

问题是：while(!doneLoading) 循环变成了无限循环。我的 xpath 正在搜索的类除了 "loading" 之外永远不会求值。

我感兴趣的一个领域是：

        <div id="Area_C" class="Auct_Area" ref="Y" arid="C">
          <div tabindex="0" class="Loading"></div>
        </div>

在 Chrome 中，计算结果为

<div id="Area_C" class="Auct_Area" ref="N" arid="C">
    <div tabindex="0" id="AITEM_1159704" class="AUCTION_ITEM PREVIEW adc-spacer" aid="1159704" rem="0" isset="1" data-test="check"><div tabindex="0"><div tabindex="0" class="adc-tab"><div tabindex="-1" class="banner-left-side"></div>This item has <strong>complimentary</strong> photos, valuation reports and title information.<div tabindex="-1" class="banner-right-side"></div></div><div tabindex="0" class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per Bankruptcy</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div tabindex="0" class="AUCTION_DETAILS adc"> <table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2008-069895-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">7,016.30</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=0420360070830" onclick="return showExitPopup();" target="_blank">04-2036-007-0830</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">618 W 65 DR</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">HIALEAH, FL- 33012</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">1,793.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">9,450.00</td></tr></tbody></table></div></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1160530" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1160530" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Sold</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">07/24/2018 09:07 AM ET</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL">Amount</div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA">8,200.00</div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL">Sold To</div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA">3rd Party Bidder</div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2012-007563-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">3,600.66</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=3069350010170" onclick="return showExitPopup();" target="_blank">30-6935-001-0170</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">26711 SW 133 CT</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">MIAMI, FL- 33032</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">4,378.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">0,539.81</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1156810" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1156810" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per County</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2013-006996-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">,060,190.67</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=3350330550030" onclick="return showExitPopup();" target="_blank">33-5033-055-0030</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">17321 SW 93 AVE</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">PALMETTO BAY, FL- 33157</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">6,458.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">Hidden</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1150473" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1150473" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per Order</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2013-009405-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">1,175.75</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=0821280140050" onclick="return showExitPopup();" target="_blank">08-2128-014-0050</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">13247 ASWAN RD</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">OPA-LOCKA, FL- 33054</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">,602.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">Hidden</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1159431" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1159431" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Sold</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">07/24/2018 09:14 AM ET</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL">Amount</div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA">0,200.00</div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL">Sold To</div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA">3rd Party Bidder</div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2014-012973-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">5,004.82</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=0141210071790" onclick="return showExitPopup();" target="_blank">01-4121-007-1790</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">3321 FROW AVE</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">MIAMI, FL- 33133</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">9,959.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">7,550.00</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1145542" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1145542" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per Bankruptcy</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2014-020812-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">4,642.58</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=3050290190190" onclick="return showExitPopup();" target="_blank">30-5029-019-0190</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">15900 SW 105 AVE</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">MIAMI, FL- 33157</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">3,877.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">Hidden</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1144806" class="AUCTION_ITEM PREVIEW adc-spacer" aid="1144806" rem="0" isset="1" data-test="check"><div tabindex="0"><div tabindex="0" class="adc-tab"><div tabindex="-1" class="banner-left-side"></div>This item has <strong>complimentary</strong> photos, valuation reports and title information.<div tabindex="-1" class="banner-right-side"></div></div><div tabindex="0" class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per Order</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div tabindex="0" class="AUCTION_DETAILS adc"> <table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2015-002717-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">0,378.48</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=3020030381200" onclick="return showExitPopup();" target="_blank">30-2003-038-1200</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">19522 NW 77 CT</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">HIALEAH , FL- 33015</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">3,509.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">Hidden</td></tr></tbody></table></div></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1155118" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1155118" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Sold</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">07/24/2018 09:20 AM ET</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL">Amount</div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA">3,100.00</div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL">Sold To</div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA">3rd Party Bidder</div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2015-004160-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">0,255.11</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=3049090040080" onclick="return showExitPopup();" target="_blank">30-4909-004-0080</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">15495 SW 13 TER</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">MIAMI, FL- 33194</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">7,671.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">9,375.00</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1155459" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1155459" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per Order</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2015-005595-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">7,737.00</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=0140010210019" onclick="return showExitPopup();" target="_blank">01-4001-021-0019</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">6261 TAMIAMI CANAL RD</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">MIAMI, FL- 33126</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">3,080.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">Hidden</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div><div tabindex="0" id="AITEM_1151163" aria-label="Auction Details" class="AUCTION_ITEM PREVIEW" aid="1151163" rem="0" isset="1"><div class="AUCTION_STATS"><div tabindex="0" class="ASTAT_MSGA ASTAT_LBL">Auction Status</div><div tabindex="0" class="ASTAT_MSGB Astat_DATA">Canceled per Order</div><div tabindex="0" class="ASTAT_MSGC ASTAT_LBL"></div> <div tabindex="0" class="ASTAT_MSGD Astat_DATA"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_Label ASTAT_LBL"></div><div tabindex="0" class="ASTAT_MSG_SOLDTO_MSG Astat_DATA"></div></div><div class="AUCTION_DETAILS"><table class="ad_tab" tabindex="0"><tbody><tr><th class="AD_LBL" scope="row">Auction Type:</th><td class="AD_DTA">FORECLOSURE</td></tr><tr><th class="AD_LBL" scope="row" aria-label="Case Number">Case #:</th><td class="AD_DTA"> 2015-022277-CA-01</td></tr><tr><th class="AD_LBL" scope="row">Final Judgment Amount:</th><td class="AD_DTA">4,062.43</td></tr><tr><th class="AD_LBL" scope="row">Parcel ID:</th><td class="AD_DTA"> <a href="http://www.miamidade.gov/propertysearch/#/?folio=3040280070030" onclick="return showExitPopup();" target="_blank">30-4028-007-0030</a></td></tr><tr><th class="AD_LBL" scope="row">Property Address:</th><td class="AD_DTA">6020 SW 92 AVE</td></tr><tr><th class="AD_LBL" scope="row"></th><td class="AD_DTA">MIAMI, FL- 33173</td></tr> <tr><th class="AD_LBL" scope="row">Assessed Value:</th><td class="AD_DTA">8,500.00</td></tr><tr><th class="AD_LBL" scope="row">Plaintiff max bid:</th><td class="AD_DTA ASTAT_MSGPB">Hidden</td></tr></tbody></table></div> </div><div class="AUCTION_ITEM_SPACER">&nbsp;</div> 
</div>

目前我不确定还可以尝试什么。我试过更改浏览器版本。我试过增加 JavaScriptTimeout。我尝试使用 NicelyResynchronizingAjaxController。我什至等到所有 javascript 处理完毕。我想我缺少一些基本的理解。有人可以告诉我吗。

Answer 1

快速检查了一下，这是 jQuery 1.6.1 的问题（参见 issue 1835）。因为这只发生在 jQuery 的最小化版本中，所以很难修复（最后必须在 Rhino 中完成）。欢迎任何帮助....

Answer 2

解决方法来了

String url = "https://miami-dade.realtaxdeed.com/index.cfm?zaction=AUCTION&Zmethod=PREVIEW&AUCTIONDATE=07/24/2018";

try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_60)) {
    webClient.setWebConnection(new FalsifyingWebConnection(webClient) {
        @Override
        public WebResponse getResponse(WebRequest webRequest) throws IOException {
            if (webRequest.getUrl().getPath().endsWith("jquery-1.6.1.min.js")) {
                String jQuery = FileUtils.readFileToString(new File("jquery-1.6.1.js"));
                return createWebResponse(webRequest, jQuery, "application/javascript");
            }
            return super.getResponse(webRequest);
        }
    });

    HtmlPage previewItemsForSalePage = webClient.getPage(url);
    webClient.waitForBackgroundJavaScript(1000);

    System.out.println("----------------");
    System.out.println(previewItemsForSalePage.asText());
}

还有一些注意事项：

通过使用 FalsifyingWebConnection，您可以根据需要拦截和操纵请求和响应
小心这个没有超时的 while 语句，许多页面都在使用某种心跳；你永远不会停止

至少在这里我得到了与真实浏览器相同的内容。

Java HtmlUnit Ajax 调用永远不会在 HtmlPage 中呈现

Java HtmlUnit Ajax call never get rendered in the HtmlPage

java

ajax

screen-scraping

htmlunit