Mongodb cursor.toArray() 太慢了

Question

我正在使用 cursor.toArray() 到 return 我的 collection.find(query) 作为列表，我的 API 的响应时间为 100 毫秒。提取到游标中的数据非常少（几百条记录），数据库在我查询的字段上建立了索引。我还设置了批量大小 cursor.batchSize(1000).

db.collection.find({"{ "ZIP" : { "$in" : [ "12345"]}}"}) 是我的查询，我的数据库在 'ZIP' 上建立了索引。我可以在 4 毫秒内在 shell 上看到相同的查询运行。

mongo shell 上的相同查询几乎不需要 5 毫秒。

我使用的Mogo驱动程序是：

<!-- https://mvnrepository.com/artifact/org.mongojack/mongojack -->
<dependency>
    <groupId>org.mongojack</groupId>
    <artifactId>mongojack</artifactId>
    <version>2.8.2</version>
</dependency>

密码

@Path("/")
@Produces(MediaType.APPLICATION_JSON)
@Api(value = "listing-mongo")
public class MlsMongoResource {

    private JacksonDBCollection<Mlsdatadao, String> collection;
    Clock clock;

    public MlsMongoResource(JacksonDBCollection<Mlsdatadao, String> collection) {
        this.collection = collection;
        this.clock =  Clock.systemUTC();
    }

    @GET
    @Path("/listings-mongo")
    @Produces(value = MediaType.APPLICATION_JSON)
    @Timed
    public List<Mlsdatadao> getListings(@BeanParam MlsListingParameters mlsBeanParam) {

        BasicDBList basicDbList = new BasicDBList();
        mlsBeanParam.validateBean();

        setLocations(basicDbList,mlsBeanParam.zipcodes);

        BasicDBObject query = new BasicDBObject("$and", basicDbList);

        DBCursor<Mlsdatadao> cursor = null;

        long start = 0;
        try{
             start = System.currentTimeMillis();
            cursor = collection.find(query);
            cursor.batchSize(1000);

        } catch (Exception e){
            System.out.println("IN collection.find() " +  e.getCause());
        }


        System.out.println("QUERY LIST IS " + basicDbList);

        if(cursor == null) {
            System.out.println("Cursor is null");
        }
        List<Mlsdatadao> result = cursor.toArray();
       cursor.close();(System.currentTimeMillis() - start));
        return result;

    }



    private void setLocations(BasicDBList basicDbList, List<String> zipcodes) {

        if (CollectionUtils.isNotEmpty(zipcodes)) {
            basicDbList.add(setZipcodes(zipcodes));
        }

    }

    private BasicDBObject setZipcodes(List<String> zipcodes) {
        return new BasicDBObject("ZIP" ,  new BasicDBObject("$in", zipcodes) );
    }
}

申请：

public class MongoApplication extends Application <MlsMongoConfiguration> {

    public static void main(String[] args) throws Exception {
        new MlsMongoApplication().run(args);
    }

    @Override
    public String getName() {
        return "mls-dropwizard-mongo";
    }

    @Override
    public void initialize(Bootstrap<MlsMongoConfiguration> bootstrap) {

        bootstrap.addBundle(new SwaggerBundle<MlsMongoConfiguration>() {
            @Override
            protected SwaggerBundleConfiguration getSwaggerBundleConfiguration(MlsMongoConfiguration configuration) {
                return configuration.swaggerBundleConfiguration;
            }
        });

    }

    @Override
    public void run(MlsMongoConfiguration configuration, Environment environment) throws Exception {


        MongoClientOptions.Builder clientOptions = new MongoClientOptions.Builder();
        clientOptions.minConnectionsPerHost(1000);//min
        clientOptions.maxWaitTime(1000);
        clientOptions.connectionsPerHost(1000);

        //Create Mongo instance
        //Mongo mongo = new Mongo(configuration.mongohost, configuration.mongoport);

        MongoClient mongoClient = new MongoClient(new ServerAddress(configuration.mongohost, configuration.mongoport), clientOptions.build());

        //Add Managed for managing the Mongo instance
        //MongoManaged mongoManaged = new MongoManaged(mongo);
        MongoManaged mongoManaged = new MongoManaged(mongoClient);
        environment.lifecycle().manage(mongoManaged);

        //Add Health check for Mongo instance. This will be used from the Health check admin page
        environment.healthChecks().register("MongoHealthCheck", new MongoHealthCheck(mongoClient));
        //Create DB instance and wrap it in a Jackson DB collection
        DB db = mongoClient.getDB(configuration.mongodb);
        JacksonDBCollection<Mlsdatadao, String> jacksonDBCollection = JacksonDBCollection.wrap(db.getCollection("mlsdata"), Mlsdatadao.class, String.class);
        environment.jersey().register(new MlsMongoResource(jacksonDBCollection));
    }
}

有什么办法可以避免cursor.toArray()？任何其他性能调整提示都会非常有用。

谢谢。

Answer 1

将我的 MongoDB 驱动程序从 mongojack 更改为本机 mongo-java-driver 3.7 并使用 com.mongodb.client.FindIterable 而不是 DBCursor 后，一切看起来不错。看起来 Mongojack 库花费了大量时间将 BSON 对象映射到 POJO。

Mongodb cursor.toArray() 太慢了

Mongodb cursor.toArray() is too slow

mongodb

mongojack