java - mongodb分片集群下,count和聚合統(tǒng)計(jì)問(wèn)題
問(wèn)題描述
在mongodb分片集群下,直接用count統(tǒng)計(jì)會(huì)不準(zhǔn)確,用聚合統(tǒng)計(jì)則可以@Testpublic void testCount() throws Exception { DynamicSqlParameter dsp = new DynamicSqlParameter(); long sT = System.currentTimeMillis(); MongoDatasource mongoDatasource = MongoDatasource.getInstance(mongoService.getDatasource()); DBCollection dbCollection = mongoDatasource.getDB().getCollection('dayFlow'); List arrayList = new ArrayList<>(); DBObject dbObject1 = new BasicDBObject(); dbObject1.put('usedDayFlow', 2); DBObject dbObject2 = new BasicDBObject(); dbObject2.put('_id', null); dbObject2.put('count', new BasicDBObject('$sum', 1)); arrayList.add(new BasicDBObject('$match', dbObject1)); arrayList.add(new BasicDBObject('$group', dbObject2)); System.out.println(JSON.serialize(arrayList)); AggregationOutput size = dbCollection.aggregate(arrayList); System.out.println(size.results()); System.out.println('運(yùn)行時(shí)間:' + ((System.currentTimeMillis() - sT) /1000) + 's');}執(zhí)行結(jié)果:
[ { '$match' : { 'usedDayFlow' : 2}} , { '$group' : { '_id' : null , 'count' : { '$sum' : 1}}}]
[{ '_id' : null , 'count' : 1002223}]
該統(tǒng)計(jì)結(jié)果比實(shí)際數(shù)據(jù)量要多一些,請(qǐng)教大神,對(duì)于分片集群的聚合統(tǒng)計(jì)要如何操作?問(wèn)題解答
回答1: 該問(wèn)題已經(jīng)解決,使用的是最新驅(qū)動(dòng)mongo-java-driver-3.4.0,通過(guò)下面的方法可以在分片集群模式下,準(zhǔn)確的統(tǒng)計(jì)到記錄數(shù)量,感謝大家的相助!mongo shell >> db.collection.aggregate([{$match:{categories:'Bakery'},{$group:{'_id':null,'count':{$sum:1}}}}])public long getCount() {String user = '用戶名';String database = 'admin';String password = '密碼';MongoCredential credential = MongoCredential.createCredential(user,database, password.toCharArray());MongoClientOptions options = MongoClientOptions.builder().connectionsPerHost(10).threadsAllowedToBlockForConnectionMultiplier(10).socketTimeout(20000).connectTimeout(15000).maxWaitTime(50000).build();MongoClient mongoClient = new MongoClient(new ServerAddress('IP地址', '端口'), Arrays.asList(credential), options);MongoDatabase mongoDatabase = mongoClient.getDatabase('數(shù)據(jù)庫(kù)');MongoCollection<Document> collection = mongoDatabase.getCollection('數(shù)據(jù)表');final long[] count = new long[1];Block<Document> printBlock = new Block<Document>() { @Override public void apply(final Document document) { count[0] = (long) document.get('count'); }};Bson bson = Filters.eq('categories', 'Bakery');collection.aggregate(Arrays.asList(Aggregates.match(bson),Aggregates.group(null, Accumulators.sum('count', 1L)))).forEach(printBlock);return count[0];}回答2:
能否補(bǔ)充一些評(píng)論中的信息。多謝!
將評(píng)論的內(nèi)容轉(zhuǎn)發(fā)在這里,方便查看:
1、count和aggregate的不同:在mongoDB中,count和aggregate是在兩支不同的程序中實(shí)現(xiàn)的,aggregate的實(shí)現(xiàn)是考慮到了shard的環(huán)境的,所以官方文檔是推薦使用aggregate來(lái)進(jìn)行shard環(huán)境下的count。
2、MongoDB shell下使用aggregate和使用Java MongoDB驅(qū)動(dòng)使用aggregate來(lái)進(jìn)行count,結(jié)果應(yīng)該是一樣的,因?yàn)閮烧叨际鞘褂玫腶ggregate。
您提到Issue大概是MongoDB shell和Java MongoDB驅(qū)動(dòng)進(jìn)行count的結(jié)果不一致。
這種不一致,我覺(jué)得可能是:
1)比較的過(guò)程有沒(méi)有紕漏;2)所用的Java MongoDB驅(qū)動(dòng)是否有紕漏。
供參考。
Love MongoDB! Have Fun!
今晚8點(diǎn),MongoDB中文社區(qū)大神在線講座,請(qǐng)大家踴躍參與;此大神常駐本版哦!
請(qǐng)戳此鏈接。
