mongodb

aggregation-framework

left-join

sharding

$lookup is new in MongoDB 3.2. It performs a left outer join to an unsharded collection in the same database to filter in documents from the “joined” collection for processing.

To use $lookup, the from collection cannot be sharded.

On the other hand, sharding is a useful horizontal scaling approach.

What's the best practise to use them together?

Solution 1

As the docs you quote indicate, you can't use $lookup on a sharded collection. So the best practice workaround is to perform the lookup yourself in a separate query.

  1. Perform your aggregate query.
  2. Pull the "localField" values from your query results into an array, possibly using Array#map.
  3. Perform a find query against the "from" collection, using a query like {foreignField: {$in: localFieldArray}}
  4. Merge your results into whatever format you need.

Don't let the $lookup limitation stop you from sharding collections that require it for scalability, just perform the lookup function yourself.

Solution 2

As mentioned in MongoDb document "In the $lookup stage, the from collection cannot be sharded. However, the collection on which you run the aggregate() method can be sharded" https://docs.mongodb.com/manual/reference/operator/aggregation/lookup/

db.shardedCollection.aggregate([
   { $lookup: { from: "unshardedCollection", ... } }
])

This is the best practise to use them together