What you should know on MongoDb Indexes

 

32MB Sort Limit in MongoDB

 MongoDb stores the search result in memory before sort. It has a 32MB memory limit. 

The workaround is to create an index on the search field.

Implementing Search feature using Text Index in MongoDb

Let's say we have 1,000,000 documents with below data schema:

[{

name : "Simon",

comment: "This is awesome article. Max also pretty good on this.",

score: 29.52711235568646,

age: 40

}, 

{

name : "Max"

comment: "You are an awesome person.",

score:47.148964115304345,

age: 72

}]

We want to implement a search feature where user can search the name and comment. The search weight of the name is higher than the comment. For example, when the user searches for Max on the above 2 documents, the search result will return both documents. However, the document with Max name will show first.

MongoDb has "Text Search" index that performs a text search of string content. The index can easily create using below command:

db.ratings.createIndex( 

{ name: "text", comment: "text" },

{ weights: { name: 10, comment: 5 }} 

)

When creating the index, the collection will be locked, you can't find, update, etc on the collection. You can create the index on the background without lock the collection using the below command. Creating an index background is slower.

db.ratings.createIndex( 

{ name: "text", comment: "text" },

{ weights: { name: 10, comment: 5 }},

{ background: true }

)

Let search Max using the below query:

db.ratings.find( { $text: { $search: "Max" } } )

To display the search score and sort based on the search score, you can use below query:

db.ratings.find( 

{ $text: { $search: "Max" } }, 

{ searchScore: { $meta: "textScore" }} 

).sort( 

{ searchScore: { $meta: "textScore" }} 

).pretty()

Here is the result:

[{

name : "Max"

comment: "You are an awesome person.",

score:47.148964115304345,

age: 72,

searchScore: 11

},{

name : "Simon",

comment: "This is awesome article. Max also pretty good on this.",

score: 29.52711235568646,

age: 40,

searchScore: 5.5

}]

References:

https://docs.mongodb.com/manual/core/index-text/

https://docs.mongodb.com/manual/text-search/

How to update an array in MongoDb using $ Operator?

 Let's say we have below dataset:

[{

name : "Simon"

hobbies : [

{ title : "badminton", frequency : 2 } ,

{ title : "Swimming", frequency : 1 } 

]

}, 

{

name : "Max"

hobbies : [

{ title : "badminton", frequency : 1 } ,

{ title : "Swimming", frequency : 2 } 

]

},]

$ is a placeholder to update the first element that matches the query condition in the array. For example, below change Simon's hobby from Badminton to Tennis and frequency from 2 to 5. 

db.hobby.updateMany(

    {"hobbies" : {$elemMatch: {title : "badminton", frequency : 2}}}, 

    {$set : {"hobbies.$" : {title : "Tennis", frequency : 5 }}}

);

You can use $[].<fieldname> to update all element in the array. For example, add expert into all hobbies.

db.hobby.updateMany(

    {}, 

    {$set : {"hobbies.$[].expert" : true }}

);

You can add a new element into the array by adding $.<fieldname>. For example, adding expert new field:

db.hobby.updateMany(

    {"hobbies" : {$elemMatch: {title : "Tennis", frequency : 5}}}, 

    {$set : {"hobbies.$.expert" : true }}

);

Reference:
https://docs.mongodb.com/manual/reference/operator/update/

How to find exact embedded documents using $elemMatch?

 Let's say we have below dataset:

[{

name : "Simon"

hobbies : [

{ title : "badminton", frequency : 2 } ,

{ title : "Swimming", frequency : 1 } 

]

}, 

{

name : "Max"

hobbies : [

{ title : "badminton", frequency : 1 } ,

{ title : "Swimming", frequency : 2 } 

]

},]

Our goal is to find a person who likes badminton and 2 badminton playings frequently.

If you using the below query, it will return Simon and Max.

db.people.find({"hobbies .title":"badminton","hobbies.frequency ":2});

db.people.find({$and: [{"hobbies .title":"badminton"},{"hobbies .frequency ":2}]})

Max returned in the search result because he has a badminton hobby and frequency 2 in swimming.

To search for the exact person who likes badminton and 2 badminton playing frequently, you can use $elemMatch operator.

db.people.find({"hobbies":{$elemMatch: {"title":"badminton","frequency ":2}}})

When we need to use $And in MongoDb Query?

 By default, MongoDb has the $and operator in find query. 

For example of this:

db.products.find({$and: [{name:"Mazda 3"}, {manufacturer:"Mazda"}]});

You can shortcut to:

db.products.find({name:"Mazda 3", manufacturer:"Mazda"});

So why still $and operator?

MongoDb builds using Javascript engine, when you search the same fields, the last search criteria will overwrite all. For example below

db.products.find({parts:"Seats", parts:"Doors"});

is equivalent to

db.products.find({parts:"Doors"});

The correct way to search the same fields is using $and operator.

db.products.find({$and:[{parts:"Seats"}, {parts:"Doors"}]});

How to avoid save conflict in mongoose?

Save conflict is when 2 persons are saving the same document. For example, Person A and Person B open the same document. Person A saves the document, followed by Person B, then Person A's data over-write by Person B.

To avoid this, you can use either version check or document lock mechanism.

Version Check:

By default, when you save the document using mongoose, mongoose saves the version number in __v field. We can use the field to do version check to avoid save conflict.

Person A opens and edits the version 1 document. Meantime, Person B also opens the edit version 1 document.

Person A success save the document using below query:

posts.update({ _id: postId, __v: versionNumber } , { $set: { 'comments': updatedText }})

When Person B try to save the document using the same query, the save will be failed because of the wrong version number.

Document Lock:

Locking when Person A edit the document, saved a lock flag to the document.

When person B tries to edit the same document, the system will prompt saying the document is currently edit (lock) by Person A.

Bonus:

If you don’t want to use versioning in your schema you can disable it by passing false for the versionKey option. 

schema.set('versionKey', false);

What if the ref document deleted from the backend in Mongoose's Populate Ref?

Mongoose's populate joins 2 collections by using the ref. 

What if the ref document deleted from the backend?

You still can see the ref when you searching using MongoDb driver and MongoDb Compass. However, you won't see the ref when you search using mongoose.

update() vs updateMany()

To avoid accidents replace all fields in many documents, I would suggest do not use update(). Use below shell methods instead:

replaceOne() : replace 1 document.

updateOne() : update 1 document on specific fields.

updateMany(): update many document on specific fields.

Good Example:

db.flight.updateOne({depart:"KLIA"}, {$set:{arrivaltime:"3pm"}})

db.flight.updateMany({depart:"KLIA"}, {$set:{arrivaltime:"3pm"}})

db.flight.replaceOne({depart:"KLIA"}, {arrivaltime:"3pm"})

Bad Example:

db.flight.update({depart:"KLIA"}, {arrivaltime:"3pm"})