SlideShare a Scribd company logo
1 of 67
Indexing with
   MongoDB
        Luke Ehresman
CopperEgg - www.copperegg.com
What is an index?
What is an index?

• Pointers to documents
What is an index?

• Pointers to documents
• Efficiently organized for quick scanning
What is an index?

• Pointers to documents
• Efficiently organized for quick scanning
• Maintained in a tree structure
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
How do we find x = 6 without an index?




{x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}




                 6 documents scanned
How do we find x = 6 with an index?
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}
How do we find x = 6 with an index?

                        {x:4}
                <                 >



        {x:2}                             {x:6}
    <           >                     <           >


{x:1}           {x:3}           {x:5}             {x:7}


    Only 2 documents scanned
In reality, it’s even better than that...
In reality, it’s even better than that...


                                    {x:0, name:‘George’, ....}
                                    {x:1, name:‘Jane’, ....}
                                    {x:2, name:‘Judy’, ....}
                                    {x:3, name:‘Elroy’, ....}
                                    {x:4, name:‘Astro’, ....}
                                    {x:5, name:‘Rosie’, ....}
                                    {x:6, name:‘Fred’, ....}
                                    {x:7, name:‘Wilma’, ....}
                                    {x:8, name:‘Barney’, ....}
                                    {x:9, name:‘Betty’, ....}
In reality, it’s even better than that...


                        {x:4}                             {x:0, name:‘George’, ....}
                                                          {x:1, name:‘Jane’, ....}
                <                 >                       {x:2, name:‘Judy’, ....}
                                                          {x:3, name:‘Elroy’, ....}
                                                          {x:4, name:‘Astro’, ....}
                                                          {x:5, name:‘Rosie’, ....}
        {x:2}                             {x:6}           {x:6, name:‘Fred’, ....}
                                                          {x:7, name:‘Wilma’, ....}
    <           >                     <           >       {x:8, name:‘Barney’, ....}
                                                          {x:9, name:‘Betty’, ....}

{x:1}           {x:3}           {x:5}             {x:7}
In reality, it’s still even better than that...
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
•The full index does not need to be in memory
In reality, it’s still even better than that...

•MongoDB stores data and indexes in extents
•Only loads extents into memory as needed
•The full index does not need to be in memory
                            {x:4}
                    <                 >



            {x:2}                             {x:6}
        <           >                     <           >


    {x:1}           {x:3}           {x:5}             {x:7}
When to index?
When to index?

• Frequently queried fields
When to index?

• Frequently queried fields
• Low response time
When to index?

• Frequently queried fields
• Low response time
• Sorting
When to index?

• Frequently queried fields
• Low response time
• Sorting
• Avoid full collection scans
Things to know...
Things to know...
• Indexes maintain order
Things to know...
• Indexes maintain order
• Indexes will slow writes
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
• Index maintenance will cause writes
Things to know...
• Indexes maintain order
• Indexes will slow writes
• Indexes take up space
• Index maintenance will cause writes
• “unique” parameter forces uniqueness
How to create an index
How to create an index
    db.people.ensureIndex({...})
How to create an index
        db.people.ensureIndex({...})

{_id:1} is already created for each collection
How to create an index
            db.people.ensureIndex({...})

   {_id:1} is already created for each collection

      You can call ensureIndex multiple times
(it only creates an index if it doesn’t already exist)
How to create an index
            db.people.ensureIndex({...})

   {_id:1} is already created for each collection

      You can call ensureIndex multiple times
(it only creates an index if it doesn’t already exist)

     Blocks your mongod process unless you
             specify background:true.
Indexes on missing fields
Indexes on missing fields

null is assumed if a field is missing
Indexes on missing fields

         null is assumed if a field is missing

db.people.insert({name:‘Fred’, children:[‘Pebbles’]});
db.people.insert({name:‘Pebbles’})

db.people.ensureIndex({children:1})

db.people.find({children:null});
(returns Pebbles and uses the index)
Indexes on missing fields

         null is assumed if a field is missing

db.people.insert({name:‘Fred’, children:[‘Pebbles’]});
db.people.insert({name:‘Pebbles’})

db.people.ensureIndex({children:1})

db.people.find({children:null});
(returns Pebbles and uses the index)

         Optional sparse indexes exclude
           empty fields for efficiency
Compound indexes
Compound indexes

{
    name: “George”,
    age:45,
    gender:”M”,
    ...
}
Compound indexes

{
    name: “George”,        {gender:1, age:-1}
    age:45,
    gender:”M”,
    ...
}
Compound indexes

{
    name: “George”,          {gender:1, age:-1}
    age:45,
    gender:”M”,
    ...
}                         gender:”M”
                            age:45
                                              {name:‘George’, ....}
                            age:12
                                              {name:‘Jane’, ....}
                            age:3
                                              {name:‘Judy’, ....}
                          gender:”F”
                                              {name:‘Elroy’, ....}
                            age:44
                                              {name:‘Astro’, ....}
                            age:16
                                              {name:‘Rosie’, ....}
                            age:10
Indexes on arrays
Indexes on arrays
{
    name: “George”,
    age:45,
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Indexes on arrays
{
    name: “George”,
    age:45,
                                  {children:1}
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Indexes on arrays
{
    name: “George”,
    age:45,
                                  {children:1}
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
                                  Actually creates a new entry
                                     in the index for each
                                     element of the array.
Indexes on arrays
 {
     name: “George”,
     age:45,
                                   {children:1}
     gender:”M”,
     children: [‘Judy’, ‘Elroy’]
     ...
 }
                                   Actually creates a new entry
                                      in the index for each
                                      element of the array.
children:‘Judy’                         {name:‘George’, ....}
children:‘Elroy’                        {name:‘Fred’, ...}
children:‘Bam-Bam’                      {name:‘Barney’ ...}
Covered indexes
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)


{
    name: “George”,
    age:45,
    gender:”M”,
    children: [‘Judy’, ‘Elroy’]
    ...
}
Covered indexes
Queries that can be resolved with only the index
(does not need to fetch the original document)


{
    name: “George”,
    age:45,                       db.people.ensureIndex({name:1, age:1});
    gender:”M”,                   db.people.find({name:‘George’}, {_id:0, age:1});
    children: [‘Judy’, ‘Elroy’]
    ...
}
How does it know which index to use?
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});

 scan


 index on name


 index on age
How does it know which index to use?

db.people.ensureIndex({name:1});
db.people.ensureIndex({age:1});

db.people.find({age:{$gte:21}});

 scan

                            terminated
 index on name


 index on age         remember
Query performance analysis


db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BasicCursor”,
    “indexBounds” : {},
    “nscanned” : 88731,
    “nscannedObjects” : 88731,
    “n” : 1,
    “millis” : 108
}
Query performance analysis


db.people.ensureIndex({name:1})
db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BtreeCursor name_1”,
    ...
}
Query performance analysis


db.people.ensureIndex({name:1})
db.people.find({name:‘Fred’}).explain()

{
    “cursor” : “BtreeCursor name_1”,
    ...
}



                     Demo
Credits
• Much of this information was gleaned from
  these two presentations:
• http://speakerdeck.com/u/mongodb/p/
  indexing-and-query-optimizer-kevin-
  hanson-10gen
• http://www.slideshare.net/mongodb/
  indexing-with-mongodb

More Related Content

What's hot

Coding Horrors
Coding HorrorsCoding Horrors
Coding HorrorsMark Baker
 
Php code for online quiz
Php code for online quizPhp code for online quiz
Php code for online quizhnyb1002
 
Improving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENImproving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENMike Hugo
 
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingGareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingYury Chemerkin
 
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...MongoDB
 
Real World CouchDB
Real World CouchDBReal World CouchDB
Real World CouchDBJohn Wood
 
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB
 
Airline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEAirline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEHimanshiSingh71
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Javaantoinegirbal
 
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB
 
Strongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasStrongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasNorberto Leite
 
Webinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasWebinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasMongoDB
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDBrogerbodamer
 
ETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBMongoDB
 
MongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMike Friedman
 
Dev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesDev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesMongoDB
 

What's hot (17)

Coding Horrors
Coding HorrorsCoding Horrors
Coding Horrors
 
Php code for online quiz
Php code for online quizPhp code for online quiz
Php code for online quiz
 
Improving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIRENImproving RDF Search Performance with Lucene and SIREN
Improving RDF Search Performance with Lucene and SIREN
 
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzingGareth hayes. non alphanumeric javascript-php and shared fuzzing
Gareth hayes. non alphanumeric javascript-php and shared fuzzing
 
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
How to Avoid Common Data Visualization Pitfalls and Being Led Astray By Your ...
 
Real World CouchDB
Real World CouchDBReal World CouchDB
Real World CouchDB
 
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local Munich 2019: Tips and Tricks++ for Querying and Indexing MongoDB
 
Airline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDEAirline reservation project using JAVA in NetBeans IDE
Airline reservation project using JAVA in NetBeans IDE
 
Building a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and JavaBuilding a Scalable Inbox System with MongoDB and Java
Building a Scalable Inbox System with MongoDB and Java
 
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDBMongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
MongoDB .local London 2019: Tips and Tricks++ for Querying and Indexing MongoDB
 
MongoDB With Style
MongoDB With StyleMongoDB With Style
MongoDB With Style
 
Strongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible SchemasStrongly Typed Languages and Flexible Schemas
Strongly Typed Languages and Flexible Schemas
 
Webinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible SchemasWebinar: Strongly Typed Languages and Flexible Schemas
Webinar: Strongly Typed Languages and Flexible Schemas
 
Schema Design with MongoDB
Schema Design with MongoDBSchema Design with MongoDB
Schema Design with MongoDB
 
ETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDBETL for Pros: Getting Data Into MongoDB
ETL for Pros: Getting Data Into MongoDB
 
MongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World ExamplesMongoDB Schema Design: Four Real-World Examples
MongoDB Schema Design: Four Real-World Examples
 
Dev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best PracticesDev Jumpstart: Schema Design Best Practices
Dev Jumpstart: Schema Design Best Practices
 

Viewers also liked

Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)MongoDB
 
Indexing with MongoDB
Indexing with MongoDBIndexing with MongoDB
Indexing with MongoDBMongoDB
 
Performance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYPerformance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYMongoDB
 
MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329Douglas Duncan
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance TuningMongoDB
 
Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)MongoSF
 
MongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB
 
MongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB
 
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]Filip Tepper
 
Geo-Indexing w/MongoDB
Geo-Indexing w/MongoDBGeo-Indexing w/MongoDB
Geo-Indexing w/MongoDBLalit Kapoor
 
User Data Management with MongoDB
User Data Management with MongoDB User Data Management with MongoDB
User Data Management with MongoDB MongoDB
 
Webinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationWebinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationMongoDB
 
Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)MongoDB
 
MongoDB Performance Tuning
MongoDB Performance TuningMongoDB Performance Tuning
MongoDB Performance TuningMongoDB
 
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB
 
Webinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBWebinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBMongoDB
 
Optimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at LocalyticsOptimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at Localyticsandrew311
 

Viewers also liked (19)

Fast querying indexing for performance (4)
Fast querying   indexing for performance (4)Fast querying   indexing for performance (4)
Fast querying indexing for performance (4)
 
Indexing with MongoDB
Indexing with MongoDBIndexing with MongoDB
Indexing with MongoDB
 
Performance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LYPerformance Tuning on the Fly at CMP.LY
Performance Tuning on the Fly at CMP.LY
 
MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329MongoDB and Indexes - MUG Denver - 20160329
MongoDB and Indexes - MUG Denver - 20160329
 
Indexing and Performance Tuning
Indexing and Performance TuningIndexing and Performance Tuning
Indexing and Performance Tuning
 
Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)Indexing and Query Optimizer (Aaron Staple)
Indexing and Query Optimizer (Aaron Staple)
 
MongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance TuningMongoDB Days UK: Indexing and Performance Tuning
MongoDB Days UK: Indexing and Performance Tuning
 
MongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative SchemasMongoDB Indexing Constraints and Creative Schemas
MongoDB Indexing Constraints and Creative Schemas
 
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
NoSQL i dlaczego go nie potrzebujesz? [OlCamp]
 
Geo-Indexing w/MongoDB
Geo-Indexing w/MongoDBGeo-Indexing w/MongoDB
Geo-Indexing w/MongoDB
 
User Data Management with MongoDB
User Data Management with MongoDB User Data Management with MongoDB
User Data Management with MongoDB
 
Webinar: Index Tuning and Evaluation
Webinar: Index Tuning and EvaluationWebinar: Index Tuning and Evaluation
Webinar: Index Tuning and Evaluation
 
Phplx mongodb
Phplx mongodbPhplx mongodb
Phplx mongodb
 
Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)Indexing and Query Optimizer (Richard Kreuter)
Indexing and Query Optimizer (Richard Kreuter)
 
Indexing
IndexingIndexing
Indexing
 
MongoDB Performance Tuning
MongoDB Performance TuningMongoDB Performance Tuning
MongoDB Performance Tuning
 
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
MongoDB Analytics: Learn Aggregation by Example - Exploratory Analytics and V...
 
Webinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDBWebinar: Working with Graph Data in MongoDB
Webinar: Working with Graph Data in MongoDB
 
Optimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at LocalyticsOptimizing MongoDB: Lessons Learned at Localytics
Optimizing MongoDB: Lessons Learned at Localytics
 

Similar to MongoDB Indexing Guide

Data Representation - Day 2
Data Representation - Day 2Data Representation - Day 2
Data Representation - Day 2blprnt
 
Processing & Dataviz
Processing & DatavizProcessing & Dataviz
Processing & Datavizblprnt
 
JSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitJSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitXavier Badosa
 
Haskell in the Real World
Haskell in the Real WorldHaskell in the Real World
Haskell in the Real Worldosfameron
 
Thinking Functionally In Ruby
Thinking Functionally In RubyThinking Functionally In Ruby
Thinking Functionally In RubyRoss Lawley
 

Similar to MongoDB Indexing Guide (6)

Data Representation - Day 2
Data Representation - Day 2Data Representation - Day 2
Data Representation - Day 2
 
Processing & Dataviz
Processing & DatavizProcessing & Dataviz
Processing & Dataviz
 
JSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript ToolkitJSON-stat & JS: the JSON-stat Javascript Toolkit
JSON-stat & JS: the JSON-stat Javascript Toolkit
 
Haskell in the Real World
Haskell in the Real WorldHaskell in the Real World
Haskell in the Real World
 
Trees
TreesTrees
Trees
 
Thinking Functionally In Ruby
Thinking Functionally In RubyThinking Functionally In Ruby
Thinking Functionally In Ruby
 

Recently uploaded

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024Lonnie McRorey
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkPixlogix Infotech
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...itnewsafrica
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxLoriGlavin3
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfIngrid Airi González
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...itnewsafrica
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfLoriGlavin3
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfNeo4j
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesThousandEyes
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPathCommunity
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxLoriGlavin3
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsRavi Sanghani
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructureitnewsafrica
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Kaya Weers
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationKnoldus Inc.
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxLoriGlavin3
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observabilityitnewsafrica
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxLoriGlavin3
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxLoriGlavin3
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integrationmarketing932765
 

Recently uploaded (20)

TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024TeamStation AI System Report LATAM IT Salaries 2024
TeamStation AI System Report LATAM IT Salaries 2024
 
React Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App FrameworkReact Native vs Ionic - The Best Mobile App Framework
React Native vs Ionic - The Best Mobile App Framework
 
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...Zeshan Sattar- Assessing the skill requirements and industry expectations for...
Zeshan Sattar- Assessing the skill requirements and industry expectations for...
 
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptxMerck Moving Beyond Passwords: FIDO Paris Seminar.pptx
Merck Moving Beyond Passwords: FIDO Paris Seminar.pptx
 
Generative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdfGenerative Artificial Intelligence: How generative AI works.pdf
Generative Artificial Intelligence: How generative AI works.pdf
 
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...Abdul Kader Baba- Managing Cybersecurity Risks  and Compliance Requirements i...
Abdul Kader Baba- Managing Cybersecurity Risks and Compliance Requirements i...
 
Moving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdfMoving Beyond Passwords: FIDO Paris Seminar.pdf
Moving Beyond Passwords: FIDO Paris Seminar.pdf
 
Connecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdfConnecting the Dots for Information Discovery.pdf
Connecting the Dots for Information Discovery.pdf
 
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyesHow to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
How to Effectively Monitor SD-WAN and SASE Environments with ThousandEyes
 
UiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to HeroUiPath Community: Communication Mining from Zero to Hero
UiPath Community: Communication Mining from Zero to Hero
 
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptxUse of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
Use of FIDO in the Payments and Identity Landscape: FIDO Paris Seminar.pptx
 
Potential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and InsightsPotential of AI (Generative AI) in Business: Learnings and Insights
Potential of AI (Generative AI) in Business: Learnings and Insights
 
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical InfrastructureVarsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
Varsha Sewlal- Cyber Attacks on Critical Critical Infrastructure
 
Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)Design pattern talk by Kaya Weers - 2024 (v2)
Design pattern talk by Kaya Weers - 2024 (v2)
 
Data governance with Unity Catalog Presentation
Data governance with Unity Catalog PresentationData governance with Unity Catalog Presentation
Data governance with Unity Catalog Presentation
 
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptxDigital Identity is Under Attack: FIDO Paris Seminar.pptx
Digital Identity is Under Attack: FIDO Paris Seminar.pptx
 
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security ObservabilityGlenn Lazarus- Why Your Observability Strategy Needs Security Observability
Glenn Lazarus- Why Your Observability Strategy Needs Security Observability
 
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptxThe Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
The Fit for Passkeys for Employee and Consumer Sign-ins: FIDO Paris Seminar.pptx
 
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptxPasskey Providers and Enabling Portability: FIDO Paris Seminar.pptx
Passkey Providers and Enabling Portability: FIDO Paris Seminar.pptx
 
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS:  6 Ways to Automate Your Data IntegrationBridging Between CAD & GIS:  6 Ways to Automate Your Data Integration
Bridging Between CAD & GIS: 6 Ways to Automate Your Data Integration
 

MongoDB Indexing Guide

  • 1. Indexing with MongoDB Luke Ehresman CopperEgg - www.copperegg.com
  • 2. What is an index?
  • 3. What is an index? • Pointers to documents
  • 4. What is an index? • Pointers to documents • Efficiently organized for quick scanning
  • 5. What is an index? • Pointers to documents • Efficiently organized for quick scanning • Maintained in a tree structure
  • 6. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 7. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 8. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 9. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 10. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 11. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 12. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 13. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9}
  • 14. How do we find x = 6 without an index? {x:0} {x:1} {x:2} {x:3} {x:4} {x:5} {x:6} {x:7} {x:8} {x:9} 6 documents scanned
  • 15. How do we find x = 6 with an index?
  • 16. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 17. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 18. How do we find x = 6 with an index? {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7} Only 2 documents scanned
  • 19. In reality, it’s even better than that...
  • 20. In reality, it’s even better than that... {x:0, name:‘George’, ....} {x:1, name:‘Jane’, ....} {x:2, name:‘Judy’, ....} {x:3, name:‘Elroy’, ....} {x:4, name:‘Astro’, ....} {x:5, name:‘Rosie’, ....} {x:6, name:‘Fred’, ....} {x:7, name:‘Wilma’, ....} {x:8, name:‘Barney’, ....} {x:9, name:‘Betty’, ....}
  • 21. In reality, it’s even better than that... {x:4} {x:0, name:‘George’, ....} {x:1, name:‘Jane’, ....} < > {x:2, name:‘Judy’, ....} {x:3, name:‘Elroy’, ....} {x:4, name:‘Astro’, ....} {x:5, name:‘Rosie’, ....} {x:2} {x:6} {x:6, name:‘Fred’, ....} {x:7, name:‘Wilma’, ....} < > < > {x:8, name:‘Barney’, ....} {x:9, name:‘Betty’, ....} {x:1} {x:3} {x:5} {x:7}
  • 22. In reality, it’s still even better than that...
  • 23. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents
  • 24. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed
  • 25. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed •The full index does not need to be in memory
  • 26. In reality, it’s still even better than that... •MongoDB stores data and indexes in extents •Only loads extents into memory as needed •The full index does not need to be in memory {x:4} < > {x:2} {x:6} < > < > {x:1} {x:3} {x:5} {x:7}
  • 28. When to index? • Frequently queried fields
  • 29. When to index? • Frequently queried fields • Low response time
  • 30. When to index? • Frequently queried fields • Low response time • Sorting
  • 31. When to index? • Frequently queried fields • Low response time • Sorting • Avoid full collection scans
  • 33. Things to know... • Indexes maintain order
  • 34. Things to know... • Indexes maintain order • Indexes will slow writes
  • 35. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space
  • 36. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space • Index maintenance will cause writes
  • 37. Things to know... • Indexes maintain order • Indexes will slow writes • Indexes take up space • Index maintenance will cause writes • “unique” parameter forces uniqueness
  • 38. How to create an index
  • 39. How to create an index db.people.ensureIndex({...})
  • 40. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection
  • 41. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection You can call ensureIndex multiple times (it only creates an index if it doesn’t already exist)
  • 42. How to create an index db.people.ensureIndex({...}) {_id:1} is already created for each collection You can call ensureIndex multiple times (it only creates an index if it doesn’t already exist) Blocks your mongod process unless you specify background:true.
  • 44. Indexes on missing fields null is assumed if a field is missing
  • 45. Indexes on missing fields null is assumed if a field is missing db.people.insert({name:‘Fred’, children:[‘Pebbles’]}); db.people.insert({name:‘Pebbles’}) db.people.ensureIndex({children:1}) db.people.find({children:null}); (returns Pebbles and uses the index)
  • 46. Indexes on missing fields null is assumed if a field is missing db.people.insert({name:‘Fred’, children:[‘Pebbles’]}); db.people.insert({name:‘Pebbles’}) db.people.ensureIndex({children:1}) db.people.find({children:null}); (returns Pebbles and uses the index) Optional sparse indexes exclude empty fields for efficiency
  • 48. Compound indexes { name: “George”, age:45, gender:”M”, ... }
  • 49. Compound indexes { name: “George”, {gender:1, age:-1} age:45, gender:”M”, ... }
  • 50. Compound indexes { name: “George”, {gender:1, age:-1} age:45, gender:”M”, ... } gender:”M” age:45 {name:‘George’, ....} age:12 {name:‘Jane’, ....} age:3 {name:‘Judy’, ....} gender:”F” {name:‘Elroy’, ....} age:44 {name:‘Astro’, ....} age:16 {name:‘Rosie’, ....} age:10
  • 52. Indexes on arrays { name: “George”, age:45, gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 53. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 54. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... } Actually creates a new entry in the index for each element of the array.
  • 55. Indexes on arrays { name: “George”, age:45, {children:1} gender:”M”, children: [‘Judy’, ‘Elroy’] ... } Actually creates a new entry in the index for each element of the array. children:‘Judy’ {name:‘George’, ....} children:‘Elroy’ {name:‘Fred’, ...} children:‘Bam-Bam’ {name:‘Barney’ ...}
  • 57. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document)
  • 58. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document) { name: “George”, age:45, gender:”M”, children: [‘Judy’, ‘Elroy’] ... }
  • 59. Covered indexes Queries that can be resolved with only the index (does not need to fetch the original document) { name: “George”, age:45, db.people.ensureIndex({name:1, age:1}); gender:”M”, db.people.find({name:‘George’}, {_id:0, age:1}); children: [‘Judy’, ‘Elroy’] ... }
  • 60. How does it know which index to use?
  • 61. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}});
  • 62. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}}); scan index on name index on age
  • 63. How does it know which index to use? db.people.ensureIndex({name:1}); db.people.ensureIndex({age:1}); db.people.find({age:{$gte:21}}); scan terminated index on name index on age remember
  • 64. Query performance analysis db.people.find({name:‘Fred’}).explain() { “cursor” : “BasicCursor”, “indexBounds” : {}, “nscanned” : 88731, “nscannedObjects” : 88731, “n” : 1, “millis” : 108 }
  • 67. Credits • Much of this information was gleaned from these two presentations: • http://speakerdeck.com/u/mongodb/p/ indexing-and-query-optimizer-kevin- hanson-10gen • http://www.slideshare.net/mongodb/ indexing-with-mongodb

Editor's Notes

  1. \n
  2. \n
  3. \n
  4. \n
  5. \n
  6. \n
  7. \n
  8. \n
  9. \n
  10. \n
  11. \n
  12. \n
  13. \n
  14. \n
  15. \n
  16. \n
  17. \n
  18. \n
  19. \n
  20. \n
  21. \n
  22. \n
  23. \n
  24. \n
  25. \n
  26. \n
  27. \n
  28. \n
  29. \n
  30. \n
  31. \n
  32. \n
  33. \n
  34. \n
  35. \n
  36. \n
  37. \n
  38. \n
  39. \n
  40. \n
  41. \n
  42. \n
  43. \n
  44. \n
  45. \n
  46. \n
  47. \n
  48. \n
  49. \n
  50. \n
  51. \n
  52. \n
  53. \n
  54. \n