Using MongoDB with Python Applications

Introduction

MongoDB can be considered as, or feels just like, working with a Python dictionary data type. The only subtlety is that you gain access to using what I call SQL commands or prompts to interact with the storage system. This enables operations like CRUD to be performed on the database.

We can imagine the storage file as being a JSON file that holds all the data, except that we connect to the database. The reason for this is that MongoDB is a NoSQL database. This makes it differ from the typical SQL databases as they have properties like relational data.

SQL databases are designed to work with queries, and NoSQL databases are designed to be flexible with what they can work with. No SQL Databases are more suited for modern web applications as they store and bring back data without having to create obfuscated queries.

While MongoDB doesn’t use a lot of the SQL syntax, there are some similarities. Here is a table with all the terminologies used and compared against each other.

SQL vs NoSQL Terminology

SQL Term	MongoDB Term	Description
Database	Database	A container that holds collections and data.
Table	Collection	A group of related records/documents.
Row	Document	A single record stored as a JSON-like object.
Column	Field	A key-value pair within a document.
Primary Key	_id field	A unique identifier automatically added to each document.
Join	Embedding / Linking	Ways of relating documents to each other.
Group By	Aggregation	Operations that summarise or transform data.
Index	Index	Improves query performance by enabling faster lookups.

MongoDB Operators

As we are using Python to execute queries to perform CRUD operations, we’ll need to get familiar with some of the operators that are available. I had already worked with Database Design and Development during my studies and so this wasn’t all new to me. Which helped me learn them all a little quicker.

Comparison Operators

Syntax pattern:

{'field': {'$operator': value}}

Operator	Meaning	Example Query
$gt	Greater than	{'age': {'$gt': 18}}
$lt	Less than	{'age': {'$lt': 65}}
$eq	Equal to	{'grade': {'$eq': 'A'}}
$ne	Not equal to	{'grade': {'$ne': 'F'}}

Example:

# Find students older than 25
query = {'age': {'$gt': 25}}
results = db.students.find(query)
for student in results:
    print(student)

---

Logical Operators

Syntax pattern:

{'$operator': [condition1, condition2]}

Operator	Purpose	Example Query
$and	All conditions must match	{'$and': [{'age': {'$gt': 18}}, {'grade': 'B'}]}
$or	Any condition can match	{'$or': [{'grade': 'A'}, {'grade': 'C'}]}
$in	Match values in a list	{'grade': {'$in': ['A', 'B']}}
$nin	Exclude values in a list	{'grade': {'$nin': ['D', 'F']}}

Example:

# Find students with grade A or B
query = {'grade': {'$in': ['A', 'B']}}
for student in db.students.find(query):
    print(student)

---

Update Operators

Update operators define how a document should be modified.

Syntax pattern:

{'$operator': {'field': value}}

Operator	Purpose	Example
$set	Set or update a field	{'$set': {'city': 'Manchester'}}
$inc	Increment a numeric value	{'$inc': {'age': 1}}
$push	Add an item to a list field	{'$push': {'scores': 95}}
$pull	Remove an item from a list field	{'$pull': {'scores': 70}}

Example:

# Increase age by 1
db.students.update_one(
    {'name': 'Sheikh Hussain'},
    {'$inc': {'age': 1}}
)

Prerequisites

Before we begin, let’s go on ahead and install some packages that are required for this all to work.

pip install pymongo
pip install dnspython
pip install flask

The reason why we install Flask is because I will later begin working with creating a web application using Python and MongoDB.

Create MongoDB Account

One of the requirements, laid out in the course, is that we create an account on MongoDB, which then gives us access to a cloud database platform to use. This will create a repository for our project that houses all the databases and it’s contents for use.

I don’t want to include all the steps in doing this as there are online tutorials available but this is what I followed:

• Create an account on MongoDB Atlas
• Create a project
• Create a database user (username and password)
• Create a cluster (this hosts your databases)
• Copy the connection string: “mongodb+srv://:@.mongodb.net/?appName=PythonApp"

Optional:

• Install the MongoDB VS Code extension
• Use the connection string to connect VS Code to the cluster (This allows you to view databases and collections directly in VS Code)

Creating Our Python File

In VS Code, we can open a normal Python file to begin with and start importing some of the installed packages to work with.

import pymongo # This is for the database connections
import flask # This is for building the web application

Connecting Our Database

Now, we have a Python file with all the imports and the MongoDB database ready too. We just need to connect them and start using.

Syntax Pattern

client = MongoClient(MONGODB_URI)
db = client['database_name']
collection = db['collection_name']

Example

URI = "mongodb+srv://<username>:<password>@<cluster>.mongodb.net/?appName=PythonApp"
client = pymongo.MongoClient(URI)
client = pymongo.MongoClient(MONGODB_URI)
db = client['thirty_days_of_python']
students = db['students']

This has now brought the database as db and the table/collection as students from that database. Now, lets perform som operations and start adding, deleting, updating, and reading information from it.

Hint: When inserting documents or rows into our database, MongoDB automatically assigns a unique ID as _id field for each row. You can have it custom made but that’s something I’ll learn later.

Inserting Documents

Syntax Patterns

collection.insert_one(document)
collection.insert_many([document1, document2])

Examples

students.insert_one({
    'name': 'Sheikh Hussain',
    'age': 40,
    'grade': 'C'
})

students.insert_many([
    {'name': 'Gabe Doe', 'age': 28, 'grade': 'B'},
    {'name': 'David Smith', 'age': 34, 'grade': 'A'}
])

This will now insert one or many rows into the table we have specified.

Finding Documents or Rows

Syntax Patterns

collection.find(query)
collection.find_one(query)

A query can be specified with the parenthesis or as a separate entity. You can create a variable called ‘query’ and then have your actual query enclosed within it and then use the variable as a parameter/argument in your function which it will then use.

query = { 'example':'query'}
collection.find(query)

Examples

# Find all students
for student in students.find():
    print(student)

# Find a single student
student = students.find_one({'name': 'Sheikh Hussain'})
print(student)

# Return specific fields only
for student in students.find({}, {'_id': 0, 'name': 1}):
    print(student)

# Limit results
for student in students.find().limit(5):
    print(student)

Using Logical Operators

I wanted to also include how we would use logical operators in our queries to narrow down our results using it. This will help when it comes to filtering or using data.

Using $and

The $and operator returns documents only if all conditions are true.

Syntax Pattern

{
  '$and': [
    condition_1,
    condition_2
  ]
}

Example

# Find students who are older than 25 AND have a grade of 'C'
query = {
    '$and': [
        {'age': {'$gt': 25}},
        {'grade': 'C'}
    ]
}

results = db.students.find(query)
for student in results:
    print(student)

Using $or

The $or operator returns documents if at least one condition is true.

Syntax Pattern

{
  '$or': [
    condition_1,
    condition_2
  ]
}

Example

# Find students who have grade 'A' OR grade 'B'
query = {
    '$or': [
        {'grade': 'A'},
        {'grade': 'B'}
    ]
}

results = db.students.find(query)
for student in results:
    print(student)

I won’t go through all of them, this will just help build the picture for us all.

Sorting Results

We may or may not use this as when working with databases we already have complete flexibility in terms of what we can do with it.

Syntax pattern:

collection.find().sort(field, order)

# Ascending order
for student in students.find().sort('name'):
    print(student)

# Descending order
for student in students.find().sort('name', -1):
    print(student)

Updating Documents

This is used to update a field or row within a table. It would be a good idea to also know that we can store a list type as data. For example, if we have a ‘grade’ field and need to store the results of more than one grade in there, this is possible.

Syntax Pattern

collection.update_one(filter, update)
collection.update_many(filter, update)

Example

db.students.update_one(
    {'name': 'David Smith'},
    {'$set': {'grade': 'A+'}}
)

Deleting Documents and Collections

This is how we delete a row or table from our database.

Syntax Patterns

collection.delete_one(filter)
collection.delete_many(filter)
collection.drop()

Examples

# Delete one document
db.students.delete_one({'name': 'John'})

# Delete multiple documents
db.students.delete_many({'grade': 'F'})

# Drop entire collection
db.students.drop()

Conclusions

Using MongoDB has been surprisingly easy. I’ll guess that’s because I’ve already used SQL databases, which have given me some of the knowledge that I need to work with MongoDB. What spiked my curiosity was the way we can build queries in Python to then perform operations on our database.

As I know, I can create many functions and methods with Python already. So it’s giving me the initiative to develop applications or functions with Python that are completely custom-made, from taking inputs as queries from a user to then bring back information from pre-built queries.

Overall, this topic has been insightful to say the least.