🔎 QuerySets and Model Managers
Django's ORM lets you read and write your database in Python instead of SQL. At its heart are two objects: the manager (your entry point, Model.objects) and the QuerySet (a lazy, chainable, cacheable collection of rows). Master these and the whole ORM opens up.
🎯 Learning Objectives
By the end of this lesson, you will be able to:
- Explain how QuerySets are lazy, chainable, and cached — and when a query actually hits the database
- Filter data with field lookups, relationship traversal, and Q objects
- Aggregate and annotate data, and use
F()expressions for field-to-field math - Write custom managers and QuerySets to keep query logic DRY
- Eliminate the N+1 query problem with
select_relatedandprefetch_related
Estimated Time: 50–65 minutes • Difficulty: Intermediate
Hands-on: Build a custom, chainable QuerySet for a blog and optimize a list view.
In This Lesson
Managers & QuerySets
Every Django model gets a manager named objects. The manager is the doorway to the database for that model — you call methods on it (Post.objects.all(), Post.objects.filter(...)) and it hands back a QuerySet: a collection of rows you can filter, order, and iterate.
The crucial insight is that a QuerySet is not a list of results. It's a description of a query that hasn't run yet. You build it up method by method, and Django only sends SQL to the database at the last possible moment.
💡 A useful analogy: The manager is a librarian, and a QuerySet is your written request slip. You can keep adding conditions to the slip — "published books, by this author, newest first" — without the librarian moving an inch. Only when you actually ask to read the results does the librarian walk the stacks once and bring back everything at once.
Post] --> B[Manager
Post.objects] B --> C[QuerySet
.filter.order_by] C -->|evaluated| D[(Database)] D -->|rows| C
from blog.models import Post
# 'objects' is the manager; .all() returns a QuerySet
all_posts = Post.objects.all()
# Managers live on the CLASS, not on instances
first = Post.objects.first()
# first.objects.all() # AttributeError — instances have no manager
Lazy Evaluation
QuerySets are lazy: building and chaining them costs nothing. The database is queried only when the results are genuinely needed. This lets you compose complex queries in several steps, and even pass a QuerySet around, without paying for a query at each step.
# No SQL runs on any of these three lines
qs = Post.objects.all()
qs = qs.filter(status="published")
qs = qs.order_by("-published_at")
# SQL runs HERE, once, when we iterate
for post in qs:
print(post.title)
📖 When does a QuerySet evaluate?
A QuerySet hits the database when you:
- Iterate over it (a
forloop, a list comprehension) - Call
list(),len(),bool(), or slice with a step - Call terminal methods like
.count(),.exists(),.get(),.first() - Render it in a template or call
repr()on it
💡 Results are cached after the first evaluation
Once a QuerySet is evaluated, its rows are cached on that object, so looping over the same QuerySet variable twice queries the database only once. But Post.objects.filter(...) written out twice creates two different QuerySets and runs two queries. Assign it to a variable if you intend to reuse the results.
Core QuerySet Methods
A handful of methods cover the vast majority of everyday querying. They split into two groups: methods that return a new QuerySet (chainable, still lazy) and methods that return a concrete result (terminal — they run the query).
| Method | Returns | Purpose |
|---|---|---|
all() | QuerySet | Every row |
filter(**kw) | QuerySet | Rows matching the conditions |
exclude(**kw) | QuerySet | Rows not matching |
order_by(*fields) | QuerySet | Sort (prefix - for descending) |
get(**kw) | one object | Exactly one row (or raises) |
first() / last() | object or None | Edge rows |
count() | int | How many rows |
exists() | bool | Are there any rows? |
# Chainable — all still lazy
published = Post.objects.filter(status="published").order_by("-published_at")
not_drafts = Post.objects.exclude(status="draft")
newest_five = Post.objects.order_by("-published_at")[:5] # slicing adds LIMIT
# Terminal — these run the query
total = Post.objects.count()
any_published = Post.objects.filter(status="published").exists()
Fetching a single object with get()
get() expects exactly one match. Guard it against the two things that can go wrong:
try:
post = Post.objects.get(slug="hello-world")
except Post.DoesNotExist:
post = None # no row matched
except Post.MultipleObjectsReturned:
raise # your assumption of uniqueness was wrong
✅ Use exists(), not count() > 0
To check "are there any?", exists() asks the database to stop at the first matching row, while count() counts them all. For an existence check, exists() is both clearer and faster.
Field Lookups & Q Objects
Field lookups are the double-underscore syntax that turns a filter keyword into a specific SQL condition. The pattern is field__lookup=value.
# Text
Post.objects.filter(title__icontains="django") # case-insensitive LIKE
Post.objects.filter(title__startswith="The")
# Numbers & ranges
Post.objects.filter(rating__gte=4) # >= 4
Post.objects.filter(rating__range=(3, 5)) # BETWEEN 3 AND 5
# Dates
Post.objects.filter(published_at__year=2026)
Post.objects.filter(published_at__date=some_date)
# NULL test
Post.objects.filter(category__isnull=True)
# Relationship traversal — follow a ForeignKey with __
Post.objects.filter(author__username="ray")
Post.objects.filter(tags__name__icontains="python")
Passing several keyword arguments to one filter() call combines them with AND. For OR, negation, or grouping, reach for Q objects:
from django.db.models import Q
# published OR featured
Post.objects.filter(Q(status="published") | Q(is_featured=True))
# published AND NOT featured
Post.objects.filter(Q(status="published") & ~Q(is_featured=True))
# grouped: (published or featured) and in the Tech category
Post.objects.filter(
(Q(status="published") | Q(is_featured=True)) & Q(category__name="Tech")
)
⚠️ | and &, not or and and
Q objects are combined with the bitwise operators | (or), & (and), and ~ (not). Python's keywords or/and won't work here — they'd evaluate the objects for truthiness instead of building a query. Watch your parentheses, too: operator precedence makes explicit grouping worthwhile.
Aggregation, Annotation & F()
Aggregation — one summary value for the whole set
aggregate() collapses a QuerySet into a dictionary of computed values:
from django.db.models import Avg, Count, Max, Min
Post.objects.aggregate(Avg("rating"))
# {'rating__avg': 4.2}
Post.objects.aggregate(
total=Count("id"),
avg_rating=Avg("rating"),
top=Max("rating"),
)
# {'total': 42, 'avg_rating': 4.2, 'top': 5.0}
Annotation — one computed value per row
annotate() attaches a calculated attribute to each object in the QuerySet — perfect for "count of related rows":
from django.db.models import Count
# Each author now has an .article_count attribute
authors = Author.objects.annotate(article_count=Count("articles"))
for author in authors:
print(author.name, author.article_count)
# Conditional counting with a filter argument
from django.db.models import Q
Post.objects.annotate(
approved_comments=Count("comments", filter=Q(comments__is_approved=True)),
)
📖 aggregate vs annotate
aggregate() returns a single dictionary summarizing the whole QuerySet. annotate() returns a QuerySet where every object carries an extra per-row value. Rule of thumb: one number for everything → aggregate; one number per object → annotate.
F() — refer to a field's value in the query
F() lets the database compare or update fields against each other, with no round-trip to Python and no race conditions:
from django.db.models import F
# Atomic increment — safe under concurrency
Post.objects.filter(pk=1).update(view_count=F("view_count") + 1)
# Compare two columns to each other
Post.objects.filter(comment_count__gt=F("view_count"))
Custom Managers & QuerySets
When the same filters appear across your views, move them into the model layer. There are two building blocks, and the modern best-practice combines them.
A custom manager (simple)
from django.db import models
class PublishedManager(models.Manager):
def get_queryset(self):
return super().get_queryset().filter(status="published")
class Post(models.Model):
# ... fields ...
objects = models.Manager() # the default manager
published = PublishedManager() # Post.published.all()
The catch: manager methods aren't chainable. Post.published gives you published posts, but you can't write Post.objects.published().recent() because published() would live on the manager, not the QuerySet.
A custom QuerySet with as_manager() (recommended)
Define your reusable filters on a QuerySet subclass so they chain, then expose it as the manager with .as_manager():
from datetime import timedelta
from django.db import models
from django.utils import timezone
class PostQuerySet(models.QuerySet):
def published(self):
return self.filter(status="published")
def recent(self, days=7):
cutoff = timezone.now() - timedelta(days=days)
return self.filter(published_at__gte=cutoff)
def by_author(self, author):
return self.filter(author=author)
class Post(models.Model):
# ... fields ...
objects = PostQuerySet.as_manager()
Now every method chains freely, in any order:
# Reads like a sentence — and it's all one SQL query
Post.objects.published().recent(days=30).by_author(ray).order_by("-published_at")
✅ Prefer QuerySet.as_manager()
Because the methods live on the QuerySet, they're chainable and reusable across relationships. This one pattern removes almost all the duplicated filter(status="published") calls scattered through a typical project.
Query Optimization
The single most common Django performance bug is the N+1 query problem: you fetch a list of N objects, then trigger one extra query per object while looping to access a related object. A page that should run 1 query ends up running 51.
# ❌ N+1: one query for the posts, then one MORE per post for its author
posts = Post.objects.all()
for post in posts:
print(post.author.username) # a new query every iteration
select_related — for ForeignKey / OneToOne
Follows the relation with a SQL JOIN, pulling the related object in the same query:
# ✅ One query total, via a JOIN
posts = Post.objects.select_related("author")
for post in posts:
print(post.author.username) # no extra queries
prefetch_related — for ManyToMany / reverse ForeignKey
Runs a second query for the related rows and joins them in Python — the right tool for "to-many" relations a JOIN can't collapse:
# ✅ Two queries total, no matter how many posts
posts = Post.objects.prefetch_related("tags")
for post in posts:
print([tag.name for tag in post.tags.all()])
# Combine both when a view needs each
posts = Post.objects.select_related("author").prefetch_related("tags")
Fetch only what you need
# Dictionaries instead of model instances
Post.objects.values("title", "author__username")
# A flat list of one column
Post.objects.values_list("title", flat=True)
# Load a subset of columns (the rest load lazily if touched)
Post.objects.only("title", "published_at")
💡 Which one? A quick rule
Following a relation to a single object (ForeignKey, OneToOneField) → select_related. Following it to many objects (ManyToManyField, reverse ForeignKey) → prefetch_related. Install the Django Debug Toolbar early — seeing the query count per page makes N+1 problems obvious.
Hands-on Exercise
🏋️ Build a chainable blog QuerySet
Objective: Encapsulate common blog queries in a custom QuerySet, then use it to power an optimized list view.
Requirements:
- Write a
PostQuerySetwith three chainable methods:published(),recent(days=7), andpopular(min_comments=5)(useannotate+Count). - Attach it to
Postviaobjects = PostQuerySet.as_manager(). - Write one expression that returns recent, popular, published posts, newest first.
- In a view, fetch those posts with
select_related("author")andprefetch_related("tags")so the template renders with a constant number of queries.
💡 Hint
For popular(), annotate a comment_count with Count("comments") and then filter(comment_count__gte=min_comments). Because each method returns self.filter(...), they chain in any order.
✅ Sample solution
from datetime import timedelta
from django.db import models
from django.utils import timezone
class PostQuerySet(models.QuerySet):
def published(self):
return self.filter(status="published")
def recent(self, days=7):
cutoff = timezone.now() - timedelta(days=days)
return self.filter(published_at__gte=cutoff)
def popular(self, min_comments=5):
return self.annotate(
comment_count=models.Count("comments")
).filter(comment_count__gte=min_comments)
class Post(models.Model):
# ... fields ...
objects = PostQuerySet.as_manager()
# One clean expression
featured = (
Post.objects.published()
.recent(days=30)
.popular(min_comments=10)
.order_by("-published_at")
)
# Optimized view query
def post_list(request):
posts = (
Post.objects.published()
.select_related("author")
.prefetch_related("tags")
.order_by("-published_at")
)
return render(request, "blog/list.html", {"posts": posts})
🎯 Quick Quiz
Question 1: At which line does this code actually query the database? qs = Post.objects.filter(status="published")qs = qs.order_by("-published_at")for post in qs: ...
Question 2: You loop over posts and print each post.author.username, causing one extra query per post. Which method fixes this best?
Question 3: You want posts that are either published or featured. Which is correct?
Best Practices
✅ Do
- Encapsulate repeated filters in a custom
QuerySetexposed viaas_manager(). - Use
select_related/prefetch_relatedto kill N+1 queries. - Use
exists()for existence checks andcount()only when you need the number. - Fetch only needed columns with
values(),values_list(), oronly(). - Use
F()for atomic updates and field-to-field comparisons. - Wrap
get()in try/except forDoesNotExist/MultipleObjectsReturned.
⚠️ Don't
- Don't access related objects in a loop without prefetching first.
- Don't combine
Qobjects with Python'sand/or— use&/|/~. - Don't call
len(qs)just to check emptiness — useexists(). - Don't re-run the same
filter()repeatedly; assign it to a variable to reuse the cache.
Summary & Quiz
🎉 Key Takeaways
- The manager (
objects) is your entry point; it returns lazy, chainable QuerySets. - QuerySets don't hit the database until iterated or forced — and results are cached afterward.
- Field lookups (
field__lookup) and Q objects express everything from simple filters to complex OR/NOT logic. - aggregate summarizes the whole set; annotate adds a value per row;
F()references fields in-database. - Custom QuerySets via
as_manager()keep query logic DRY, andselect_related/prefetch_relatedfix N+1.
📚 Further Reading
- Django Docs — Making queries
- Django Docs — QuerySet API reference
- Django Docs — Managers
- Django Docs — Aggregation
- Django Docs — Database access optimization
🚀 What's Next?
You can now read and write data fluently. In the next lesson, Function-Based Views, you'll wire these queries into request handlers that turn URLs into HTTP responses — the layer where your models finally meet the browser.
🎉 Great work!
You've mastered the ORM's core. Efficient, readable queries are now within reach.