Skip to main content

🔎 QuerySets and Model Managers

Django's ORM lets you read and write your database in Python instead of SQL. At its heart are two objects: the manager (your entry point, Model.objects) and the QuerySet (a lazy, chainable, cacheable collection of rows). Master these and the whole ORM opens up.

🎯 Learning Objectives

By the end of this lesson, you will be able to:

  • Explain how QuerySets are lazy, chainable, and cached — and when a query actually hits the database
  • Filter data with field lookups, relationship traversal, and Q objects
  • Aggregate and annotate data, and use F() expressions for field-to-field math
  • Write custom managers and QuerySets to keep query logic DRY
  • Eliminate the N+1 query problem with select_related and prefetch_related

Estimated Time: 50–65 minutes  •  Difficulty: Intermediate

Hands-on: Build a custom, chainable QuerySet for a blog and optimize a list view.

In This Lesson

Managers & QuerySets

Every Django model gets a manager named objects. The manager is the doorway to the database for that model — you call methods on it (Post.objects.all(), Post.objects.filter(...)) and it hands back a QuerySet: a collection of rows you can filter, order, and iterate.

The crucial insight is that a QuerySet is not a list of results. It's a description of a query that hasn't run yet. You build it up method by method, and Django only sends SQL to the database at the last possible moment.

💡 A useful analogy: The manager is a librarian, and a QuerySet is your written request slip. You can keep adding conditions to the slip — "published books, by this author, newest first" — without the librarian moving an inch. Only when you actually ask to read the results does the librarian walk the stacks once and bring back everything at once.
flowchart LR A[Model class
Post] --> B[Manager
Post.objects] B --> C[QuerySet
.filter.order_by] C -->|evaluated| D[(Database)] D -->|rows| C
from blog.models import Post

# 'objects' is the manager; .all() returns a QuerySet
all_posts = Post.objects.all()

# Managers live on the CLASS, not on instances
first = Post.objects.first()
# first.objects.all()   # AttributeError — instances have no manager

Lazy Evaluation

QuerySets are lazy: building and chaining them costs nothing. The database is queried only when the results are genuinely needed. This lets you compose complex queries in several steps, and even pass a QuerySet around, without paying for a query at each step.

# No SQL runs on any of these three lines
qs = Post.objects.all()
qs = qs.filter(status="published")
qs = qs.order_by("-published_at")

# SQL runs HERE, once, when we iterate
for post in qs:
    print(post.title)

📖 When does a QuerySet evaluate?

A QuerySet hits the database when you:

  • Iterate over it (a for loop, a list comprehension)
  • Call list(), len(), bool(), or slice with a step
  • Call terminal methods like .count(), .exists(), .get(), .first()
  • Render it in a template or call repr() on it

💡 Results are cached after the first evaluation

Once a QuerySet is evaluated, its rows are cached on that object, so looping over the same QuerySet variable twice queries the database only once. But Post.objects.filter(...) written out twice creates two different QuerySets and runs two queries. Assign it to a variable if you intend to reuse the results.

Core QuerySet Methods

A handful of methods cover the vast majority of everyday querying. They split into two groups: methods that return a new QuerySet (chainable, still lazy) and methods that return a concrete result (terminal — they run the query).

MethodReturnsPurpose
all()QuerySetEvery row
filter(**kw)QuerySetRows matching the conditions
exclude(**kw)QuerySetRows not matching
order_by(*fields)QuerySetSort (prefix - for descending)
get(**kw)one objectExactly one row (or raises)
first() / last()object or NoneEdge rows
count()intHow many rows
exists()boolAre there any rows?
# Chainable — all still lazy
published = Post.objects.filter(status="published").order_by("-published_at")
not_drafts = Post.objects.exclude(status="draft")
newest_five = Post.objects.order_by("-published_at")[:5]   # slicing adds LIMIT

# Terminal — these run the query
total = Post.objects.count()
any_published = Post.objects.filter(status="published").exists()

Fetching a single object with get()

get() expects exactly one match. Guard it against the two things that can go wrong:

try:
    post = Post.objects.get(slug="hello-world")
except Post.DoesNotExist:
    post = None            # no row matched
except Post.MultipleObjectsReturned:
    raise                  # your assumption of uniqueness was wrong

✅ Use exists(), not count() > 0

To check "are there any?", exists() asks the database to stop at the first matching row, while count() counts them all. For an existence check, exists() is both clearer and faster.

Field Lookups & Q Objects

Field lookups are the double-underscore syntax that turns a filter keyword into a specific SQL condition. The pattern is field__lookup=value.

# Text
Post.objects.filter(title__icontains="django")     # case-insensitive LIKE
Post.objects.filter(title__startswith="The")

# Numbers & ranges
Post.objects.filter(rating__gte=4)                  # >= 4
Post.objects.filter(rating__range=(3, 5))           # BETWEEN 3 AND 5

# Dates
Post.objects.filter(published_at__year=2026)
Post.objects.filter(published_at__date=some_date)

# NULL test
Post.objects.filter(category__isnull=True)

# Relationship traversal — follow a ForeignKey with __
Post.objects.filter(author__username="ray")
Post.objects.filter(tags__name__icontains="python")

Passing several keyword arguments to one filter() call combines them with AND. For OR, negation, or grouping, reach for Q objects:

from django.db.models import Q

# published OR featured
Post.objects.filter(Q(status="published") | Q(is_featured=True))

# published AND NOT featured
Post.objects.filter(Q(status="published") & ~Q(is_featured=True))

# grouped: (published or featured) and in the Tech category
Post.objects.filter(
    (Q(status="published") | Q(is_featured=True)) & Q(category__name="Tech")
)

⚠️ | and &, not or and and

Q objects are combined with the bitwise operators | (or), & (and), and ~ (not). Python's keywords or/and won't work here — they'd evaluate the objects for truthiness instead of building a query. Watch your parentheses, too: operator precedence makes explicit grouping worthwhile.

Aggregation, Annotation & F()

Aggregation — one summary value for the whole set

aggregate() collapses a QuerySet into a dictionary of computed values:

from django.db.models import Avg, Count, Max, Min

Post.objects.aggregate(Avg("rating"))
# {'rating__avg': 4.2}

Post.objects.aggregate(
    total=Count("id"),
    avg_rating=Avg("rating"),
    top=Max("rating"),
)
# {'total': 42, 'avg_rating': 4.2, 'top': 5.0}

Annotation — one computed value per row

annotate() attaches a calculated attribute to each object in the QuerySet — perfect for "count of related rows":

from django.db.models import Count

# Each author now has an .article_count attribute
authors = Author.objects.annotate(article_count=Count("articles"))
for author in authors:
    print(author.name, author.article_count)

# Conditional counting with a filter argument
from django.db.models import Q
Post.objects.annotate(
    approved_comments=Count("comments", filter=Q(comments__is_approved=True)),
)

📖 aggregate vs annotate

aggregate() returns a single dictionary summarizing the whole QuerySet. annotate() returns a QuerySet where every object carries an extra per-row value. Rule of thumb: one number for everything → aggregate; one number per object → annotate.

F() — refer to a field's value in the query

F() lets the database compare or update fields against each other, with no round-trip to Python and no race conditions:

from django.db.models import F

# Atomic increment — safe under concurrency
Post.objects.filter(pk=1).update(view_count=F("view_count") + 1)

# Compare two columns to each other
Post.objects.filter(comment_count__gt=F("view_count"))

Custom Managers & QuerySets

When the same filters appear across your views, move them into the model layer. There are two building blocks, and the modern best-practice combines them.

A custom manager (simple)

from django.db import models


class PublishedManager(models.Manager):
    def get_queryset(self):
        return super().get_queryset().filter(status="published")


class Post(models.Model):
    # ... fields ...
    objects = models.Manager()          # the default manager
    published = PublishedManager()      # Post.published.all()

The catch: manager methods aren't chainable. Post.published gives you published posts, but you can't write Post.objects.published().recent() because published() would live on the manager, not the QuerySet.

A custom QuerySet with as_manager() (recommended)

Define your reusable filters on a QuerySet subclass so they chain, then expose it as the manager with .as_manager():

from datetime import timedelta
from django.db import models
from django.utils import timezone


class PostQuerySet(models.QuerySet):
    def published(self):
        return self.filter(status="published")

    def recent(self, days=7):
        cutoff = timezone.now() - timedelta(days=days)
        return self.filter(published_at__gte=cutoff)

    def by_author(self, author):
        return self.filter(author=author)


class Post(models.Model):
    # ... fields ...
    objects = PostQuerySet.as_manager()

Now every method chains freely, in any order:

# Reads like a sentence — and it's all one SQL query
Post.objects.published().recent(days=30).by_author(ray).order_by("-published_at")

✅ Prefer QuerySet.as_manager()

Because the methods live on the QuerySet, they're chainable and reusable across relationships. This one pattern removes almost all the duplicated filter(status="published") calls scattered through a typical project.

Query Optimization

The single most common Django performance bug is the N+1 query problem: you fetch a list of N objects, then trigger one extra query per object while looping to access a related object. A page that should run 1 query ends up running 51.

# ❌ N+1: one query for the posts, then one MORE per post for its author
posts = Post.objects.all()
for post in posts:
    print(post.author.username)      # a new query every iteration

select_related — for ForeignKey / OneToOne

Follows the relation with a SQL JOIN, pulling the related object in the same query:

# ✅ One query total, via a JOIN
posts = Post.objects.select_related("author")
for post in posts:
    print(post.author.username)      # no extra queries

prefetch_related — for ManyToMany / reverse ForeignKey

Runs a second query for the related rows and joins them in Python — the right tool for "to-many" relations a JOIN can't collapse:

# ✅ Two queries total, no matter how many posts
posts = Post.objects.prefetch_related("tags")
for post in posts:
    print([tag.name for tag in post.tags.all()])

# Combine both when a view needs each
posts = Post.objects.select_related("author").prefetch_related("tags")

Fetch only what you need

# Dictionaries instead of model instances
Post.objects.values("title", "author__username")

# A flat list of one column
Post.objects.values_list("title", flat=True)

# Load a subset of columns (the rest load lazily if touched)
Post.objects.only("title", "published_at")

💡 Which one? A quick rule

Following a relation to a single object (ForeignKey, OneToOneField) → select_related. Following it to many objects (ManyToManyField, reverse ForeignKey) → prefetch_related. Install the Django Debug Toolbar early — seeing the query count per page makes N+1 problems obvious.

Hands-on Exercise

🏋️ Build a chainable blog QuerySet

Objective: Encapsulate common blog queries in a custom QuerySet, then use it to power an optimized list view.

Requirements:

  1. Write a PostQuerySet with three chainable methods: published(), recent(days=7), and popular(min_comments=5) (use annotate + Count).
  2. Attach it to Post via objects = PostQuerySet.as_manager().
  3. Write one expression that returns recent, popular, published posts, newest first.
  4. In a view, fetch those posts with select_related("author") and prefetch_related("tags") so the template renders with a constant number of queries.
💡 Hint

For popular(), annotate a comment_count with Count("comments") and then filter(comment_count__gte=min_comments). Because each method returns self.filter(...), they chain in any order.

✅ Sample solution
from datetime import timedelta
from django.db import models
from django.utils import timezone


class PostQuerySet(models.QuerySet):
    def published(self):
        return self.filter(status="published")

    def recent(self, days=7):
        cutoff = timezone.now() - timedelta(days=days)
        return self.filter(published_at__gte=cutoff)

    def popular(self, min_comments=5):
        return self.annotate(
            comment_count=models.Count("comments")
        ).filter(comment_count__gte=min_comments)


class Post(models.Model):
    # ... fields ...
    objects = PostQuerySet.as_manager()


# One clean expression
featured = (
    Post.objects.published()
    .recent(days=30)
    .popular(min_comments=10)
    .order_by("-published_at")
)

# Optimized view query
def post_list(request):
    posts = (
        Post.objects.published()
        .select_related("author")
        .prefetch_related("tags")
        .order_by("-published_at")
    )
    return render(request, "blog/list.html", {"posts": posts})

🎯 Quick Quiz

Question 1: At which line does this code actually query the database?
qs = Post.objects.filter(status="published")
qs = qs.order_by("-published_at")
for post in qs: ...

Question 2: You loop over posts and print each post.author.username, causing one extra query per post. Which method fixes this best?

Question 3: You want posts that are either published or featured. Which is correct?

Best Practices

✅ Do

  • Encapsulate repeated filters in a custom QuerySet exposed via as_manager().
  • Use select_related / prefetch_related to kill N+1 queries.
  • Use exists() for existence checks and count() only when you need the number.
  • Fetch only needed columns with values(), values_list(), or only().
  • Use F() for atomic updates and field-to-field comparisons.
  • Wrap get() in try/except for DoesNotExist / MultipleObjectsReturned.

⚠️ Don't

  • Don't access related objects in a loop without prefetching first.
  • Don't combine Q objects with Python's and/or — use &/|/~.
  • Don't call len(qs) just to check emptiness — use exists().
  • Don't re-run the same filter() repeatedly; assign it to a variable to reuse the cache.

Summary & Quiz

🎉 Key Takeaways

  • The manager (objects) is your entry point; it returns lazy, chainable QuerySets.
  • QuerySets don't hit the database until iterated or forced — and results are cached afterward.
  • Field lookups (field__lookup) and Q objects express everything from simple filters to complex OR/NOT logic.
  • aggregate summarizes the whole set; annotate adds a value per row; F() references fields in-database.
  • Custom QuerySets via as_manager() keep query logic DRY, and select_related/prefetch_related fix N+1.

📚 Further Reading

🚀 What's Next?

You can now read and write data fluently. In the next lesson, Function-Based Views, you'll wire these queries into request handlers that turn URLs into HTTP responses — the layer where your models finally meet the browser.

🎉 Great work!

You've mastered the ORM's core. Efficient, readable queries are now within reach.