Search: optimize queryset when searching for subprojects with organizations #12635
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Documentation build overview
Show files changed (8 files in total): 📝 7 modified | ➕ 0 added | ➖ 1 deleted
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes database queries when searching projects with organizations by implementing strategic caching and prefetching. The key optimization is that since all subprojects share the same organization as their parent project, the organization can be cached and reused across all related projects.
Key Changes
- Converted
Project.organizationfrom@propertyto@cached_propertyto cache organization lookups - Added
prefetch_organization()queryset method for efficient bulk loading of organizations - Implemented manual organization caching in search executor for subprojects via
_organizationsattribute - Replaced all
project.organizations.first()calls withproject.organizationthroughout the codebase - Added resolver caching for organization-level feature checks (custom domains)
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| readthedocs/projects/models.py | Converted organization property to cached_property with RTD_ALLOW_ORGANIZATIONS check |
| readthedocs/projects/querysets.py | Added prefetch_organization() method for bulk loading with select_related support |
| readthedocs/search/api/v3/executor.py | Manually cache organization for subprojects by setting _organizations attribute; added prefetch_organization to project queries |
| readthedocs/search/api/v2/serializers.py | Reuse single Resolver instance across projects to leverage organization caching |
| readthedocs/core/resolver.py | Added organization-level caching for custom domain checks with @cache decorator |
| readthedocs/telemetry/models.py | Updated to use project.organization property |
| readthedocs/core/permissions.py | Updated to use project.organization property |
| readthedocs/builds/querysets.py | Updated to use project.organization property |
| readthedocs/organizations/tasks.py | Updated to use project.organization property |
| readthedocs/audit/models.py | Updated to use project.organization property |
| readthedocs/api/v3/tests/test_subprojects.py | Removed erroneous organization.projects.add() line |
| readthedocs/search/api/v3/tests/test_api.py | Added comprehensive tests validating query counts with organizations |
| readthedocs/rtd_tests/tests/test_project_querysets.py | Added @override_settings decorator and cache clearing for organization tests |
| readthedocs/builds/tests/test_build_queryset.py | Added @override_settings decorator for organization tests |
| readthedocs/invitations/tests/test_views.py | Added @override_settings decorator for organization tests |
| readthedocs/audit/tests/test_tasks.py | Added @override_settings decorator for organization tests |
| readthedocs/audit/tests/test_models.py | Added @override_settings decorator for organization tests |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
ericholscher
approved these changes
Dec 9, 2025
Member
ericholscher
left a comment
ericholscher
left a comment
There was a problem hiding this comment.
Looks good with a couple questions.
Documentation build overview
Show files changed (4 files in total): 📝 4 modified | ➕ 0 added | ➖ 0 deleted
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
On .com having organizations introduces some extra queries. Since all subprojects share the same organization as the parent project, we can cache that organization, so it's shared across all subprojects.
This also allows caching the .organization property, and skips querying organizations if we are on .org.
Results:
I had to move tests to .com, so they are more accurate with production, as in .com we do several overrides.