-
Notifications
You must be signed in to change notification settings - Fork 1
updates for time #27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
updates for time #27
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR updates how indexing time is measured across the retrieval optimizer codebase. Instead of relying on Redis's index.info()["total_indexing_time"], the code now manually measures wall-clock time from when indexing starts until completion (percent_indexed == 1). The measured time is persisted to Redis for reuse in subsequent runs that don't reload data.
Key changes:
- Introduced new utility functions
get_last_indexing_time()andset_last_indexing_time()to persist and retrieve indexing time measurements - Updated
bayes_study.py,grid_study.py, andsearch_study.pyto measure indexing time usingtime.time()instead of queryingindex.info() - Added documentation clarifying that
total_indexing_timeis measured in seconds using wall-clock time
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
redis_retrieval_optimizer/utils.py |
Added two new utility functions for getting/setting last indexing time in Redis |
redis_retrieval_optimizer/search_study.py |
Updated to retrieve persisted indexing time instead of using index.info() |
redis_retrieval_optimizer/grid_study.py |
Added wall-clock timing measurement when indexing data and persistence of timing |
redis_retrieval_optimizer/bayes_study.py |
Added wall-clock timing measurement with conditional logic for recreate vs. reuse scenarios |
README.md |
Added documentation explaining the wall-clock timing methodology for indexing time |
docs/examples/search_study/search_study_walkthrough.ipynb |
New example notebook demonstrating search study workflow with existing index |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| "source": [ | ||
| "# Search study\n", | ||
| "\n", | ||
| "Let's say you have an existing Redis database with a search index seeded. You may wish to quickly test different search method against the existing index without having to recreate data and/or recreate the index. This demo will walk you though how to set this up and get going.\n", |
Copilot
AI
Dec 8, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The spelling of "though" is incorrect. It should be "through" in this context.
Suggested fix: Change "though" to "through"
| "Let's say you have an existing Redis database with a search index seeded. You may wish to quickly test different search method against the existing index without having to recreate data and/or recreate the index. This demo will walk you though how to set this up and get going.\n", | |
| "Let's say you have an existing Redis database with a search index seeded. You may wish to quickly test different search method against the existing index without having to recreate data and/or recreate the index. This demo will walk you through how to set this up and get going.\n", |
| "\n", | ||
| "For the search_study we are assuming that the search index already exists. The cell below will create a Redis search index and populate it with our test data for example purposes but is assumed with a search study is populated and running within your data. \n", | ||
| "\n", | ||
| "Note: the demo assumes you have a instance of redis running on localhost:6379. If this is not the case, update the redis_url to direct to your running instance or start a local instance with the following command. \n", |
Copilot
AI
Dec 8, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The documentation states the "demo assumes you have a instance of redis" which should be "an instance" instead of "a instance".
Suggested fix: Change "a instance" to "an instance"
| "Note: the demo assumes you have a instance of redis running on localhost:6379. If this is not the case, update the redis_url to direct to your running instance or start a local instance with the following command. \n", | |
| "Note: the demo assumes you have an instance of redis running on localhost:6379. If this is not the case, update the redis_url to direct to your running instance or start a local instance with the following command. \n", |
f1c7663 to
40a6fbd
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
No description provided.