-
Notifications
You must be signed in to change notification settings - Fork 749
Creating Paramaterized Test For Quantizers For Easier Testing #16098
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
…6089) Summary: We test the quantizer we added in D87996796 correctly annotates the graph. We use the graph builder to build the graph with metadata(that's needed for quantizer.annotate to recognize the nodes), and we ensure that the quantization params are as expected. Reviewed By: hsharma35 Differential Revision: D88053808
…6097) Summary: We test the CadenceWith16BitLinearActivationQuantizer. We use the graph builder to build the graph with metadata(that's needed for quantizer.annotate to recognize the nodes), and we ensure that the quantization params are as expected. Reviewed By: hsharma35 Differential Revision: D88054651
Summary: We consolidate the two tests we created into a single testing function using parameterization. This will make testing future Quantizers much easier, and there will be a lot less code duplication. Reviewed By: hsharma35 Differential Revision: D88054917
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/16098
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ You can merge normally! (1 Unrelated Failure)As of commit c924c92 with merge base 4014597 ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
This PR needs a
|
…h#16098) Summary: We consolidate the two tests we created into a single testing function using parameterization. This will make testing future Quantizers much easier, and there will be a lot less code duplication. Reviewed By: hsharma35 Differential Revision: D88054917
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
This PR consolidates quantizer annotation tests into a single parameterized test function, reducing code duplication and making it easier to add new quantizer tests. The refactoring uses the parameterized library to test both matmul and linear operations with their respective 16-bit activation quantizers.
Key changes:
- Introduced parameterized testing approach using
@parameterized.expanddecorator - Created reusable graph builder helper methods (
_build_matmul_graph,_build_linear_graph) - Added new imports for GraphBuilder, parameterized library, and quantization annotation types
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| backends/cadence/aot/tests/test_quantizer_ops.py | Added parameterized test infrastructure with QuantizerAnnotationTest class containing graph builder helpers and consolidated test method for verifying quantizer annotations on matmul and linear operations |
| backends/cadence/aot/TARGETS | Added dependencies for parameterized testing library, graph_builder module, pass_base, and torchao to support the new test infrastructure |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| name: str, | ||
| graph_builder_fn: GraphBuilderFn, | ||
| quantizer: CadenceQuantizer, | ||
| target: OpOverload, |
Copilot
AI
Dec 5, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The target parameter is unused in the test function. Consider removing it from the parameter list (lines 94 and 103) if it's not needed, or use it to validate that op_node.target matches the expected target operation for additional test robustness.
| actual_specs = [ | ||
| annotation.input_qspec_map[op_node.args[i]] | ||
| for i in range(len(expected_input_qspecs)) | ||
| ] |
Copilot
AI
Dec 5, 2025
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Potential KeyError: accessing annotation.input_qspec_map[op_node.args[i]] could fail if op_node.args[i] is not in the map. Consider adding a check or using .get() with a clear error message to make debugging easier if the key is missing.
| actual_specs = [ | |
| annotation.input_qspec_map[op_node.args[i]] | |
| for i in range(len(expected_input_qspecs)) | |
| ] | |
| actual_specs = [] | |
| for i in range(len(expected_input_qspecs)): | |
| key = op_node.args[i] | |
| if key not in annotation.input_qspec_map: | |
| raise KeyError( | |
| f"Key {key!r} not found in input_qspec_map. " | |
| f"Available keys: {list(annotation.input_qspec_map.keys())}" | |
| ) | |
| actual_specs.append(annotation.input_qspec_map[key]) |
Summary:
We consolidate the two tests we created into a single testing function using parameterization.
This will make testing future Quantizers much easier, and there will be a lot less code duplication.
Reviewed By: hsharma35
Differential Revision: D88054917