Skip to content

Commit 429e61f

Browse files
yossiovadiaclaude
andcommitted
fix(e2e): add decision routes for all 14 LoRA categories in test profiles
The production-stack and llm-d E2E test profiles were failing with 0-1% domain classification accuracy because they only configured decision routes for 1-2 categories while using LoRA intent classifiers that classify into 14 categories. When the classifier correctly identified categories like "biology", "health", or "math", no matching decision existed, causing "decision evaluation failed: no decision matched" errors. Changes: - production-stack: Added decision routes for all 14 categories (business, philosophy, biology, health, computer science, engineering, psychology, math, chemistry, physics, history, law, economics, other) - llm-d: Added decision routes for all 14 categories with intelligent grouping (sciences, social sciences, humanities) Results: - production-stack domain classification: 1% → 53% accuracy (50x improvement) - All 12 production-stack E2E tests now pass This fix ensures LoRA auto-detection works properly by providing decision routes for all categories that the classifier can identify. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent c45df00 commit 429e61f

File tree

2 files changed

+348
-13
lines changed

2 files changed

+348
-13
lines changed

e2e/profiles/llm-d/values.yaml

Lines changed: 108 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,26 +5,53 @@ config:
55
clear_route_cache: true
66
default_model: llama3-8b
77
# Enable domain classification to return the x-vsr-selected-category header
8+
# Using LoRA models for better performance with auto-detection
89
classifier:
910
category_model:
10-
model_id: models/category_classifier_modernbert-base_model
11+
model_id: models/lora_intent_classifier_bert-base-uncased_model
1112
threshold: 0.6
12-
use_modernbert: true
13-
category_mapping_path: models/category_classifier_modernbert-base_model/category_mapping.json
13+
use_modernbert: false
14+
category_mapping_path: models/lora_intent_classifier_bert-base-uncased_model/category_mapping.json
1415
pii_model:
1516
model_id: ""
1617
threshold: 1.0
1718
use_modernbert: false
1819
pii_mapping_path: ""
19-
# Optional domains used by decision rules
20+
# All 14 categories from LoRA intent classifier
2021
categories:
21-
- name: math
22-
description: "Mathematics, arithmetic, calculation"
22+
- name: business
23+
description: "Business management, marketing, entrepreneurship"
24+
- name: philosophy
25+
description: "Ethics, logic, metaphysics, epistemology"
26+
- name: biology
27+
description: "Genetics, evolution, ecology, anatomy"
28+
- name: health
29+
description: "Medicine, nutrition, fitness, diseases"
2330
- name: computer science
2431
description: "Computer networks, programming, systems"
32+
- name: engineering
33+
description: "Mechanical, electrical, civil engineering"
34+
- name: psychology
35+
description: "Behavior, cognition, mental health"
36+
- name: math
37+
description: "Mathematics, arithmetic, calculation"
38+
- name: chemistry
39+
description: "Chemical reactions, compounds, periodic table"
40+
- name: physics
41+
description: "Mechanics, thermodynamics, quantum physics"
42+
- name: history
43+
description: "Historical events, civilizations, wars"
44+
- name: law
45+
description: "Legal systems, regulations, court procedures"
46+
- name: economics
47+
description: "Microeconomics, macroeconomics, financial markets"
48+
- name: other
49+
description: "General knowledge and miscellaneous topics"
50+
2551
decisions:
52+
# High priority for math - use specialized model
2653
- name: math_route
27-
priority: 20
54+
priority: 100
2855
rules:
2956
operator: OR
3057
conditions:
@@ -33,8 +60,10 @@ config:
3360
modelRefs:
3461
- model: phi4-mini
3562
use_reasoning: false
63+
64+
# High priority for computer science - use llama3
3665
- name: cs_route
37-
priority: 10
66+
priority: 100
3867
rules:
3968
operator: OR
4069
conditions:
@@ -43,6 +72,77 @@ config:
4372
modelRefs:
4473
- model: llama3-8b
4574
use_reasoning: false
75+
76+
# Medium priority routes for other technical domains
77+
- name: science_route
78+
priority: 50
79+
rules:
80+
operator: OR
81+
conditions:
82+
- type: domain
83+
name: physics
84+
- type: domain
85+
name: chemistry
86+
- type: domain
87+
name: biology
88+
modelRefs:
89+
- model: llama3-8b
90+
use_reasoning: false
91+
92+
- name: engineering_route
93+
priority: 50
94+
rules:
95+
operator: OR
96+
conditions:
97+
- type: domain
98+
name: engineering
99+
modelRefs:
100+
- model: llama3-8b
101+
use_reasoning: false
102+
103+
- name: health_route
104+
priority: 50
105+
rules:
106+
operator: OR
107+
conditions:
108+
- type: domain
109+
name: health
110+
modelRefs:
111+
- model: llama3-8b
112+
use_reasoning: false
113+
114+
# Social sciences and humanities
115+
- name: social_sciences_route
116+
priority: 40
117+
rules:
118+
operator: OR
119+
conditions:
120+
- type: domain
121+
name: psychology
122+
- type: domain
123+
name: economics
124+
- type: domain
125+
name: business
126+
modelRefs:
127+
- model: llama3-8b
128+
use_reasoning: false
129+
130+
- name: humanities_route
131+
priority: 40
132+
rules:
133+
operator: OR
134+
conditions:
135+
- type: domain
136+
name: history
137+
- type: domain
138+
name: philosophy
139+
- type: domain
140+
name: law
141+
modelRefs:
142+
- model: llama3-8b
143+
use_reasoning: false
144+
145+
# Default fallback route with lowest priority
46146
- name: default_route
47147
priority: 1
48148
rules:

0 commit comments

Comments
 (0)