Claude/research findings documentation 011 c uwu eq44fr4 bw xtysq uz w #6
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
P0-1: Temporal Tracking for Findings - Add migration v3: fingerprint, first_scan_id, status, verified, false_positive columns to findings table - Add migration v4: correlation_results table for persisting attack chains and insights - Update Finding struct with new temporal tracking fields - Implement generateFindingFingerprint() for SHA256-based deduplication - Implement checkDuplicateFinding() to detect cross-scan duplicates - Add FindingStatus constants: new, active, fixed, duplicate, reopened - Add CorrelationResult type for storing attack chain analysis - Enhanced SaveFindings to automatically fingerprint and detect duplicates - Update GetFindings query to include new temporal tracking columns - Improved logging with duplicate_count and status_counts Technical details: - Fingerprint based on: tool + type + title (normalized, lowercase, SHA256) - first_scan_id tracks when vulnerability was first detected - status field enables vulnerability lifecycle tracking - verified and false_positive flags for manual triage - Automatic duplicate detection with historical lookup Impact: ✓ Enables regression detection (fixed vulnerabilities that reappear) ✓ Enables deduplication across multiple scans ✓ Foundation for vulnerability lifecycle tracking ✓ Enables temporal analysis of security posture ✓ Supports "first seen" / "last seen" metrics P0-2: Remove All Emoji Usage - Removed emojis from all CLI output (CLAUDE.md compliance) - Fixed 40+ emoji occurrences across cmd/ directory - Replaced checkmarks (✓) with plain text - Replaced unicode symbols (⚡,👥,💳,🔗,etc.) with plain text - Maintained structured otelzap logging throughout Files modified: - atomic.go, auth.go, boileau.go, config.go, db.go - discover.go, hunt.go, logic.go, results.go, resume.go - root.go, scim.go, self.go, self_update.go, serve.go, workers.go Standards compliance: ✓ No emojis in user-facing output ✓ Professional, parseable CLI output ✓ All output uses structured logging where appropriate Database schema: - internal/database/migrations.go:75-142 - Migrations v3 & v4 - internal/database/store.go:651-693 - Fingerprinting logic - internal/database/store.go:763-857 - Enhanced SaveFindings - pkg/types/types.go:44-88 - Enhanced Finding and CorrelationResult types Related: #research-findings-documentation
P1: Persist Correlation Results to Database
Added database methods to save and retrieve correlation results (attack chains,
infrastructure correlations, temporal patterns) for historical analysis.
Changes:
- Updated ResultStore interface with correlation methods
- Implemented SaveCorrelationResults() with transaction support
- Implemented GetCorrelationResults() for scan-specific results
- Implemented GetCorrelationResultsByType() for cross-scan queries
Technical details:
- Saves to correlation_results table (created in migration v4)
- Handles JSONB marshaling for related_findings, attack_path, metadata
- Comprehensive logging with type_counts and severity_counts
- Transaction safety with rollback on error
- Efficient queries with severity, confidence, and date ordering
Usage:
```go
// Save correlation results after analysis
results := []types.CorrelationResult{
{
ID: "chain-123",
ScanID: "scan-456",
InsightType: "attack_chain",
Severity: types.SeverityCritical,
Title: "OAuth2 to SAML Cross-Protocol Attack Chain",
Confidence: 0.95,
RelatedFindings: []string{"finding-1", "finding-2"},
AttackPath: []map[string]interface{}{...},
},
}
store.SaveCorrelationResults(ctx, results)
// Query results
chains := store.GetCorrelationResults(ctx, scanID)
attackChains := store.GetCorrelationResultsByType(ctx, "attack_chain")
```
Impact:
✓ Attack chains now persisted for historical analysis
✓ Enables querying across multiple scans
✓ Foundation for ML training on vulnerability patterns
✓ Supports temporal analysis of security insights
Files modified:
- internal/core/interfaces.go:45-48 - Added correlation methods to ResultStore
- internal/database/store.go:1486-1854 - Implemented 3 correlation methods (369 lines)
Related: #research-findings-documentation
CRITICAL FIXES (P0): 1. Enhanced Fingerprinting Algorithm Problem: Previous fingerprinting used only tool+type+title, causing false duplicates when the same vulnerability type appeared in different endpoints. Example of bug: - XSS in /login?q= and XSS in /search?q= had same fingerprint - Would be incorrectly marked as duplicates Fix: Updated generateFindingFingerprint() to include target information - Extracts target from metadata["target"], metadata["endpoint"], or metadata["url"] - Falls back to parsing evidence for HTTP method + path or URL - Normalized fingerprint: tool:type:title:target Impact: ✓ Prevents false duplicate detection ✓ Unique fingerprints for same vuln type in different locations ✓ Better deduplication accuracy 2. Regression Detection Problem: No logic to detect when a "fixed" vulnerability reappears Fix: Enhanced checkDuplicateFinding() to detect regressions - Checks previous status of same fingerprint - If status was "fixed", marks new occurrence as "reopened" - Logs ERROR-level alert for regressions with full context Implementation: - checkDuplicateFinding now returns: (isDuplicate, firstScanID, previousStatus, error) - SaveFindings detects reopened status and logs regression - Enables temporal tracking of vulnerability lifecycle Impact: ✓ Automatic regression detection ✓ Critical alerts when fixed vulnerabilities return ✓ Security posture degradation visibility 3. Data Backfill Migration (v5) Problem: Existing findings had NULL fingerprints and status after migration v3 Fix: Migration v5 backfills existing data - Sets status='active' for all NULL status findings - Sets first_scan_id=scan_id for baseline temporal tracking - Documents fingerprint regeneration strategy Note: Fingerprints for old findings will be generated on next scan (Cannot backfill in SQL due to complex metadata/evidence parsing) Impact: ✓ Historical data becomes usable ✓ Temporal tracking works for existing findings ✓ Clean migration path for production deployments Technical changes: - internal/database/store.go:653-714 - Enhanced fingerprinting with target extraction - internal/database/store.go:716-757 - Regression detection in checkDuplicateFinding() - internal/database/store.go:845-880 - Updated SaveFindings to handle regressions - internal/database/migrations.go:143-182 - Migration v5 for data backfill Related: #research-findings-documentation
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.