Skip to content

Commit 817c3de

Browse files
theakshaypanttheakshaypant
committed
feat(error-detection): add multiple pattern support
Add support for configuring multiple error detection regex patterns at both global (ConfigMap) and repository (CR) levels. This allows users to match different error formats from various linters and tools in a single pipeline. Global Configuration (ConfigMap): - Changed error-detection-simple-regexp to support arrays - Supports 3 formats: single pattern (backward compatible), multi-line YAML, and JSON array Repository CR: - Added ErrorDetectionSettings with patterns array and max_number_of_lines - Patterns are additive with global patterns - Per-repository max_number_of_lines override Jira: https://issues.redhat.com/browse/SRVKP-7237 Signed-off-by: Akshay Pant <akshay.akshaypant@gmail.com> Assisted-by: Cursor <noreply@cursor.com>
1 parent 4d75ddc commit 817c3de

File tree

10 files changed

+426
-59
lines changed

10 files changed

+426
-59
lines changed

‎config/300-repositories.yaml‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -446,6 +446,43 @@ spec:
446446
- roles
447447
- secret_ref
448448
type: object
449+
error_detection:
450+
description: |-
451+
ErrorDetection configures error detection for this repository. Error detection scans
452+
container logs for error patterns and creates annotations (currently GitHub-only).
453+
The global error-detection-from-container-logs setting must be enabled for this to work.
454+
If not specified, uses only global error detection pattern.
455+
properties:
456+
enabled:
457+
description: |-
458+
Enabled controls whether error detection is active for this repository.
459+
This field is required when error_detection settings are specified.
460+
Set to true to enable or false to disable error detection for this repository.
461+
type: boolean
462+
max_number_of_lines:
463+
description: |-
464+
MaxNumberOfLines specifies how many lines to scan from the end of container logs
465+
when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
466+
Higher values may increase memory usage. Use -1 for unlimited.
467+
If not specified, uses the global setting (default: 50).
468+
minimum: -1
469+
type: integer
470+
patterns:
471+
description: |-
472+
Patterns is an array of regular expressions used to detect errors in container logs.
473+
Each pattern must use named groups to capture: filename, line, and error.
474+
The column group is optional. Example pattern:
475+
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
476+
477+
Multiple patterns can be specified to match different error formats.
478+
Repository-specific patterns are tried first, followed by global patterns.
479+
If not specified or empty, only the global error-detection-simple-regexp patterns are used.
480+
items:
481+
type: string
482+
type: array
483+
required:
484+
- enabled
485+
type: object
449486
github:
450487
properties:
451488
comment_strategy:

‎config/302-pac-configmap.yaml‎

Lines changed: 13 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -101,7 +101,19 @@ data:
101101
# memory usage. Use -1 for unlimited lines.
102102
error-detection-max-number-of-lines: "50"
103103

104-
# The default regexp used when we use the simple error detection
104+
# The default regexp(s) used for simple error detection.
105+
# Supports multiple formats for backward compatibility:
106+
#
107+
# 1. Single pattern (backward compatible):
108+
# error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
109+
#
110+
# 2. YAML list format (multiple patterns, one per line):
111+
# error-detection-simple-regexp: |-
112+
# ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
113+
# ^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
114+
# ^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
115+
#
116+
# Each pattern must include named groups: filename, line, and error (column is optional)
105117
error-detection-simple-regexp: |-
106118
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
107119

‎docs/content/docs/guide/repositorycrd.md‎

Lines changed: 112 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -185,6 +185,118 @@ spec:
185185
comment_strategy: "disable_all"
186186
```
187187

188+
## Error Detection
189+
190+
Error detection scans container logs for error patterns and creates inline
191+
annotations on Pull Requests. This feature is currently **only supported for
192+
GitHub Apps**.
193+
194+
By default, error detection uses the global pattern configured in the
195+
`pipelines-as-code` ConfigMap via the `error-detection-simple-regexp` setting.
196+
However, you can customize error detection patterns on a per-repository basis
197+
using the Repository CR.
198+
199+
### Configuring Error Detection Patterns
200+
201+
You can specify multiple regex patterns to detect different error formats in
202+
your repository:
203+
204+
```yaml
205+
apiVersion: "pipelinesascode.tekton.dev/v1alpha1"
206+
kind: Repository
207+
metadata:
208+
name: my-repo
209+
spec:
210+
url: "https://github.com/owner/repo"
211+
settings:
212+
error_detection:
213+
enabled: true
214+
patterns:
215+
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
216+
- "^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)"
217+
max_number_of_lines: 100
218+
```
219+
220+
**Pattern Requirements:**
221+
222+
Each pattern must use [named groups](https://www.regular-expressions.info/named.html) to capture:
223+
224+
- `filename`: The file path where the error occurred
225+
- `line`: The line number
226+
- `error`: The error message
227+
- `column`: (optional) The column number
228+
229+
**Configuration Options:**
230+
231+
- `enabled`: **Required** boolean flag to enable or disable error detection for this
232+
repository. Set to `true` to enable or `false` to disable error detection.
233+
This field must be explicitly specified when configuring error detection settings.
234+
- `patterns`: Array of regex patterns. Repository-specific patterns are tried
235+
first, followed by global patterns. If not specified or empty, only the
236+
global patterns are used. **Note:** Providing an empty array does not disable
237+
error detection; it falls back to using only the global patterns defined in
238+
the `pipelines-as-code` ConfigMap.
239+
- `max_number_of_lines`: Number of log lines to scan (overrides global
240+
setting). Default is 50. Use -1 for unlimited.
241+
242+
### Examples
243+
244+
**Multiple error formats:**
245+
246+
```yaml
247+
spec:
248+
settings:
249+
error_detection:
250+
enabled: true
251+
patterns:
252+
# Standard format (make, gcc, eslint, etc.)
253+
- "^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)"
254+
# Python traceback format
255+
- 'File "(?P<filename>[^"]+)", line (?P<line>[0-9]+).*\n.*(?P<error>.*)'
256+
# Custom CI format
257+
- "^\\[(?P<filename>[^\\]]+)\\]:(?P<line>[0-9]+) - (?P<error>.*)"
258+
max_number_of_lines: 200
259+
```
260+
261+
**Disabling error detection for a repository:**
262+
263+
You can explicitly disable error detection for a specific repository, even if
264+
it's enabled globally:
265+
266+
```yaml
267+
spec:
268+
settings:
269+
error_detection:
270+
enabled: false
271+
```
272+
273+
**Using only global patterns:**
274+
275+
If you want to use only the global patterns defined in the `pipelines-as-code`
276+
ConfigMap with custom settings, provide an empty `patterns` array:
277+
278+
```yaml
279+
spec:
280+
settings:
281+
error_detection:
282+
enabled: true
283+
patterns: [] # Uses only global patterns
284+
max_number_of_lines: 100 # Can still override line count
285+
```
286+
287+
If you don't need to customize error detection at all, simply omit the
288+
entire `error_detection` field and the global settings will be used.
289+
290+
{{< hint info >}}
291+
**Pattern Priority:** When repository-specific patterns are defined, they are
292+
tried first for each log line. If no repository pattern matches, the global
293+
patterns are then tried. This allows you to add repository-specific patterns
294+
while still benefiting from the global patterns as a fallback.
295+
{{< /hint >}}
296+
297+
For more information about error detection and log snippets, see the
298+
[Status documentation]({{< relref "/docs/guide/statuses.md" >}}).
299+
188300
## Concurrency
189301

190302
`concurrency_limit` allows you to define the maximum number of PipelineRuns running at any time for a Repository.

‎docs/content/docs/guide/statuses.md‎

Lines changed: 12 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -84,8 +84,10 @@ You can customize the regular expression used for detecting errors with the
8484
`error-detection-simple-regexp` setting. The regular expression uses [named
8585
groups](https://www.regular-expressions.info/named.html) to provide flexibility
8686
in specifying the matching criteria. The necessary groups for matching are
87-
filename, line, and error (the column group is not used). The default regular
88-
expression is defined in the configuration map.
87+
filename, line, and error (the column group is optional). The default regular
88+
expression is defined in the configuration map. Multiple patterns are supported
89+
using multi-line YAML or JSON array format, allowing you to detect errors from
90+
different tools with various output formats.
8991

9092
By default, Pipelines-as-Code searches for errors in only the last 50 lines of
9193
the container logs. However, you can increase this limit by setting the
@@ -94,6 +96,14 @@ system will search through all available lines for errors. Keep in mind that
9496
increasing this maximum number of lines may increase the memory usage of the
9597
watcher.
9698

99+
{{< hint info >}}
100+
**Repository-level configuration:** You can also configure error detection
101+
patterns on a per-repository basis using the Repository CR. This allows you to
102+
define multiple patterns and customize settings for individual repositories.
103+
See the [Repository CR documentation]({{< relref "/docs/guide/repositorycrd.md#error-detection" >}})
104+
for more details.
105+
{{< /hint >}}
106+
97107
![annotations](/images/github-annotation-error-failure-detection.png)
98108

99109
## Namespace Event stream

‎docs/content/docs/install/operator_installation.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,7 @@ spec:
4545
hub-url: 'https://artifacthub.io'
4646
hub-catalog-type: 'artifacthub'
4747
error-detection-max-number-of-lines: '50'
48+
# Single pattern example. For multiple patterns, use multi-line format (see settings docs)
4849
error-detection-simple-regexp: >-
4950
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+):([
5051
]*)?(?P<error>.*)

‎docs/content/docs/install/settings.md‎

Lines changed: 31 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -279,6 +279,10 @@ A few settings are available to configure this feature:
279279
and expose them as annotations on Pull Request.
280280

281281
Only GitHub apps is supported.
282+
283+
**Note:** Individual repositories can override this setting using the `error_detection.enabled`
284+
field in the Repository CR. See the [Repository CRD documentation]({{< relref "/docs/guide/repositorycrd.md#error-detection" >}})
285+
for more details.
282286

283287
* `error-detection-max-number-of-lines`
284288

@@ -291,6 +295,33 @@ A few settings are available to configure this feature:
291295
By default the error detection only support a simple output, the way GCC or
292296
Make will output error, which is supported by most linters and command line tools.
293297

298+
**Multiple Patterns Support:** You can now specify multiple regex patterns to
299+
match different error formats. The setting supports two formats:
300+
301+
1. **Single pattern** (backward compatible):
302+
303+
```yaml
304+
error-detection-simple-regexp: '^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)'
305+
```
306+
307+
2. **Multi-line YAML** (for multiple patterns):
308+
309+
```yaml
310+
error-detection-simple-regexp: |-
311+
^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
312+
^ERROR: (?P<filename>[^ ]+) line (?P<line>[0-9]+): (?P<error>.*)
313+
^\[(?P<filename>[^\]]+)\]:(?P<line>[0-9]+) - (?P<error>.*)
314+
```
315+
316+
Each pattern will be tried in order until one matches. This allows you to detect
317+
errors from multiple tools with different output formats.
318+
319+
**Pattern Requirements:** Each pattern must use regexp named groups to capture:
320+
* `(?P<filename>...)` - The file path where the error occurred
321+
* `(?P<line>...)` - The line number
322+
* `(?P<error>...)` - The error message
323+
* `(?P<column>...)` - Column number (optional)
324+
294325
An example of an error that is supported is :
295326

296327
```console

‎pkg/apis/pipelinesascode/v1alpha1/types.go‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -164,6 +164,13 @@ type Settings struct {
164164
// AIAnalysis contains AI/LLM analysis configuration for automated CI/CD pipeline analysis.
165165
// +optional
166166
AIAnalysis *AIAnalysisConfig `json:"ai,omitempty"`
167+
168+
// ErrorDetection configures error detection for this repository. Error detection scans
169+
// container logs for error patterns and creates annotations (currently GitHub-only).
170+
// The global error-detection-from-container-logs setting must be enabled for this to work.
171+
// If not specified, uses only global error detection pattern.
172+
// +optional
173+
ErrorDetection *ErrorDetectionSettings `json:"error_detection,omitempty"`
167174
}
168175

169176
type GitlabSettings struct {
@@ -184,6 +191,33 @@ type GithubSettings struct {
184191
CommentStrategy string `json:"comment_strategy,omitempty"`
185192
}
186193

194+
// ErrorDetectionSettings configures how errors are detected from container logs and
195+
// exposed as annotations on Pull Requests. Currently only supported for GitHub Apps.
196+
type ErrorDetectionSettings struct {
197+
// Enabled controls whether error detection is active for this repository.
198+
// This field is required when error_detection settings are specified.
199+
// Set to true to enable or false to disable error detection for this repository.
200+
Enabled bool `json:"enabled"`
201+
// Patterns is an array of regular expressions used to detect errors in container logs.
202+
// Each pattern must use named groups to capture: filename, line, and error.
203+
// The column group is optional. Example pattern:
204+
// ^(?P<filename>[^:]*):(?P<line>[0-9]+):(?P<column>[0-9]+)?([ ]*)?(?P<error>.*)
205+
//
206+
// Multiple patterns can be specified to match different error formats.
207+
// Repository-specific patterns are tried first, followed by global patterns.
208+
// If not specified or empty, only the global error-detection-simple-regexp patterns are used.
209+
// +optional
210+
Patterns []string `json:"patterns,omitempty"`
211+
212+
// MaxNumberOfLines specifies how many lines to scan from the end of container logs
213+
// when looking for errors. This overrides the global error-detection-max-number-of-lines setting.
214+
// Higher values may increase memory usage. Use -1 for unlimited.
215+
// If not specified, uses the global setting (default: 50).
216+
// +optional
217+
// +kubebuilder:validation:Minimum=-1
218+
MaxNumberOfLines *int `json:"max_number_of_lines,omitempty"`
219+
}
220+
187221
func (s *Settings) Merge(newSettings *Settings) {
188222
if newSettings.PipelineRunProvenance != "" && s.PipelineRunProvenance == "" {
189223
s.PipelineRunProvenance = newSettings.PipelineRunProvenance
@@ -197,6 +231,9 @@ func (s *Settings) Merge(newSettings *Settings) {
197231
if newSettings.AIAnalysis != nil && s.AIAnalysis == nil {
198232
s.AIAnalysis = newSettings.AIAnalysis
199233
}
234+
if newSettings.ErrorDetection != nil && s.ErrorDetection == nil {
235+
s.ErrorDetection = newSettings.ErrorDetection
236+
}
200237
}
201238

202239
type Policy struct {

0 commit comments

Comments
 (0)