Added logging for No indices matched #6342
Conversation
Utkarsh-Aga
requested review from
KarstenSchnitter,
chenqi0805,
dinujoh,
dlvenable,
engechas,
graytaylor0,
kkondaka,
oeyh,
san81,
sb2k16 and
srikanthjg
as code owners
Utkarsh-Aga
force-pushed
the
improved-logging
branch
from
88e478b to
b4b5bbe
Compare
| } | ||
| } | ||
| if (matchCount == 0) { | ||
| LOG.info(NOISY, "No indices matched the configured regex patterns after applying include/exclude filters"); |
There was a problem hiding this comment.
Initially thought to have it at warn level, but then thought since the include_name_regex and exclude_name_regex are user configured, so this log line just provides a information that based on the filters there are no matching index, hence kept it at INFO level. I am open to suggestion, if we should move it to warn log ?
There was a problem hiding this comment.
@Utkarsh-Aga , Do you think there is particular value in running this without finding any indexes? We generally use WARN level for configuration problems in the pipeline. This seems like it would indicate that the user configured their pipeline incorrectly to me.
@graytaylor0 , Any thoughts on this?
There was a problem hiding this comment.
I do lean towards agreeing with @dlvenable here. It's unlikely someone would configure a pipeline that has no existing indexes matching the pattern on startup.
Technically, they could have a new index get created that matches the pattern after the pipeline starts, but we would then process an empty or near empty index since it's a one-time migration via point in time. Either way, if someone intends that, then WARN can't hurt. It is far more likely that someone misconfigures the pattern.
...h/dataprepper/plugins/source/opensearch/worker/OpenSearchIndexPartitionCreationSupplier.java
Show resolved
Hide resolved
|
@Utkarsh-Aga , Let's change the log level to WARN. You can keep the metrics as-is since we have another metric we can use here. |
Signed-off-by: Utkarsh Agarwal <126544832+Utkarsh-Aga@users.noreply.github.com>
Utkarsh-Aga
force-pushed
the
improved-logging
branch
from
2958051 to
b0d5ccf
Compare
|
Okay sure, I have updated the log level to warn |
3 participants
Description
Adds a logging in case there are no matching indices. Have added a Noisy marker also, as this log can appear multiple times due to Partition Creation
Issues Resolved
Resolves #6341
Check List
[ ] New functionality includes testing.[] New functionality has a documentation issue. Please link to it in this PR.[ ] New functionality has javadoc addedBy submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.