Skip to content

Conversation

@egegunes
Copy link
Contributor

@egegunes egegunes commented Dec 16, 2025

K8SPSMDB-1451 Powered by Pull Request Badge

CHANGE DESCRIPTION

Problem:
Operator blindly waits for 5 seconds after running rs.initiate.

Solution:
Check if primary is elected with backoff retry.

CHECKLIST

Jira

  • Is the Jira ticket created and referenced properly?
  • Does the Jira ticket have the proper statuses for documentation (Needs Doc) and QA (Needs QA)?
  • Does the Jira ticket link to the proper milestone (Fix Version field)?

Tests

  • Is an E2E test/test case added for the new feature/change?
  • Are unit tests added where appropriate?
  • Are OpenShift compare files changed for E2E tests (compare/*-oc.yml)?

Config/Logging/Testability

  • Are all needed new/changed options added to default YAML files?
  • Are all needed new/changed options added to the Helm Chart?
  • Did we add proper logging messages for operator actions?
  • Did we ensure compatibility with the previous version or cluster upgrade process?
  • Does the change support oldest and newest supported MongoDB version?
  • Does the change support oldest and newest supported Kubernetes version?

@egegunes egegunes added this to the v1.22.0 milestone Dec 16, 2025
@egegunes egegunes requested a review from hors as a code owner December 16, 2025 12:24
Copilot AI review requested due to automatic review settings December 16, 2025 12:24
@pull-request-size pull-request-size bot added the size/M 30-99 lines label Dec 16, 2025

out := strings.Trim(stdout.String(), "\n")
if out != "true" {
return errors.New("is not the writable primary")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
return errors.New("is not the writable primary")
return errors.New(pod.Name+" is not the writable primary")

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves the replica set initialization process by replacing a blind 5-second wait after rs.initiate with an intelligent backoff retry mechanism that polls for primary election. The change makes the initialization process more reliable by actively checking when the primary is elected rather than assuming 5 seconds is always sufficient.

  • Replaces fixed 5-second sleep with a backoff retry mechanism that actively checks for primary election
  • Uses MongoDB's hello command to verify the node is a writable primary
  • Adds exponential backoff with 5 retry steps and configurable timing parameters

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mayankshah1607
mayankshah1607 previously approved these changes Dec 18, 2025
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings December 18, 2025 12:30
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings December 18, 2025 12:45
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@JNKPercona
Copy link
Collaborator

Test Name Result Time
arbiter passed 00:11:29
balancer passed 00:17:36
cross-site-sharded passed 00:17:48
custom-replset-name passed 00:10:53
custom-tls passed 00:14:26
custom-users-roles passed 00:10:47
custom-users-roles-sharded passed 00:11:42
data-at-rest-encryption passed 00:12:55
data-sharded passed 00:23:07
demand-backup passed 00:16:03
demand-backup-eks-credentials-irsa passed 00:00:07
demand-backup-fs passed 00:21:44
demand-backup-if-unhealthy passed 00:08:18
demand-backup-incremental passed 00:45:06
demand-backup-incremental-sharded passed 01:01:57
demand-backup-physical-parallel passed 00:08:08
demand-backup-physical-aws passed 00:11:47
demand-backup-physical-azure passed 00:11:43
demand-backup-physical-gcp-s3 passed 00:11:51
demand-backup-physical-gcp-native passed 00:11:54
demand-backup-physical-minio passed 00:20:25
demand-backup-physical-minio-native passed 00:20:17
demand-backup-physical-sharded-parallel passed 00:10:40
demand-backup-physical-sharded-aws passed 00:17:38
demand-backup-physical-sharded-azure passed 00:17:13
demand-backup-physical-sharded-gcp-native passed 00:17:05
demand-backup-physical-sharded-minio passed 00:17:09
demand-backup-physical-sharded-minio-native passed 00:17:02
demand-backup-sharded passed 00:24:55
expose-sharded passed 00:33:50
finalizer passed 00:09:54
ignore-labels-annotations passed 00:07:30
init-deploy passed 00:13:04
ldap passed 00:08:40
ldap-tls passed 00:12:56
limits passed 00:06:24
liveness passed 00:08:51
mongod-major-upgrade passed 00:12:55
mongod-major-upgrade-sharded passed 00:21:31
monitoring-2-0 passed 00:24:58
monitoring-pmm3 passed 00:29:00
multi-cluster-service passed 00:14:07
multi-storage passed 00:18:55
non-voting-and-hidden passed 00:17:04
one-pod passed 00:07:59
operator-self-healing-chaos passed 00:12:48
pitr passed 00:31:48
pitr-physical passed 01:01:11
pitr-sharded passed 00:22:09
pitr-to-new-cluster passed 00:25:24
pitr-physical-backup-source passed 00:54:20
preinit-updates passed 00:05:19
pvc-resize passed 00:12:30
recover-no-primary passed 00:26:16
replset-overrides passed 00:16:34
replset-remapping passed 00:08:42
replset-remapping-sharded passed 00:16:34
rs-shard-migration passed 00:13:53
scaling passed 00:11:03
scheduled-backup passed 00:16:43
security-context passed 00:07:21
self-healing-chaos passed 00:15:16
service-per-pod passed 00:18:23
serviceless-external-nodes passed 00:07:25
smart-update passed 00:08:44
split-horizon passed 00:07:59
stable-resource-version passed 00:04:39
storage passed 00:07:27
tls-issue-cert-manager passed 00:29:03
upgrade passed 00:09:54
upgrade-consistency passed 00:06:15
upgrade-consistency-sharded-tls passed 00:54:43
upgrade-sharded passed 00:19:41
upgrade-partial-backup failure 00:13:27
users passed 00:17:14
version-service passed 00:25:02
Summary Value
Tests Run 76/76
Job Duration 04:05:39
Total Test Time 22:17:46

commit: 65db8e0
image: perconalab/percona-server-mongodb-operator:PR-2149-65db8e00

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/M 30-99 lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants