Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 9, 2025

Feeds can lose their retrieval cron jobs due to various failures (cron issues, server restarts, database problems). This adds a lightweight, non-cron-based health check that ensures all feeds have scheduled retrieval jobs.

Implementation

  • Hook: Runs on init (priority 20) once per hour, throttled by timestamp
  • Batch processing: Checks 100 feeds per run (filterable via pf_feed_cron_check_batch_size)
  • Persistent state: Uses pf_feed_cron_check_offset to track position across runs
  • Auto-cycling: Resets offset to 0 when all feeds checked, enabling continuous monitoring

Example flow for 250 feeds

Hour 0: Check feeds 1-100   → offset: 100
Hour 1: Check feeds 101-200 → offset: 200
Hour 2: Check feeds 201-250 → offset: 0 (reset, cycle complete)
Hour 3: Check feeds 1-100   → offset: 100 (repeats)

Key decisions

  • Uses init hook instead of cron to avoid duplicate point of failure
  • Spreads work across page loads to prevent timeout/memory issues on large feed counts
  • Preserves existing schedules; only adds when missing
  • Filter allows tuning batch size for different hosting environments

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • https://api.github.com/repos/DaveChild/Text-Statistics/zipball/e83d5f82726db80e662ae305fce3b3c41299b4f5
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/Masterminds/html5-php/zipball/f5ac2c0b0a2eefca70b2ce32a5809992227e75a6
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/PHPCSStandards/PHP_CodeSniffer/zipball/2d1b63db139c3c6ea0c927698e5160f8b3b8d630
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/PHPCSStandards/composer-installer/zipball/4be43904336affa5c2f70744a348312336afd0da
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/doctrine/instantiator/zipball/c6222283fa3f4ac679f8b9ced9a4e23f163e80d0
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/firebase/php-jwt/zipball/8f718f4dfc9c5d5f0c994cdfd103921b43592712
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/fivefilters/readability.php/zipball/2fd58a6cc626124a05721efb9b3632e91783d487
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/mattwright/URLResolver.php/zipball/416039192cb6d9158bdacd68349bceff8739b857
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/paragonie/random_compat/zipball/996434e5492cb4c3edcb9168db6fbb1359ef965a
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/php-fig/http-message/zipball/cb6ce4845ce34a8ad9e68117c10ee90a29919eba
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/php-fig/log/zipball/d49695b909c3b7628b6289db5479a1c204601f11
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/phpstan/extension-installer/zipball/85e90b3942d06b2326fba0403ec24fe912372936
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/phpstan/phpstan/zipball/29201e7a743a6ab36f91394eab51889a82631428
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/sebastianbergmann/resource-operations/zipball/05d5692a7993ecccd56a03e40cd7e5b09b1d404e
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/sebastianbergmann/type/zipball/75e2c2a32f5e0b3aef905b9ed0b179b953b3d7c7
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/sebastianbergmann/version/zipball/c6c1022351a901512170118436c764e473f6de8c
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/K51SEl /usr/bin/composer install --quiet or/psr/log user.name (http block)
  • https://api.github.com/repos/simplehtmldom/simplehtmldom/zipball/3c87726400e59d8e1bc4709cfe82353abeb0f4d1
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/thephpleague/uri-interfaces/zipball/00e7e2943f76d8cb50c7dfdc2f6dee356e15e383
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)
  • https://api.github.com/repos/thephpleague/uri/zipball/09da64118eaf4c5d52f9923a1e6a5be1da52fd9a
    • Triggering command: /usr/bin/php8.3 /usr/bin/php8.3 -n -c /tmp/EQ0xmC /usr/bin/composer install --no-dev --quiet (http block)

If you need me to access, download, or install something from one of these locations, you can either:

Original prompt

This section details on the original issue you should resolve

<issue_title>Health check for feed-retrieval cron jobs</issue_title>
<issue_description>We should have a way of checking whether each feed has a retrieval cron job set. If the cron event doesn't exist, we should schedule it.

We already do something similar on 'save_post' with PressForward\Core\Schema\Feeds\set_up_feed_retrieval_cron_job(). We could use this same callback to do the check-and-set.

The architectural question is: how best to trigger this check? It needs to happen on a per-feed basis, and there could be hundreds of feeds. Using a cron job for it risks duplicating a point of failure: if cron is faulty on the installation, causing retrieval events to be lost, then a cron-based fix may also not work. WP ensures that certain cron events exist on every pageload, but this is likely to heavy when we need to check many feeds.

Here I would like a creative solution that will trigger retrieval-event checks regularly, but will do so in a lightweight way. Perhaps something like: on 'init', check against a stored timestamp; if more than an hour has passed since that timestamp, check all feeds. Or store a counter with the timestamp (like max ID) and then, every hour, check only 50 or 100 items and then bump the max_ID; when you reach the end, set it back to zero; in this way, you'd get a full check after a couple hours.

If you have a better idea, please provide a sketch.</issue_description>

Comments on the Issue (you are @copilot in this section)


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits December 9, 2025 19:15
- Implements check_feed_retrieval_cron_jobs() method in Feeds class
- Runs on 'init' hook once per hour
- Processes feeds in batches of 100 (filterable via pf_feed_cron_check_batch_size)
- Uses offset counter to track progress across multiple runs
- Schedules missing retrieval cron jobs for feeds
- Adds comprehensive test coverage

Co-authored-by: boonebgorges <246627+boonebgorges@users.noreply.github.com>
Co-authored-by: boonebgorges <246627+boonebgorges@users.noreply.github.com>
- Move timestamp update to after successful completion
- Rename $new_offset to $calculated_offset for clarity
- Store filter callback in variable for proper cleanup in tests

Co-authored-by: boonebgorges <246627+boonebgorges@users.noreply.github.com>
Copilot AI changed the title [WIP] Add health check for feed-retrieval cron jobs Add incremental health check for feed retrieval cron jobs Dec 9, 2025
Copilot AI requested a review from boonebgorges December 9, 2025 19:26
@boonebgorges boonebgorges marked this pull request as ready for review December 9, 2025 20:07
@boonebgorges boonebgorges merged commit ecc4288 into 5.9.x Dec 9, 2025
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants