t/porting/deprecation.t: Cannot handle change in status of deprecation warnings #23987
Description
I'm having a problem in my work on pull request 23782: one porting test is FAILing. You can use this branch to examine the problem:
https://github.com/jkeenan/perl5/tree/goto-label-fatal-debugging-20251206
As you can see from this display of commits in the branch, the first commit is simply the current state of GH #23782, wherein all tests are PASSing.
The 2nd commit makes some documentation changes. All tests continue to PASS.
The 3rd commit makes more documentation changes. All tests continue to PASS.
Since the goal of this project is to change a deprecation warning into an exception, at some point we not only have to change some C-level code; we also have to update the database of warnings from which we ultimately generate lib/warnings.pm. The 4th commit in this branch removes the deprecation warning from regen/warnings.pl -- but that has to be immediately followed by make regen in order to update the generated files lib/warnings.pm and warnings.h. AFAICT these files are correctly regenerated. HOWEVER, if at this point I call:
$ make test_porting
... I get a failure in test 10 of t/porting/deprecated.t:
$ cd t;./perl harness -v porting/deprecation.t; cd -
ok 1 - Subject 'Unicode Delimiter Will be Paired' has a category specified
...
not ok 10 - Deprecated category 'deprecated::goto_construct' (Goto Block Construct) exists in ./regen/warnings.pl
ok 11 - Subject 'Goto Block Construct' has a category specified
ok 12 - Deprecated category 'deprecated::version_downgrade' (Downgrading a C<use VERSION> to below v5.11) exists in ./regen/warnings.pl
...
# Failed test 10 - Deprecated category 'deprecated::goto_construct' (Goto Block Construct) exists in ./regen/warnings.pl at porting/deprecation.t line 60
...
Failed 1/20 subtests
Test Summary Report
-------------------
porting/deprecation.t (Wstat: 0 Tests: 20 Failed: 1)
Failed test: 10
Files=1, Tests=20, 0 wallclock secs ( 0.01 usr 0.00 sys + 0.04 cusr 0.12 csys = 0.17 CPU)
Result: FAIL
Let's pause to take a look at how t/porting/deprecation.t works. It essentially traverses pod/perldeprecation.pod line-by-line, comparing its findings to a look-up table of data about anticipated deprecations.
23 my $pod_file = "./pod/perldeprecation.pod";
24 my $warnings_file = "./regen/warnings.pl";
25
26 do $warnings_file;
27 our $WARNING_TREE;
28
29 my $deprecated = $WARNING_TREE->{all}[1]{deprecated}[2];
The program runs another program found in the core distribution, regen/warnings.pl, which populates the hashref $WARNING_TREE. It extracts one particular part of that tree to populate hashref $deprecated. In blead (e278c37), that looks like this:
$deprecated
{
'deprecated::dot_in_inc' => [ '5.025011', 1 ],
'deprecated::subsequent_use_version' => [ '5.039008', 1 ],
'deprecated::delimiter_will_be_paired' => [ '5.03501', 1 ],
'deprecated::version_downgrade' => [ '5.035009', 1 ],
'deprecated::unicode_property_name' => [ '5.011003', 1 ],
'deprecated::missing_import_called_with_args' => [ '5.039002', 1 ],
'deprecated::goto_construct' => [ '5.011003', 1 ]
};
... which tells us that goto_construct was introduced during the 5.11
development cycle (hence, from the 5.12 production release forward).
When we run this test file in blead, we see 3 unit tests pertaining to the Goto Block Construct deprecation:
ok 10 - Deprecated category 'deprecated::goto_construct' (Goto Block Construct) exists in ./regen/warnings.pl
ok 11 - Version change for 'deprecated::goto_construct' (Goto Block Construct) is sufficiently after deprecation date
ok 12 - Subject 'Goto Block Construct' has a category specified
That is, one more test (Version change for ...) appears in blead than in this branch. Here's the first relevant stanza in t/porting/deprecation.t:
54 elsif (/^Category: "([::\w]+)"/) {
55 my $category = $1;
56 $category_seen{$category} = $removed_in_version;
57 $subject_has_category{$subject} = $category;
58 next if $removed_in_version eq "unscheduled";
59 my $tuple = $deprecated->{$category};
60 ok( $tuple, "Deprecated category '$category' ($subject) exists in $warnings_file")
61 or next;
62 my $added_in_version = $tuple->[0];
63 $added_in_version =~ s/(5\.\d{3})\d+/$1/;
64
65 my $diff = $removed_in_version - $added_in_version;
66 cmp_ok($diff, ">=", 0.004, # two production cycles
67 "Version change for '$category' ($subject) is sufficiently after deprecation date")
68 }
There's a successful pattern match at line 54, so deprecated::goto_construct is assigned to $category. $removed_in_version has previously been assigned 5.42. $deprecated->{$category} is a Perl-true value ([ '5.011003', 1 ]), so the ok( $tuple...) test at line 60 PASSes. Execution continues to the cmp_ok test at line 66, whih also PASSes. The next line to be processed (line 91 in pod/perldeprecation.pod) triggers a different stanza:
46 elsif (/^=head3 (.*)/) {
47 my $new_subject = $1;
48 if (!$in_legacy and $subject) {
49 ok($subject_has_category{$subject},
50 "Subject '$subject' has a category specified");
51 }
52 $subject = $new_subject;
53 }
Execution proceeds to line 49, where the ok($subject_has_category{$subject} test PASSes, so we get the third passing test for 'Goto Block Construct'.
In branch
In the goto-label-fatal-debugging-20251206 branch, however, this line:
'deprecated::goto_construct' => [ '5.011003', 1 ]
... has been removed from regen/warnings.pl. make regen removes data from lib/warnings.pm and from warnings.h as well. As t/porting/deprecation.t proceeds line-by-line through pod/perldeprecation.pod, it encounters:
Category: "deprecated::goto_construct"
deprecated::goto_construct is assigned to $category. $removed_in_version has previously been assigned 5.44. However, here $deprecated->{$category} is a Perl-false value so the ok( $tuple...) test at line 60 FAILs and the or next moves the program along to the next line in pod/perldeprecation.pod, thereby ensuring that the Version change test is never reached.
Impact
If we do the right thing and de-list a deprecated warning from regen/warnings.pl, then pod/perldeprecation.t is going to FAIL a test for that warning as it proceeds through the POD file.
The underlying problem, IMO, is that t/pod/perldeprecation.t is not a structured data file where the data within particular data records changes over time. It is a (POD-)structured documentation file. If we want to have a porting test which checks the status of unscheduled deprecations (or fatalizations), deprecations scheduled but not yet coded, deprecations scheduled and coded but not yet production-released and deprecations scheduled, coded and production-released, then that data needs careful structuring. That data needs to be stored in something like ... (dare I say it?) ... XML. Something like the maint-votes.xml file in which Perl 5 committers track the status of code proposed for backporting.
How should we proceed?