Skip to content

Conversation

@blms
Copy link

@blms blms commented Dec 23, 2025

In this PR

  • Added a couple of notes to the readme about Poetry 2+ and npm/tailwind
  • Added documentation about loading fixtures, some RST cleanup
  • Cleaned up all_data.json fixtures to resolve ContentType and other issues (see notes)

Notes

Fixture cleanup:

  • I was getting errors about ContentTypes not existing, so I wrote and ran a quick python script to map ContentType.pk references to their names (natural key), as they are populated during migration but with mismatched PKs from the fixture.
  • I also got some errors about Permission instances, so I removed those from the fixture file; they are also populated via migrations
  • I was getting errors on LogEntry (and wagtail log entry) instances that referenced deleted models, so I just removed all of them from the fixture since they aren't needed for local dev
  • At a certain point wagtail indices (indexentry and referenceindex) were causing errors, so I excluded them because they can be regenerated with management commands
  • Finally, I just removed DB user sessions, since those are also not needed
Python script to clean fixtures
import json

with open("manuscript/fixtures/all_data.json") as all_data:
    data = json.load(all_data)

# map content_type_id --> natural key to prevent ContentType DoesNotExist on pk match
content_type_map = {}
for obj in data:
    if obj["model"] == "contenttypes.contenttype":
        pk = obj["pk"]
        fields = obj["fields"]
        content_type_map[pk] = [fields["app_label"], fields["model"]]

print(f"Found {len(content_type_map)} content types")

EXCLUDED_MODELS = [
    # wagtail indices
    "wagtailsearch.indexentry",
    "wagtailcore.referenceindex",
    # content types and permissions (populated by migrations)
    "contenttypes.contenttype",
    "auth.permission",
    # log entries (may refer to previously removed ContentTypes)
    "wagtailcore.modellogentry",
    "admin.logentry",
    # database user sessions
    "sessions.session",
]

fixtures = []
removed_fixtures = 0
remapped_contenttypes = 0
for obj in data:
    # remove fixtures for all excluded models from all_data
    if obj["model"] in EXCLUDED_MODELS:
        removed_fixtures += 1
        continue

    # remap content type to natural keys
    fields = obj.get("fields", {})
    if "content_type" in fields:
        ct_id = fields["content_type"]
        fields["content_type"] = content_type_map[ct_id]
        remapped_contenttypes += 1

    # add fixture
    fixtures.append(obj)

with open("manuscript/fixtures/all_data_portable.json", "w") as outfile:
    json.dump(fixtures, outfile, indent=4)

print(f"Removed {removed_fixtures} problematic fixtures")
print(f"Remapped {remapped_contenttypes} content_type_id references")

@blms blms marked this pull request as ready for review January 5, 2026 19:43
@blms blms requested review from hepplerj and qtrinh2 as code owners January 5, 2026 19:43
@blms blms merged commit c1a63cd into develop Jan 5, 2026
@blms blms deleted the blms/docs branch January 5, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants