Skip to content

Conversation

@amannn
Copy link
Owner

@amannn amannn commented Dec 3, 2025

Notes

Changes


Note

Introduces codec-based message formats (built-in JSON/PO + custom), overhauls the useExtracted pipeline with file watching, consistent reference handling and pruning, and updates docs, tests, plugin loaders, examples, workflows, and dependencies.

  • Extractor/Core:
    • Add codec system (defineCodec) with built-in json/po codecs and support for custom formats; expose via next-intl/extractor.
    • Refactor extraction flow: new extractAll, CatalogManager improvements (metadata merge, reference dedupe/sort, pruning obsolete messages), robust error handling, and source watching via @parcel/watcher.
    • Update loaders (extractionLoader, catalogLoader) and format resolution (getFormatExtension, resolveCodec).
    • Expand tests extensively for JSON/PO/custom formats, references, races, and errors; add SourceFileWatcher.
  • Docs:
    • Revise useExtracted docs with Formats section (JSON/PO/Custom) and examples; minor plugin/config wording tweaks.
  • Examples:
    • Rename example ClientCounter; increase Playwright retries on CI.
  • Tooling/CI:
    • Adjust release/prerelease workflows (setup-node config, trusted publishers note).
    • Build: add CJS chunkFileNames in Rollup.
  • Deps:
    • Add @parcel/watcher; bump po-parser to v2; update Lerna and conventional-changelog packages.

Written by Cursor Bugbot for commit d43359c. This will update automatically on new commits. Configure here.

# Conflicts:
#	packages/next-intl/.size-limit.ts
#	packages/next-intl/__mocks__/react.tsx
#	packages/next-intl/src/navigation/shared/createSharedNavigationFns.tsx
#	packages/next-intl/src/react-server/index.test.tsx
#	packages/next-intl/src/server/react-server/RequestLocale.tsx
#	packages/next-intl/src/server/react-server/getConfig.tsx
In Next.js 15.3, [Turbopack config has become
stable](https://nextjs.org/blog/next-15-3#turbopack-configuration-in-nextconfigts-stable).
With this fix, the new option is used in order to avoid a deprecation
warning.
# Conflicts:
#	packages/next-intl/src/plugin/getNextConfig.tsx
…on APIs (#1922)

With #959, the middleware
already handled decoding of non-ASCII characters.

This allows you to define localized
[`pathnames`](https://next-intl.dev/docs/routing#pathnames) like so:

```tsx
import {defineRouting} from 'next-intl/routing';
 
export const routing = defineRouting({
  locales: ['en', 'ja'],
  defaultLocale: 'en',
  pathnames: {
    '/about': {
      'de': '/über-uns'
  }
}
```

Since Next.js automatically encodes incoming pathnames, this supports
incoming requests both for decoded pathnames (e.g. `/de/über-uns`), as
well as encoded ones (e.g. `/de/%C3%BCber-uns`).

One piece has been missing though: Pathnames returned from [navigation
APIs](https://next-intl.dev/docs/routing/navigation) should be turned
into an encoded form.

Now, `next-intl` handles this as well:

```tsx
import {Link, getPathname} from '@/i18n/navigation';

// href="/de/%C3%BCber-uns"
<Link href="/about" locale="de" />

// pathname = "/de/%C3%BCber-uns"
const pathname = getPathname({href: '/about', locale: 'de'});
```

This change brings the navigation APIs in line with [Google's
recommendation to encode non-ASCII
pathnames](https://developers.google.com/search/docs/crawling-indexing/url-structure).
@amannn
Copy link
Owner Author

amannn commented Dec 9, 2025

When there's a dot, it splits the message into msgctxt and msgid

@ceolinwill Ah yep, that's a great point! I've also discovered that namespaces were not supported by the custom PO codec. I've now fixed both of these bugs in #2163.

This also required a change in po-parser which removed some hardcoded logic from the shared library that was only relevant to the default PO codec. In turn, keys can now be persisted e.g. to msgctxt to avoid regenerating them during decode.

Could you give this another go? If this looks fine to you, I can publish po-parser@2 along with this PR.

My comment above about toJSONString turned out to be false alarm, I think we're good here.

@ceolinwill
Copy link
Contributor

keys can now be persisted e.g. to msgctxt to avoid regenerating them during decode.

@amannn Great idea.

The formatting issues I had before are gone. Running next build now correctly puts the id in msgctxt and the source in msgid.

In my custom codec I’d likely invert that: keep the id in msgid and put the source in msgctxt, since the source text is mainly useful for context when reading PO files. It feels more natural:

  • msgid → id
  • msgctxt → source/context

If the source message were placed in msgctxt by default, I probably wouldn’t need a custom codec at all.


I did notice a potential new issue but I'm not sure:

  1. I deleted all PO contents to start fresh with the new codec.
  2. next build generated the PO files correctly.
  3. I ran pnpm dev (which runs next dev across all apps). Everything looked fine.
  4. After navigating through a few pages, the 3rd page threw: Error: msgctxt is required.
  5. Checking the PO files, all msgctxt entries were gone and the id had been moved back to msgid.

Not sure yet if this is a real bug because I couldn't reproduce it again. I didn’t clear .next, so maybe some stale cache caused it. I’ll report back if it happens again.

@amannn
Copy link
Owner Author

amannn commented Dec 10, 2025

@ceolinwill Thanks for the feedback!

The formatting issues I had before are gone.

Nice! 👍

In my custom codec I’d likely invert that: keep the id in msgid and put the source in msgctxt, since the source text is mainly useful for context when reading PO files. It feels more natural:

  • msgid → id
  • msgctxt → source/context

Yeah, that's surely fine too!

The PO "spec" has some hints here:

  • msgid is described as "the original string" (that's also what some TMS interpret)
  • On the other hand, msgctxt is used to disambiguate messages with the same original string. But still, both allow arbitrary strings.

Many i18n libs like react-intl for example also allow you to define an explicit ID for certain cases, which can be helpful to disambiguate identical strings—which is also what next-intl offers. And then, msgid is no longer the "original string", but an actual ID. Also, in this case, msgctxt becomes more like a namespace. AFAIK, originally, GNU gettext wasn't really designed to work with any IDs at all, so it needs some interpretation.

Not sure yet if this is a real bug because I couldn't reproduce it again. I didn’t clear .next, so maybe some stale cache caused it. I’ll report back if it happens again.

Thanks! I also tried reproducing it in a project and investigated a bit, but no luck so far. If you encounter it again, definitely let me know—thanks!

@amannn amannn mentioned this pull request Dec 10, 2025
29 tasks
…2166)

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
@amannn
Copy link
Owner Author

amannn commented Dec 11, 2025

@ceolinwill I've updated this PR with a few fixes, some of them quite significant. It's only internal changes though.

This PR now also includes fixes for file references to be consistently updated and messages pruned when no longer used (both for builds as well as dev). Clearing .next or similar should surely not be necessary.

Would you mind giving this a quick sanity check in your app? That would be really helpful 🙏 Hopefully I can finally release to stable then, including the new feature for custom formats.

next-intl@0.0.0-canary-d4f4cde

@amannn amannn changed the title feat: Custom formats for useExtracted feat: Custom formats for useExtracted and consistency fixes for file references Dec 11, 2025
@amannn amannn changed the title feat: Custom formats for useExtracted and consistency fixes for file references feat: Custom formats for useExtracted and consistency fixes for file references & pruning of messages Dec 11, 2025
@amannn amannn changed the title feat: Custom formats for useExtracted and consistency fixes for file references & pruning of messages feat: Custom formats for useExtracted, consistency fixes for file references and pruning of messages Dec 11, 2025
@ceolinwill
Copy link
Contributor

@amannn Just tested it and everything looks great

  • Deleting a file correctly removed the translation
  • Renaming a file correctly updated the reference
  • Tested several scenarios with next dev running for all apps in the monorepo while also running next build, and everything behaved as expected

One promising note:

In this monorepo, I upgraded only one app to the new version and left another on the old version.

When I ran pnpm build at the root, the app still on the old version showed a diff where some translations changed order. The app on the new version did not show this issue, which is a good sign.

Great job!

@amannn
Copy link
Owner Author

amannn commented Dec 12, 2025

Nice, that's really good to hear! Ok, let's get this over the finish line then …

@amannn amannn changed the title feat: Custom formats for useExtracted, consistency fixes for file references and pruning of messages feat: Custom formats for useExtracted, consistency fixes for file references, pruning of messages and sorting of keys Dec 12, 2025
@amannn amannn merged commit c02818e into main Dec 12, 2025
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants