Increase Airbyte emitted at timestamp precision from milliseconds to nanoseconds for improved event ordering

[lahdirakram]

Summary

This change increases the precision of the Airbyte emitted_at timestamp from milliseconds to nanoseconds when emitting records from the PlanetScale Airbyte source.

This enhancement ensures that each record receives a more accurate and unique emission timestamp, improving event ordering and temporal consistency — especially during high-throughput syncs.

Problem

Previously, the emitted_at field used millisecond precision, meaning multiple records emitted within the same millisecond would share identical timestamps.
This led to:

• Ambiguous ordering in destination systems
• Possible reordering or race conditions during CDC replay
• Reduced accuracy for latency tracking and audit metrics

Solution

The timestamp precision has been increased to nanoseconds, providing distinct emitted_at values even when records are emitted in rapid succession.
This allows downstream systems and destinations to order events more accurately and deterministically.

Implementation Details

• Updated timestamp generation to use time.Now().UnixNano().
• No change to the structure or semantics of the emitted_at field — only its numeric precision increased.
• The timestamp remains an integer epoch-based value (Unix time in nanoseconds).

⚠️ Breaking Change Notice

This modification may be considered a breaking change for any destination or downstream system that:

• Assumes emitted_at is in milliseconds
• Performs arithmetic or formatting on the timestamp using the old precision
• Uses integer division or fixed-width timestamp parsing logic

Affected systems may need to update their parsing logic to handle nanosecond precision values correctly.

Impact

• ✅ Improved deterministic event ordering
• ✅ Higher temporal resolution for CDC streams
• ✅ Backward-compatible with most destinations that store timestamps as numeric values
• ⚠️ Potential breaking change for custom destinations assuming millisecond precision

Example

Old precision	New precision
emitted_at: 1730312345123	emitted_at: 1730312345123456789

---

Testing

This change was tested by performing multiple update operations within the same SQL transaction on a PlanetScale database.
In the resulting Airbyte output, all emitted records had distinct emitted_at values, confirming that nanosecond precision successfully differentiates events that previously shared identical timestamps.

[maxenglander]

this appears to be a violation of the airbyte protocol, which says emitted_at should be milliseconds:

https://docs.airbyte.com/platform/1.8/understanding-airbyte/airbyte-protocol