Skip to content

Conversation

@xnacly
Copy link
Owner

@xnacly xnacly commented Dec 26, 2025

This pr implements at least a better lexing approach and at most will combine lexing and parsing into a single in memory go value producer. Reducing the need for tokens and replacing the recursive descent parser with a dfa->go values system, hopefully this will outperform encoding/json by even more that the current 2.5-4x.

// converts any valid JSON to go values, result may be:
//
//	T = map[string]T, []T, string, float64, true, false, nil
//
// deserialize merges the concepts of lexical analysis with semantic and
// syntactical analysis, while producing direct go values out of the
// aforementioned. This results in a large performance improvements over the
// traditional approach, since the deserialisation process no longer requires
// multiple passes and intermediate values.
//
// At a high level this works by representing JSON as a table of states and
// possible input characters determining the follow state (table driven DFA).
// This makes deserialisation of large JSON inputs very fast.
func deserialize(src []byte) (any, error)

@xnacly xnacly changed the title Reengineering the lexer to be a dfa Reengineering the library to directly produce Go values instead of lexing and parsing beforehand Dec 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants