Rewrite parser logic with C# #71
Conversation
|
@FabienTschanz Wow I bet this one is leaps and bounds better than the current parser but we really got to give some time to test it, in fact even if it works wonders I'd kinda consider it as "breaking change" as far as m365dsc goes, the same way that the powershell graph sdk team is making changes around WAM but also the module loading priority issues and we cannot update the graph version in m365dsc because of that. can this be put on hold at least for a couple of weeks? My solution depends 100% on DSCParser (and ReverseDSC as well) and whenever it deviates even if just so slightly everything breaks on my side. Not saying it should not go ahead, just asking for some time to test it, I'm waiting for a new M365DSC version so that I can cross a major version and can get rid of the several patches I have on my side and after that I'll check this one out. |
|
No problem with waiting from my side. I did test it with full exports of all workloads with the most complex and nested configurations you can achieve. This version also fixes quite some issues with indentation and deeply nested objects when using I have similar optimizations running and under development for M365DSC. All of the changes here are also in a private ADO repo which I'm developing to use for my company, and since I started developing this version around Christmas, it was getting better and better. Right now, I have no issues left to solve from my side. If you discover any issue once you start testing it, let me know how I can reproduce them and I will gladly try to fix them. Using the AST with its strongly typed code, I can debug the parsed configuration way easier than before and it also performs way faster. My 20k lines configuration is parsed in 4-6 seconds, depending on Windows PowerShell vs PowerShell 7. PowerShell 7 is a little bit slower because |
|
And with my automated export and report pipeline, I discovered so many issues with M365DSC (and the other modules) that I almost can't keep up with fixing them. I guess you saw the number of PRs I put up in the M365DSC repo, and I'm still not done. Quite some more things to come, including new development things. My priority is directed to making the modules faster and much more stable since we want to use them for our customers very soon. |
|
For sure, I actually also have an exporter script as part of my pipelines and after that another script which uses ConvertTo-DSCObject and whenever there's a misbehaving resource which breaks the pipeline my customers complain about it, just had one that reported a problem that actually is already fixed in M365DSC latest version but I still have my Frankenstein version so I had to apply yet another workaround and hate doing that so I just hope Nik releases a new version without this patch so I have time to look into it. Then I have yet another script with my own version of ConvertFrom-DSCObject (because I first need to convert Markdown policies to PowerShell objects) which gets all indentations right so they all look "right" comparing to what M365DSC exports :) |
|
I guess we're the masters of "patching" our stuff together so it just works 😆 |
2 participants
This PR completely rewrites the parser logic with C# for improved type handling, memory handling and performance. It also gets rid of the CIM instance instantiation, which was previously used for checking the underlying resource state. This is now no longer a part of the module.
What currently is not being ported over is the metadata enrichment functionality. During implementation, I was very confused how it actually worked and gave up after several attempts.
Some figures: The new logic is massively quicker to convert from and to dsc objects. Even without CIM instantiations, the parsing is between 2 and 5 times faster, and the moment CIM instantiations (with elevated privileges in PowerShell) are required, it cannot be compared anymore. The bigger the configuration to parse, the greater the performance benefits.