Implement a Language Server and build an extension for VS Code

[brendanzab]

Currently Pikelet is implemented in a very traditional way - ie. it consumes a source string, does some processing, and spits something out. This isn't a very good approach when wanting to support interactive editing, however, but the more more modern, query-driven approaches to language implementation are still sparsley documented.

One way to make some of the requirements for this more concrete is to try our hands at implementing a language server communicating via the LSP to a VS Code extension. This could help direct these efforts, before we accumulate too much tech debt!

Plan

1. create a skeleton language extension for VS Code
2. implement a syntax highlighter
3. build a rudimentary language server using languageserver-types and codespan-lsp, possibly upstreaming helpful stuff to codespan. Others are currently working on similar problems (which is a good thing!), so we should coordinate our work on the codespan Gitter and on Support for the Language Server Protocol brendanzab/codespan#8.
4. Add integration tests for the language server
   • eg. tests/language_server.rs
   • eg. rust-analyzer/crates/ra_lsp_server/tests/heavy_tests

TODO: Improve this plan!

Resources

• Language Extension Guidelines
• Example - Language Server
• Language Server Protocol Specification
• Pikelet Book: Implementation
• Current Rust LSPs:
  • https://github.com/gluon-lang/gluon_language-server
  • https://github.com/rust-lang-nursery/rls
  • https://github.com/wycats/wasm
  • https://github.com/lark-exploration/lark/tree/master/components/lark-language-server
• Other interesting projects
  • https://github.com/leanprover/lean

[brendanzab]

I've started messing around with an extension with basic syntax highlighting on my vscode-extension branch. Not sure about whether we'd want to split this out into a separate repo though.

[brendanzab]

I'm thinking that it might be good to put the extension and client in a separate repo. This might make it easier to version it independently. But I think we should keep the language server implementation in the main Pikelet repo. Feel free to disagree though, I'm curious to hear what people think!

[brendanzab]

rust-analyzer seems to put the editor support under an editors directory

[brendanzab]

Here's a nice example of a language server integration test: lark-exploration/lark#98

[sophiajt]

@brendanzab thanks for the shoutout -- still trying to figure out a good approach. If you come up with improvements on it, do pass them along 😄

[brendanzab]

Oh! Sorry for the ping @jonathandturner, but I guess I'm glad it was appreciated! 😅

I've been plugging away at my language server on this branch, you may-or-may not find it interesting. connection.rs is interesting, I've been trying to do a clean job of it, with a view to split it out eventually into some sort of crate.

This also links in with #175, which I'm currently mulling over... I've been trying to figure out how Lark is tackling its top level API, but been having a bit of trouble navigating around it, I'm guessing things are still in a bit of flux?

[sophiajt]

I'm guessing things are still in a bit of flux?

Hehe, yeah definitely. We've got some ideas, and we're just toying around, really.

[brendanzab]

If you come up with improvements on it, do pass them along

@jonathandturner What is the best way to contact you in the future? I have a number of Gitter channels:

• https://gitter.im/pikelet-lang/Lobby
• https://gitter.im/codespan-rs/Lobby

I'm also on the /r/ProgrammingLanguages discord and IRC, the Rust discords and on Twitter. 🤔

[sophiajt]

I'm generally on twitter pretty regularly, though I don't post often these days.