Async chat #164
Async chat #164
Conversation
added new print_debug_threaded function that will work in threads. It expects that `update_debug_variables` will be called before to update python global variables based on vim configuration. We can't access vim variables from the thread.
|
Hi, thank you for your work! It is going to be a big thing and I think it needs some more testing and tuning. It might also require to change provider contract a little bit, so it might break some things. I wonder if it would be possible to keep the current functionality and add an option that would enable async chat. This way it would be much safer to roll out |
|
Good point. I think I could relatively easily add new vim variable that could toggle between old/new behavior. This way, even other providers that expect to run in the same thread will still work fine with the sync path. You could then roll it out gradually, first default to sync, then switch default to async, but keep the option to switch back to sync. |
|
Done, you can take a look if you like the approach, I would still like to keep testing for a couple of days. |
|
One thing I noticed while testing, undo history is cluttered in the async chat and that also breaks I have also pushed some small changes, hope I didn't break anything. |
|
Busy with other things, but hopefully will will get to this soon again. |
2 participants
This changes behavior of AIChat to be asynchronous...
As written in #39, I am only changing the AIChat functionality, I do not think this would be too helpful to have it also elsewhere.
I took approach of moving the request call into a thread on the chat.py level. And then I added an async timer in vimscript that is trying to pickup any data coming out from the thread and append them at the end of the buffer.
The main tradeoff here is that in the call to
provider.request(messages)you can't access anything from vim anymore as it is not in the main thread.That means that I have to move the apikey and orgid acquisition from the request to the init part, which is IMHO fine. And the
print_debugis also touching vim variables, so I will have to deal somehow with that as well.Other than that, I have added a "<<< thinking ..." line with changing number of dots when the request is happening in the background so user knows that something is happening.
Also in case of exception in the
provider.request(messages), I just dump that into chat as<<< Error getting responsesegment.I am creating the pull request to gather some feedback, if you do not have problems with the approach I have taken, I would give it one more a week at least of usage by myself to see if I run into some problems.