Libuv callbacks only contain 1 protocol buffer message

[sheldonth]

Quoted

It is important to realize that since the message send is async, the callback may be invoked immediately after uv_async_send is called in another thread, or it may be invoked after some time. libuv may also combine multiple calls to uv_async_send and invoke your callback only once. The only guarantee that libuv makes is – The callback function is called at least once after the call to uv_async_send. If you have no pending calls to uv_async_send, the callback won’t be called. If you make two or more calls, and libuv hasn’t had a chance to run the callback yet, it may invoke your callback only once for the multiple invocations of uv_async_send. Your callback will never be called twice for just one event.

http://nikhilm.github.io/uvbook/threads.html

It currently uses uv_async_send on every callback to the static c++ callback for streaming data.

[sheldonth]

After some thought I see the possible solutions here as:

• collect protocol buffer messages in c++ and when the libuv calls back to the main thread to send data, instead of sending just one message, pack the entire queue of messages together into one large byte array and also send the offsets of each message. I.e. messages start at byte offset 0x72, 0x1af, etc. In javascript then seperate the byte array and parse the protocol buffer messages and emit the events.

or

• collect protocol buffers messages and after each libuv callback if there is another message queued up immediately tell libuv to fire again. There is no guartuntee a message queue would be fully drained in this approach, but it's a little simpler.