Allow multiple whiskers

[paulsonnentag]

I wanted to experiment with multiple whiskers but creating the second whisker failed with the error:

Error[program:0:0]: "Cannot read property 'getContext' of null"

The problem seems to be that paper.get('supporterCanvas') only returns a reference to the canvas on the first call. I assume the reason for this is that you can call transferControlToOffscreen only once. Instead of just returning null I think it would be nicer if paper.get('supporterCanvas') would cache the reference to the canvas on the worker side and return the cached reference instead. I think this would be generally useful independent of the whisker issue. To improve handling multiple whiskers I would like to change the current implementation so only a single setInterval is created by paper.whenPointsAt().

[janpaul123]

Did this get fixed by #51 or not yet?

[paulsonnentag]

No this is a separate issue. The problem is the following line, where the whisker canvas is requested:

paperprograms/client/paper/entry.js

Line 125 in a35abb9

const supporterCanvas = await workerContext.paper.get('supporterCanvas', { id: 'whisker' });

If this is called twice the second time null is returned instead of a reference to the canvas. I quick fix would be to generate a unique id for each whisker canvas.

[paulsonnentag]

I thought about this a little bit more, and I want to propose some changes to the whiskers API.

First I want to illustrate our use case. We started to experiment with sound. In our demo, each paper represents an audio node. These can be chained together to form a chain which generates sound. We are using Tone.js for the sound generation/playback in the background.

Demo

watch video

Explanation

Each paper has a sound input and a sound output. Inside your paper program, you can define audio nodes and connect them to the input/output of the paper.

If two papers are placed beside each other the output of one paper is connected with the input of the other paper.

It's possible to have more complex programs on a single paper or split them up across multiple papers

Overall this allows to compose and combine different sound programs very naturally. For example, in the demo, it just works if you put multiple sound sources on the speaker or visualizer node.

API Changes

Creation / Removal of whiskers

Currently, the API is missing a way to remove whiskers again. I would like to add a whisker only if the paper has something connected to the output. The program can change audio nodes dynamically so it should be possible to remove whiskers again if there is no longer anything connected to the sound output.

var whisker = await paper.get('whisker', {
  direction,      // "up" (default), "down", "left", "right"
  whiskerLength,  // as fraction of the side (default 0.7)
  requiredData,   // array of data fields that must be present in the other paper
  paperNumber,    // paper number to do this for (default is own paper number)
});

// destroy
whisker.destroy();

Different whisker color

Papers can have additional whiskers besides the "audio output" whisker. In this example, the sound generator paper reads the frequency from another paper:

In this case, it would be nice if you could change the color of the whisker to differentiate them better. Instead of a color change, the "connected" state could be indicated by the moving dot animation, displaying only a static line if no program is connected.

var whisker = await paper.get('whisker', {
  direction: "up",
  color: "blue", // new color attribute (default: red)
  requiredData: [ "frequency" ]
});

Separate event handlers for placement and removal of papers

This isn't directly related to the sounds demo. But I think would be cleaner to have two separate events for when a paper is placed and when it's removed. This makes it easier to explain the API. It's not necessary to check for null and in the removed paper event handler we can also pass in the information about the paper which was removed.

whisker.onPaperPlaced(({paperNumber, paperObj}) => { /* do something */ })
whisker.onPaperRemoved(({paperNumber, paperObj}) => { /* do something */})

I know that's a lot of changes. I wanted to hear your input what you think. I hope my explanation was clear enough. I believe these changes will be useful for other use cases as well. Maybe there is also a completely different solution that I'm not seeing or there are current use cases which you could do with the current API and not with these changes.

[janpaul123]

Those changes roughly make sense to me! Quick thoughts:

1. What would happen to a blue whisker once it connects? (current whiskers have two colours, red and green; which one would the new color attribute replace)
2. Maybe instead of whisker.onPaperPlaced use whisker.on('paperPlaced') so you can make whisker an EventTarget?

[paulsonnentag]

1. My idea was to use the "moving dot" animation as a state indicator:
   • If the paper is not connected just a static line is projected
   • If the paper is connected the dot starts moving
2. I agree, makes more sense to use EventTarget here

[janpaul123]

Ah that's a good idea! Maybe we should drop the two different colours in the default case too and just use the moving dot animation? In any case, sounds good to me. 💯