Transfer functions ignore scripted activity

[zflo]

Meaning: I specify a neuron's activity at every iteration via net.setActivation(), but its output stays zero. Also the bias has no effect anymore. Looks like the neuron's "internal mechanisms" are completely switched off, as soon as I give it anything via script. Is that the case? And can someone please fix it?

(So far I used net.setOutput, but I would like to have a selectable transfer function afterwards, without inserting an additional neuron.)

[cybott]

Hi Flo,

this is due to the execution order or the neurons during an update. The
activation is calculated right before the output, which means that your
manually set activation is overwritten by the newly calculated
activation whenever the network is updated. Therefore, it is not
possible to set the activation directly via script, only the output.

What you can do is

• to set the bias instead of the activation.
• to call the transfer function manually: net.setOutput(neuron,
  net.applyTransferFunction(neuron, myActivity)); This will call the
  current transfer function of the neuron on the given myActivity and
  assign the result as the neuron output. This allows you to set the
  transfer function as usual in the editor.

Your observation concerning the zero activity despite a non-zero bias is
strange. Try the above strategies. If this does not help, then tell me
and I'll have a look at the code (but not before Sunday).

Chris

On 01/24/2014 07:31 PM, zflo wrote:

Meaning: I specify a neuron's activity at every iteration via
|net.setActivation()|, but its output stays zero. Also the bias has no
effect anymore. Looks like the neuron's "internal mechanisms" are
completely switched off, as soon as I give it anything via script. Is
that the case? And can someone please fix it?

(So far I used |net.setOutput|, but I would like to have a selectable
transfer function afterwards, without inserting an additional neuron.)

—
Reply to this email directly or view it on GitHub
#30.

[zflo]

it is not
possible to set the activation directly via script, only the output.

But I think it is. Both the field "activity" in the network editor and an activation plotter show the neuron's activity according to what I'm handing over in my script via net.setActivation(). So the neuron's activity does follow my script.

Your observation concerning the zero activity despite a non-zero bias is
strange.

Not zero activity, but zero output. That's why I speculated that the whole transfer function is switched off - neither the activity nor the bias have an effect on the output anymore. So the behaviour looks like

if ANY neuron property is overwritten via script
    set ALL neuron properties according to script
else
    set ALL neuron properties according to network functions
end

I like your second workaround suggestion. I'll do it that way once I really need the functionality.

[cybott]

It depends from where you use the net.setActivation(). If you have a
customized ScriptedActivationFunction, then obviously you are
overwriting the entire activation function with your own function. Thus,
you remove the calculation of the activation as sum of all inputs
(including bias). In this case, use the functions
setDefaultActivationFunction and applyDefaultActivationFunction (or
similar, couldn't look that up, but it is documented). If you use a
scripted activation function, you must make sure that you RETURN the
calculated neuron activation, instead of setting it with
net.setActivation(). The returned value will be set to the activation of
the neuron.

But if you are using net.setActivation from e.g. a scripted fitness
function, then the above should not be necessary, but then the behavior
should not be as described by you above.

On 01/24/2014 09:25 PM, zflo wrote:

it is not
possible to set the activation directly via script, only the output.

But I think it is. Both the field "activity" in the network editor and
an activation plotter show the neuron's activity according to what I'm
handing over in my script via |net.setActivation()|. So the neuron's
activity does follow my script.

Your observation concerning the zero activity despite a non-zero
bias is
strange.

Not zero activity, but zero output. That's why I speculated that the
whole transfer function is switched off - neither the activity nor the
bias have an effect on the output anymore. So the behaviour looks like

|if ANY neuron property is overwritten via script
set ALL neuron properties according to script
else
set ALL neuron properties according to network functions
end
|

I like your second workaround suggestion. I'll do it that way once I
really need the functionality.

—
Reply to this email directly or view it on GitHub
#30 (comment).

---

Dr. rer. nat. Christian Rempis

St. Leonhard Strasse 17
89346 Bibertal-Echlishausen

Mobil: 0151 50009753

[zflo]

But if you are using net.setActivation from e.g. a scripted fitness
function, then the above should not be necessary, but then the behavior
should not be as described by you above.

Yes, I'm running a universal script in parallel with the simulation. I'm sorry I haven't made that clear. And the behaviour IS as described. So, the neuron has a real activation (assigned by the script) which I can read out and plot and everything, but its transfer function (about which nothing is specified in the script) stays mute. Even if I choose a scripted (i.e. in the network editor) transfer function and set it to return 1.0; the output stays zero.