Question regarding Bias Implementation in Complex Layers #39
Description
I'm currently studying the details of CVNN implementations, referencing literature such as the comprehensive survey "Complex-Valued Neural Networks: A Comprehensive Survey" by Lee et al. (2022).
Based on the standard mathematical definition presented in such literature, a complex linear or convolution layer with complex weights W=Wr+iWi and a complex bias B=Br+iBi operating on a complex input X=Xr+iXi is defined such that the bias is added directly to the real and imaginary components of the output after the main linear/convolution operation
Re(Y) = X_r * W_r - X_i * W_i + B_r
Im(Y) = X_r * W_i + X_i * W_r + B_i
This means the real part of the output receives Br, and the imaginary part receives Bi, independently.
However, based on how complexPyTorch appears to be implemented internally (e.g., using underlying real-valued convolution/linear operations, potentially each with its own bias), the effective bias applied to the final complex output seems to be a combination of internal bias parameters, differing from the direct Br and Bi addition described above.
Re(Y) = X_r * W_r - X_i * W_i + B_r - B_i
Im(Y) = X_r * W_i + X_i * W_r + B_r + B_i
This leads to a clear difference in how the bias affects the real and imaginary parts of the output compared to the standard definition.
Could you please clarify the specific bias implementation in complexPyTorch layers (like ComplexConv2d, ComplexLinear)? Specifically:
- Is there a particular reason or design consideration for implementing bias addition in this manner, different from the standard Br,Bi addition as described in papers like Lee et al. (2022)?
- Are there any plans to potentially modify the bias implementation in the future to align more closely with the standard definition (Re(Y)←Yr+Br, Im(Y)←Yi+Bi)?
Understanding the details of this implementation difference and the reasons behind it would be very helpful for users of the library.
Thank you for your time and your work on this project.