thank you for your contribution. I test the caffemodel of which the weights and bias were converted from tensorflow model. result from each layer was verified as below
caffe(fp32) : h0 h1 h2 ps caffe_output_tanh
Tensorflow(fp32): tf_h0 tf_h1 tf_h2 tf_output_tanh

indeed, there are still some different between caffe and TF, but i think it may not cause by python layer. I think the designed ps layer is right!