@olivepossum @greenprophet add and cat would do different things, add is what the residual networks typically use. I usually just do it as x = x + x1
Also you have two nonlinearities back-to-back at the end there, a relu followed by a sigmoid, which I’m guessing is probably not what you want, unless for some reason you want your outputs to be between 0.5 and 1
1 Like
@mdo thanks for the clarification and to point to the two back-to-back nonlinearities, as you mentioned it’s not what I wanted.
Thanks!