“In the early days of neural networks, sigmoid and tanh were the common activation functions with two important characteristics — they are smooth, differentiable functions with a range between [0,1] ...