can be any non-linear differentiable function like sigmoid, tanh, ReLU, etc. (commonly used in the deep learning community). Learning in neural networks is nothing but finding the optimum weight ...

These adjustable parameters, often called weights, are real numbers that can be seen as 'knobs' that define the input–output function of the machine. In a typical deep-learning system ...