ReLU is a non-linear activation function that supports the accuracy of CNN, and without ReLU, the performance of CNN will deteriorate. ReLU is expressed by the following formula and graph.