| | |
- builtins.object
-
- Activation
class Activation(builtins.object) |
| |
This class contains various activation functions and their corresponding derivatives for use in neural networks.
Methods:
relu: Rectified Linear Unit activation function. Returns the input directly if it's positive, otherwise returns 0.
leaky_relu: Leaky ReLU activation function. A variant of ReLU that allows a small gradient when the input is negative.
tanh: Hyperbolic tangent activation function. Maps input to range [-1, 1]. Commonly used for normalized input.
sigmoid: Sigmoid activation function. Maps input to range [0, 1]. Commonly used for binary classification.
softmax: Softmax activation function. Maps input into a probability distribution over multiple classes.
|
| |
Static methods defined here:
- leaky_relu(z, alpha=0.01)
- Leaky ReLU activation function: f(z) = z if z > 0, else alpha * z.
Allows a small, non-zero gradient when the input is negative to address the dying ReLU problem.
- leaky_relu_derivative(z, alpha=0.01)
- Derivative of the Leaky ReLU function: f'(z) = 1 if z > 0, else alpha.
Returns 1 for positive input, and alpha for negative input.
- relu(z)
- ReLU (Rectified Linear Unit) activation function: f(z) = max(0, z).
Returns the input directly if it's positive, otherwise returns 0.
- relu_derivative(z)
- Derivative of the ReLU function: f'(z) = 1 if z > 0, else 0.
Returns 1 for positive input, and 0 for negative input.
- sigmoid(z)
- Sigmoid activation function: f(z) = 1 / (1 + exp(-z)).
Maps input to the range [0, 1], commonly used for binary classification.
- sigmoid_derivative(z)
- Derivative of the sigmoid function: f'(z) = sigmoid(z) * (1 - sigmoid(z)).
Used for backpropagation through the sigmoid activation.
- softmax(z)
- Softmax activation function: f(z)_i = exp(z_i) / sum(exp(z_j)) for all j.
Maps input into a probability distribution over multiple classes. Used for multiclass classification.
- tanh(z)
- Hyperbolic tangent (tanh) activation function: f(z) = (exp(z) - exp(-z)) / (exp(z) + exp(-z)).
Maps input to the range [-1, 1], typically used for normalized input.
- tanh_derivative(z)
- Derivative of the tanh function: f'(z) = 1 - tanh(z)^2.
Used for backpropagation through the tanh activation.
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
| |