| | |
- builtins.object
-
- CuPyActivation
- CuPyDenseLayer
class CuPyActivation(builtins.object) |
| |
Activation functions for neural networks using CuPy.
|
| |
Static methods defined here:
- leaky_relu(z, alpha=0.01)
- Leaky ReLU activation function: f(z) = z if z > 0, else alpha * z.
Allows a small, non-zero gradient when the input is negative to address the dying ReLU problem.
- leaky_relu_derivative(z, alpha=0.01)
- Derivative of the Leaky ReLU function: f'(z) = 1 if z > 0, else alpha.
Returns 1 for positive input, and alpha for negative input.
- relu(z)
- ReLU (Rectified Linear Unit) activation function: f(z) = max(0, z).
Returns the input directly if it's positive, otherwise returns 0.
- relu_derivative(z)
- Derivative of the ReLU function: f'(z) = 1 if z > 0, else 0.
Returns 1 for positive input, and 0 for negative input.
- sigmoid(z)
- Sigmoid activation function: f(z) = 1 / (1 + exp(-z)).
Maps input to the range [0, 1], commonly used for binary classification.
- sigmoid_derivative(z)
- Derivative of the sigmoid function: f'(z) = sigmoid(z) * (1 - sigmoid(z)).
Used for backpropagation through the sigmoid activation.
- softmax(z)
- Softmax activation function: f(z)_i = exp(z_i) / sum(exp(z_j)) for all j.
Maps input into a probability distribution over multiple classes. Used for multiclass classification.
- tanh(z)
- Hyperbolic tangent (tanh) activation function: f(z) = (exp(z) - exp(-z)) / (exp(z) + exp(-z)).
Maps input to the range [-1, 1], typically used for normalized input.
- tanh_derivative(z)
- Derivative of the tanh function: f'(z) = 1 - tanh(z)^2.
Used for backpropagation through the tanh activation.
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
|
class CuPyDenseLayer(builtins.object) |
| |
CuPyDenseLayer(input_size, output_size, activation='relu')
Initializes a Layer object.
Args:
input_size (int): The size of the input to the layer.
output_size (int): The size of the output from the layer.
activation (str): The activation function to be used in the layer.
|
| |
Methods defined here:
- __init__(self, input_size, output_size, activation='relu')
- Initializes the layer with weights, biases, and activation function.
Args:
input_size: (int) - The number of input features to the layer.
output_size: (int) - The number of output features from the layer.
activation: (str), optional - The activation function to use (default is "relu").
Supported values: "relu", "leaky_relu", or others.
Attributes:
weights: (cp.ndarray) - The weight matrix initialized using He initialization for "relu" or "leaky_relu".
biases: (cp.ndarray) - The bias vector initialized to zeros.
weight_gradients: (cp.ndarray) - Gradients of the weights, initialized to zeros.
bias_gradients: (cp.ndarray) - Gradients of the biases, initialized to zeros.
input_size: (int) - The number of input features to the layer.
output_size: (int) - The number of output features from the layer.
activation: (str) - The activation function used in the layer.
- activate(self, Z)
- Apply activation function.
- activation_derivative(self, Z)
- Apply activation derivative.
- zero_grad(self)
- Reset the gradients of the weights and biases to zero.
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
| |