| | |
- builtins.object
-
- ConvLayer
- DenseLayer
- FlattenLayer
- RNNLayer
class ConvLayer(builtins.object) |
| |
ConvLayer(in_channels, out_channels, kernel_size, stride=1, padding=0, activation='relu')
A convolutional layer implementation for neural networks.
This layer performs 2D convolution operations, commonly used in convolutional neural networks (CNNs).
The implementation uses the im2col technique for efficient computation, transforming the convolution operation into matrix multiplication.
An optional activation function is applied element-wise to the output.
Args:
in_channels (int): Number of input channels (depth of input volume).
out_channels (int): Number of output channels (number of filters).
kernel_size (int): Size of the convolutional kernel (square kernel assumed).
stride (int, optional): Stride of the convolution. Default: 1.
padding (int, optional): Zero-padding added to both sides of the input. Default: 0.
activation (str, optional): Activation function to use. Options are "relu", "sigmoid", "tanh", or None. Default: "relu".
Attributes:
in_channels (int): Number of input channels.
out_channels (int): Number of output channels.
kernel_size (int): Size of the square convolutional kernel.
stride (int): Stride of the convolution.
padding (int): Zero-padding added to both sides of the input.
weights (numpy.ndarray): Learnable weights of shape (out_channels, in_channels, kernel_size, kernel_size).
biases (numpy.ndarray): Learnable biases of shape (out_channels, 1).
activation (str): Type of activation function.
weight_gradients (numpy.ndarray): Gradients with respect to weights.
bias_gradients (numpy.ndarray): Gradients with respect to biases.
input_cache (numpy.ndarray): Cached input for use in backward pass.
X_cols (numpy.ndarray): Cached column-transformed input.
X_padded (numpy.ndarray): Cached padded input.
h_out (int): Height of output feature maps.
w_out (int): Width of output feature maps.
input_size (int): Size of input (same as in_channels).
output_size (int): Size of output (same as out_channels).
Methods:
zero_grad(): Reset gradients to zero.
_im2col(x, h_out, w_out): Convert image regions to columns for efficient convolution.
forward(X): Perform forward pass of the convolutional layer.
_col2im(dcol, x_shape): Convert column back to image format for the backward pass.
backward(d_out, reg_lambda=0): Perform backward pass of the convolutional layer.
|
| |
Methods defined here:
- __init__(self, in_channels, out_channels, kernel_size, stride=1, padding=0, activation='relu')
- Initializes a convolutional layer object for neural networks.
This layer performs 2D convolution operations, commonly used in convolutional neural networks (CNNs).
Args:
in_channels: (int) - Number of input channels (depth of input volume).
out_channels: (int) - Number of output channels (number of filters).
kernel_size: (int) - Size of the convolutional kernel (square kernel assumed).
stride: (int), optional - Stride of the convolution (default is 1).
padding: (int), optional - Zero-padding added to both sides of the input (default is 0).
activation: (str), optional - Activation function to use (default is "relu").
Attributes:
in_channels: (int) - Number of input channels.
out_channels: (int) - Number of output channels.
kernel_size: (int) - Size of the square convolutional kernel.
stride: (int) - Stride of the convolution.
padding: (int) - Zero-padding added to both sides of the input.
weights: (np.ndarray) - Learnable weights of shape (out_channels, in_channels, kernel_size, kernel_size).
biases: (np.ndarray) - Learnable biases of shape (out_channels, 1).
activation: (str) - Type of activation function.
weight_gradients: (np.ndarray or None) - Gradients with respect to weights, initialized to None.
bias_gradients: (np.ndarray or None) - Gradients with respect to biases, initialized to None.
input_cache: (np.ndarray or None) - Cached input for use in backward pass.
input_size: (int) - Size of input (same as in_channels).
output_size: (int) - Size of output (same as out_channels).
- activate(self, Z)
- Apply activation function.
- backward(self, d_out, reg_lambda=0)
- Optimized backward pass using im2col technique.
Args:
d_out: (np.ndarray) - Gradient of the loss with respect to the layer output,
shape (batch_size, out_channels, h_out, w_out)
reg_lambda: (float, optional) - Regularization parameter.
Returns:
dX: Gradient with respect to the input X.
- forward(self, X)
- Perform forward pass of the convolutional layer.
Args:
X: numpy array with shape (batch_size, in_channels, height, width)
Returns:
Output feature maps after convolution and activation.
- zero_grad(self)
- Reset the gradients of the weights and biases to zero.
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
|
class DenseLayer(builtins.object) |
| |
DenseLayer(input_size, output_size, activation='relu')
Initializes a fully connected layer object, where each neuron is connected to all neurons in the previous layer.
Each layer consists of weights, biases, and an activation function.
Args:
input_size (int): The size of the input to the layer.
output_size (int): The size of the output from the layer.
activation (str): The activation function to be used in the layer.
Attributes:
weights (np.ndarray): Weights of the layer.
biases (np.ndarray): Biases of the layer.
activation (str): Activation function name.
weight_gradients (np.ndarray): Gradients of the weights.
bias_gradients (np.ndarray): Gradients of the biases.
input_cache (np.ndarray): Cached input for backpropagation.
output_cache (np.ndarray): Cached output for backpropagation.
Methods:
zero_grad(): Resets the gradients of the weights and biases to zero.
forward(X): Performs the forward pass of the layer.
backward(dA, reg_lambda): Performs the backward pass of the layer.
activate(Z): Applies the activation function.
activation_derivative(Z): Applies the derivative of the activation function.
|
| |
Methods defined here:
- __init__(self, input_size, output_size, activation='relu')
- Initializes the layer with weights, biases, and activation function.
Args:
input_size: (int) - The number of input features to the layer.
output_size: (int) - The number of output features from the layer.
activation: (str), optional - The activation function to use (default is "relu").
Attributes:
weights: (np.ndarray) - The weight matrix initialized using He initialization for ReLU or Leaky ReLU, or standard initialization otherwise.
biases: (np.ndarray) - The bias vector initialized to zeros.
input_size: (int) - The number of input features to the layer.
output_size: (int) - The number of output features from the layer.
activation: (str) - The activation function to use.
weight_gradients: (np.ndarray or None) - Gradients of the weights, initialized to None.
bias_gradients: (np.ndarray or None) - Gradients of the biases, initialized to None.
- activate(self, Z)
- Apply activation function.
- activation_derivative(self, Z)
- Apply activation derivative.
- backward(self, dA, reg_lambda)
- Backward pass of the layer.
- forward(self, X)
- Forward pass of the layer.
- zero_grad(self)
- Reset the gradients of the weights and biases to zero.
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
|
class FlattenLayer(builtins.object) |
| |
A layer that flattens multi-dimensional input into a 2D array (batch_size, flattened_size).
Useful for transitioning from convolutional layers to dense layers.
Attributes:
input_shape: (tuple) - Shape of the input data (excluding batch size).
output_size: (int) - Size of the flattened output vector.
input_cache: (np.ndarray) - Cached input for backpropagation.
input_size: (int) - Size of the input (same as input_shape).
output_size: (int) - Size of the output (same as output_size).
|
| |
Methods defined here:
- __init__(self)
- Initializes the layer with default attributes.
Attributes:
input_shape: (tuple or None) - Shape of the input data, to be set dynamically during the forward pass.
output_size: (int or None) - Size of the output data, to be set dynamically during the forward pass.
input_cache: (any or None) - Cache to store input data for use during backpropagation.
input_size: (int or None) - Flattened size of the input, calculated as channels * height * width.
output_size: (int or None) - Flattened size of the output, same as input_size.
- backward(self, dA, reg_lambda=0)
- Reshapes the gradient back to the original input shape.
Args:
dA (np.ndarray): Gradient of the loss with respect to the layer's output,
shape (batch_size, flattened_size)
reg_lambda (float): Regularization parameter (unused in FlattenLayer).
Returns:
np.ndarray: Gradient with respect to the input, reshaped to original input shape.
- forward(self, X)
- Flattens the input tensor.
Args:
X: (np.ndarray) - Input data of shape (batch_size, channels, height, width)
or any multi-dimensional shape after batch dimension.
Returns:
np.ndarray: Flattened output of shape (batch_size, flattened_size)
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
|
class RNNLayer(builtins.object) |
| |
RNNLayer(input_size, hidden_size, activation='tanh')
Will be implemented later.
|
| |
Methods defined here:
- __init__(self, input_size, hidden_size, activation='tanh')
- Will be implemented later.
Data descriptors defined here:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
| |