| | |
- sega_learn.neural_networks.neuralNetworkBase.NeuralNetworkBase(builtins.object)
-
- BaseBackendNeuralNetwork
class BaseBackendNeuralNetwork(sega_learn.neural_networks.neuralNetworkBase.NeuralNetworkBase) |
| |
BaseBackendNeuralNetwork(layers, dropout_rate=0.2, reg_lambda=0.01, activations=None, loss_function=None, regressor=False)
A class representing a backend implementation of a neural network with support for forward propagation, backward propagation, training, evaluation, and hyperparameter tuning.
This class extends the `NeuralNetworkBase` class and provides additional functionality
for managing layers, applying dropout, calculating loss, and optimizing weights and biases.
Attributes:
layers (list): List of layer objects in the neural network.
layer_outputs (list): Outputs of each layer during forward propagation.
weights (list): Weights of each layer.
biases (list): Biases of each layer.
train_loss (list): Training loss values over epochs.
train_accuracy (list): Training accuracy values over epochs.
val_loss (list): Validation loss values over epochs.
val_accuracy (list): Validation accuracy values over epochs.
train_precision (list): Training precision values over epochs.
train_recall (list): Training recall values over epochs.
train_f1 (list): Training F1-score values over epochs.
val_precision (list): Validation precision values over epochs.
val_recall (list): Validation recall values over epochs.
val_f1 (list): Validation F1-score values over epochs.
learning_rates (list): Learning rates over epochs.
Methods:
__init__(layers, dropout_rate=0.2, reg_lambda=0.01, activations=None, loss_function=None, regressor=False):
Initializes the neural network with the specified layers, dropout rate,
regularization parameter, activation functions, and optional loss function.
initialize_new_layers():
Initializes the layers of the neural network with random weights and biases.
forward(X, training=True):
Performs forward propagation through the neural network.
backward(y):
Performs backward propagation to calculate gradients for weight and bias updates.
fit(X_train, y_train, X_val=None, y_val=None, optimizer=None, epochs=100, batch_size=32, ...):
Fits the neural network to the training data.
train(X_train, y_train, X_val=None, y_val=None, optimizer=None, epochs=100, batch_size=32, ...):
Trains the neural network model with optional validation and early stopping.
evaluate(X, y):
Evaluates the model on the given data and returns accuracy and predictions.
predict(X):
Predicts the output for the given input data.
calculate_loss(X, y):
Calculates the loss with L2 regularization for the given input and target labels.
_create_optimizer(optimizer_type, learning_rate, JIT=False):
Helper method to create optimizer instances based on the specified type.
tune_hyperparameters(X_train, y_train, X_val, y_val, param_grid, ...):
Performs hyperparameter tuning using grid search.
train_with_animation_capture(X_train, y_train, X_val=None, y_val=None, ...):
Trains the neural network while capturing training metrics in real-time animation.
|
| |
- Method resolution order:
- BaseBackendNeuralNetwork
- sega_learn.neural_networks.neuralNetworkBase.NeuralNetworkBase
- builtins.object
Methods defined here:
- __init__(self, layers, dropout_rate=0.2, reg_lambda=0.01, activations=None, loss_function=None, regressor=False)
- Initializes the Numba backend neural network.
Args:
layers: (list) - List of layer sizes or Layer objects.
dropout_rate: (float) - Dropout rate for regularization.
reg_lambda: (float) - L2 regularization parameter.
activations: (list) - List of activation functions for each layer.
loss_function: (callable) optional - Custom loss function to use (default is None, which uses the default calculate_loss implementation).
regressor: (bool) - Whether the model is a regressor (default is False).
- backward(self, y)
- Performs backward propagation to calculate the gradients.
Args:
y: (ndarray) - Target labels of shape (m, output_size).
- calculate_loss(self, X, y)
- Calculates the loss with L2 regularization.
Args:
X: (np.ndarray) - Input feature data.
y: (np.ndarray) - Target labels.
Returns:
loss: (float) - The calculated loss value with L2 regularization.
- evaluate(self, X, y)
- Evaluates the model's performance on the given data.
Args:
X: (np.ndarray) - Input feature data for evaluation.
y: (np.ndarray) - True target labels corresponding to the input data.
Returns:
metric: (float) - The evaluation metric (accuracy for classification, MSE for regression).
predicted: (np.ndarray) - The predicted labels or values for the input data.
- fit(self, X_train, y_train, X_val=None, y_val=None, optimizer=None, epochs=100, batch_size=32, early_stopping_threshold=10, lr_scheduler=None, p=False, use_tqdm=False, n_jobs=1, track_metrics=False, track_adv_metrics=False, save_animation=False, save_path='training_animation.mp4', fps=1, dpi=100, frame_every=1)
- Fits the neural network to the training data.
- forward(self, X, training=True)
- Performs forward propagation through the neural network.
Args:
X: (ndarray): - Input data of shape (batch_size, input_size).
training: (bool) - Whether the network is in training mode (applies dropout).
Returns:
ndarray: Output predictions of shape (batch_size, output_size).
- initialize_new_layers(self)
- Initializes the layers of the neural network.
Each layer is created with the specified number of neurons and activation function.
- predict(self, X)
- Generates predictions for the given input data.
Args:
X: (np.ndarray) - Input feature data for which predictions are to be made.
Returns:
outputs: (np.ndarray) - Predicted outputs. If the model is binary, returns the raw outputs.
Otherwise, returns the class indices with the highest probability.
- train(self, X_train, y_train, X_val=None, y_val=None, optimizer=None, epochs=100, batch_size=32, early_stopping_threshold=10, lr_scheduler=None, p=True, use_tqdm=True, n_jobs=1, track_metrics=False, track_adv_metrics=False, save_animation=False, save_path='training_animation.mp4', fps=1, dpi=100, frame_every=1)
- Trains the neural network model.
Args:
X_train: (ndarray) - Training data features.
y_train: (ndarray) - Training data labels.
X_val: (ndarray) - Validation data features, optional.
y_val: (ndarray) - Validation data labels, optional.
optimizer: (Optimizer) - Optimizer for updating parameters (default: Adam, lr=0.0001).
epochs: (int) - Number of training epochs (default: 100).
batch_size: (int) - Batch size for mini-batch gradient descent (default: 32).
early_stopping_threshold: (int) - Patience for early stopping (default: 10).
lr_scheduler: (Scheduler) - Learning rate scheduler (default: None).
p: (bool) - Whether to print training progress (default: True).
use_tqdm: (bool) - Whether to use tqdm for progress bar (default: True).
n_jobs: (int) - Number of jobs for parallel processing (default: 1).
track_metrics: (bool) - Whether to track training metrics (default: False).
track_adv_metrics: (bool) - Whether to track advanced metrics (default: False).
save_animation: (bool) - Whether to save the animation of metrics (default: False).
save_path: (str) - Path to save the animation file. File extension must be .mp4 or .gif (default: 'training_animation.mp4').
fps: (int) - Frames per second for the saved animation (default: 1).
dpi: (int) - DPI for the saved animation (default: 100).
frame_every: (int) - Capture frame every N epochs (to reduce file size) (default: 1).
- train_with_animation_capture(self, X_train, y_train, X_val=None, y_val=None, optimizer=None, epochs=100, batch_size=32, early_stopping_threshold=10, lr_scheduler=None, save_path='training_animation.mp4', fps=1, dpi=100, frame_every=1)
- Trains the neural network model while capturing training metrics in real-time animation.
Args:
X_train: (np.ndarray) - Training feature data.
y_train: (np.ndarray) - Training target data.
X_val: (np.ndarray), optional - Validation feature data (default is None).
y_val: (np.ndarray), optional - Validation target data (default is None).
optimizer: (Optimizer), optional - Optimizer for updating parameters (default is None).
epochs: (int), optional - Number of training epochs (default is 100).
batch_size: (int), optional - Batch size for mini-batch gradient descent (default is 32).
early_stopping_threshold: (int), optional - Patience for early stopping (default is 10).
lr_scheduler: (Scheduler), optional - Learning rate scheduler (default is None).
save_path: (str), optional - Path to save the animation file (default is 'training_animation.mp4').
fps: (int), optional - Frames per second for the saved animation (default is 1).
dpi: (int), optional - DPI for the saved animation (default is 100).
frame_every: (int), optional - Capture frame every N epochs (default is 1).
Returns:
None
- tune_hyperparameters(self, X_train, y_train, X_val, y_val, param_grid, layer_configs=None, optimizer_types=None, lr_range=(0.0001, 0.01, 5), epochs=30, batch_size=32)
- Performs hyperparameter tuning using grid search.
Args:
X_train: (np.ndarray) - Training feature data.
y_train: (np.ndarray) - Training target data.
X_val: (np.ndarray) - Validation feature data.
y_val: (np.ndarray) - Validation target data.
param_grid: (dict) - Dictionary of parameters to try.
layer_configs: (list), optional - List of layer configurations (default is None).
optimizer_types: (list), optional - List of optimizer types (default is None).
lr_range: (tuple) - Tuple of (min_lr, max_lr, num_steps) for learning rates.
epochs: (int) - Maximum epochs for each trial.
batch_size: (int) - Batch size for training.
Returns:
best_params: (dict) - Best hyperparameters found.
best_accuracy: (float) - Best validation accuracy.
Methods inherited from sega_learn.neural_networks.neuralNetworkBase.NeuralNetworkBase:
- apply_dropout(self, X)
- Applies dropout to the activation X.
Args:
X: (ndarray) - Activation values.
Returns:
ndarray: Activation values after applying dropout.
- calculate_precision_recall_f1(self, X, y)
- Calculates precision, recall, and F1 score.
Args:
X: (ndarray) - Input data
y: (ndarray) - Target labels
Returns:
precision: (float) - Precision score
recall: (float) - Recall score
f1: (float) - F1 score
- compute_l2_reg(self, weights)
- Computes the L2 regularization term.
Args:
weights: (list) - List of weight matrices.
Returns:
float: L2 regularization term.
- create_scheduler(self, scheduler_type, optimizer, **kwargs)
- Creates a learning rate scheduler.
- initialize_layers(self)
- Initializes the weights and biases of the layers.
- plot_metrics(self, save_dir=None)
- Plots the training and validation metrics.
Data descriptors inherited from sega_learn.neural_networks.neuralNetworkBase.NeuralNetworkBase:
- __dict__
- dictionary for instance variables
- __weakref__
- list of weak references to the object
| |