aiml.load_data package
Submodules
aiml.load_data.generate_parameter module
generate_parameter.py
This module provides a function for generating parameters based on input dataset.
- aiml.load_data.generate_parameter.generate_parameter(input_shape, clip_values, nb_classes, dataset_test, dataloader_test)[source]
Generate the image shape, clip values, and number of classes based on the input dataset.
- Parameters:
input_shape (numpy array) – Image shape; if given (not None), it will remain unchanged.
clip_values (tuple) – Data range; if given (not None), it will remain unchanged.
nb_classes (int) – Number of classes; if given (not None), it will remain unchanged.
dataset_test – The input dataset for reference.
dataloader_test – DataLoader for the input dataset.
- Returns:
A tuple containing (input_shape, clip_values, nb_classes).
- Return type:
tuple
aiml.load_data.load_model module
load_model.py
This script is responsible for loading the model.
- aiml.load_data.load_model.load_model(model, device)[source]
Load a machine learning model.
- Parameters:
model (model or string) – If a string is provided, it will search for the target model by detectors.
device (string) – The device to use, either ‘cpu’ or ‘gpu’.
- Returns:
The loaded machine learning model.
- Return type:
model
aiml.load_data.load_set module
load_set.py
This script is responsible for loading the test dataset for model evaluation.
aiml.load_data.normalize_datasets module
normalize_datasets.py
This module contains functions for normalizing and denormalizing a dataset.
- aiml.load_data.normalize_datasets.check_normalize(dataloader)[source]
Check if the data in a dataloader is normalized.
- Parameters:
dataloader (torch.utils.data.DataLoader) – A PyTorch DataLoader containing the dataset.
- Returns:
True if the data is normalized (mean close to 0, std close to 1), False otherwise.
- Return type:
bool
- aiml.load_data.normalize_datasets.denormalize(batch)[source]
Denormalize a batch of normalized data using mean and standard deviation values.
- Parameters:
batch (torch.Tensor) – A batch of normalized data.
- Returns:
The denormalized batch of data with the same shape as the input.
- Return type:
torch.Tensor
- aiml.load_data.normalize_datasets.get_mean_std(dataset)[source]
Get the mean and standard deviation of the dataset’s image channels.
- Parameters:
dataset (dataset) – The dataset containing images.
- Returns:
None.
- aiml.load_data.normalize_datasets.get_transforms()[source]
Get a list of transformations for datasets, including normalization.
- Returns:
A composition of transformations.
- Return type:
torchvision.transforms.Compose
- aiml.load_data.normalize_datasets.normalize_and_check_datasets(num_workers, batch_size_test, batch_size_train, dataset_test, dataset_train)[source]
Normalize and check the given test and optionally, training datasets for normalization.
- Parameters:
num_workers (int) – Number of workers for data loading.
batch_size_test (int) – Batch size for the test dataset.
batch_size_train (int) – Batch size for the training dataset (if provided).
test_dataset – The test dataset.
train_dataset (optional) – The training dataset (Default is None).
- Returns:
If normalization is required, returns a tuple containing the normalized test and training datasets along with their data loaders. If no normalization is needed, returns the test dataset as-is.
- Return type:
Tuple
- aiml.load_data.normalize_datasets.normalize_datasets(dataset_test, dataset_train=None)[source]
Normalize the training and testing datasets.
- Parameters:
dataset_test (dataset) – The testing dataset.
dataset_train (dataset, optional) – The training dataset (Default is None).
- Returns:
A tuple containing the normalized testing dataset and, if provided, the normalized training dataset.
- Return type:
tuple