The image data enhancement operation in the dataset:ĭatasets.MnistDataset: Convert the dataset into MindSpore trainable data.ĬV.Resize: Resize image data pixels to meet the data size requirements of the LeNet network.ĬV.Rescale: Standardize and normalize image data so that the value of each pixel is in the range (0,1), which can improve training efficiency.ĬV.HWC2CHW: Transform the image data tensor, the tensor form is changed from height x width x channel (HWC) to channel x height x width (CHW), which is convenient for data training. The label data enhancement operation in the dataset:Ĭ.TypeCast: Convert the data type to int32. get_dataset_size ())Īfter the data augmentation function is called, the dataset size changes from 60000 to 1875, which meets the expectations of the mnist_ds.batch operation in data augmentation ( \(60000/32=1875\)). repeat ( repeat_size ) return mnist_ds ms_dataset = create_dataset ( train_data_path ) print ( 'Number of groups in the dataset:', ms_dataset. batch ( batch_size, drop_remainder = True ) mnist_ds = mnist_ds. shuffle ( buffer_size = buffer_size ) mnist_ds = mnist_ds. map ( operations = hwc2chw_op, input_columns = "image", num_parallel_workers = num_parallel_workers ) # process the generated dataset buffer_size = 10000 mnist_ds = mnist_ds. map ( operations = rescale_nml_op, input_columns = "image", num_parallel_workers = num_parallel_workers ) mnist_ds = mnist_ds. map ( operations = rescale_op, input_columns = "image", num_parallel_workers = num_parallel_workers ) mnist_ds = mnist_ds. map ( operations = resize_op, input_columns = "image", num_parallel_workers = num_parallel_workers ) mnist_ds = mnist_ds. map ( operations = type_cast_op, input_columns = "label", num_parallel_workers = num_parallel_workers ) mnist_ds = mnist_ds. int32 ) # using map to apply operations to a dataset mnist_ds = mnist_ds. Rescale ( rescale, shift ) hwc2chw_op = CV. Rescale ( rescale_nml, shift_nml ) rescale_op = CV. Resize (( resize_height, resize_width ), interpolation = Inter. MnistDataset ( data_path ) # define some parameters needed for data enhancement and rough justification resize_height, resize_width = 32, 32 rescale = 1.0 / 255.0 shift = 0.0 rescale_nml = 1 / 0.3081 shift_nml = - 1 * 0.1307 / 0.3081 # according to the parameters, generate the corresponding data enhancement method resize_op = CV. Import _transforms as CV import _transforms as C from import Inter from mindspore import dtype as mstype def create_dataset ( data_path, batch_size = 32, repeat_size = 1, num_parallel_workers = 1 ): """ create dataset for train or test Args: data_path (str): Data path batch_size (int): The number of data records in each group repeat_size (int): The number of replicated data records num_parallel_workers (int): The number of parallel workers """ # define dataset mnist_ds = ds. Applying a Gradient Accumulation Algorithm.Parallel Distributed Training Interfaces.Distributed Inference With Multi Devices.Saving and Loading Models in Hybrid Parallel Mode.Inference on the Ascend 910 AI processor.Loading a Model for Inference and Transfer Learning.Checking the Loss Value of the Model with the Change of Training Steps.Defining the Loss Function and Optimizer.Custom Callback Function to Collect the Model Loss Value and Precision Value.
Defining the Dataset and Data Operations.Implementing an Image Classification Application.Implementing Simple Linear Function Fitting.
0 kommentar(er)
