Introduction
Since deep learning was introduced, it has become an indispensable tool in pattern recognition applications with big datasets. We all know that a deep learning neural network consists of a lot of layers with a certain architecture, and we also know that a deeper layer neural network architecture might solve complicated problems that are involved in a very big dataset. The question is how deep a neural network model can be achieved.
With traditional deep learning models such as multiple perception layers (MLP) and convolutional layers, we will face training speed and saturate accuracy problems when we increase the number of layers. To solve this issue, Residual Neural Network architecture is introduced to allow us to increase the number of layers while improving accuracy and avoiding the problem of vanishing gradients. More information on Residual Neural Networks can be found here.
This article aims to demonstrate how to design a deep learning Residual Neural Network in DLHUB software with ease.
Data preparation
The first step is to prepare a training dataset. The Cifar-10 dataset is provided by Toronto University, and it can be downloaded directly from the website.
This Cifar-10 dataset consists of 60,000 32x32 color images in 10 classes, with 6000 images per class. There are 50,000 training images and 1000 test images. The dataset structure contains an image folder with training and test map files.
In this article, only a small subset of the Cifar dataset is used: 23,973 images for training and 139 images for testing.
Design a neural network using DLHUB.
Step 1: Load training set
DLHUB supports the Cifar-10 dataset described in part 2. Loading the Cifar-10 dataset can be done in a few simple steps.
First, we need to browse the dataset location (1) and select the training map file (2). Then, we need to make sure that the input shape for the Cifar-10 image is set correctly (3). Since the Cifar-10 image is in 32x32 color format, the input shape will be 32x32x3. The image augmentation option is also supported on this loading page to improve accuracy during the training process.
Step 2: Configure Convolution Neural Network
Constructing a Residual Neural Network (ResNet) in DLHUB is a simple task as DLHUB provides a built-in ResNet layer.
First, select a ResNet layer in the Select Function palette (1). This ResNet consists of a deep learning Residual Neural Network architecture with deep layers. The ResNet parameters can be configured in the configuration panel (2). In this example, we only need to change the number of output nodes, but advanced users can tweak the ResNet architecture by ticking the Advance checkbox in the configuration panel. After the NetNet layer is added, a fully connected (Dense) layer will be used as an output layer with 10 nodes for 10 different classes. The Softmax is chosen as the activation function for this Dense output layer (3). The model is then verified before we can move to the training page.
Step 3: Train Neural Network
In this example, the training algorithm is the Stochastic Gradient Descent (SGD) with a learning rate of 0.0079. The Softmax Cross Entropy is used as the loss function (1). The mini-batch training technique is used, and the batch size is configured as 64. The training epochs are set to be 200 (2).
After around 29 minutes, the training process is finished. As can be seen in the training graph, the loss value continues to go down if we continue to train the deep ResNet, implying the accuracy will continue to improve.
Step 4: Evaluate the Neural Network
The trained deep ResNet can be evaluated with the test dataset with around 80% accuracy for only 200 training epochs. The accuracy will be improved if the number of training epochs increases.
Step 5: Test the Neural Network
The trained ResNet model can be tested on the built-in test page by browsing the test image folder location (1), selecting an image for testing (2), visualizing the selected image (3), and confirming the predicted result (4).
Export trained neural network model to weight file
When the trained neural network has been evaluated and tested with a new test data set, it can be exported using the export function in DLHUB.
Conclusion
In this article, we have shown how to design a deep-learning ResNet to classify 10 classes in the Cifar-10 dataset in DLHUB. Compared to similar Python code with Microsoft CNTK deep learning framework, the DLHUB is much simpler and easier to use. The trained model can be exported directly into LabVIEW for real-time application/deployment.
Comments