Data Manipulation for Deep Learning

Datasets drive our choices for all the deep learning hyperparameters and peculiarities involved in making accurate predictions. We split the dataset into a training set, test set, and validation set. Our model is trained on the training set, and we choose the hyperparameters (usually by trial and error) on the validation set. We select the hyperparameters that have the highest accuracy on the validation set and then fix our model. The test set is reserved to be used only once at the very end of our pipeline to evaluate our model. Our algorithm is run exactly once on the test set and that accuracy gives us a proper estimate of our model's performance on truly unseen data.

Now that we have created our training set to perform the optimization process, let's look at how manipulating this data can help us train our model.


Data Preprocessing

The loss function is computed on the mini-batch of the training data at every step of the optimization algorithm. Therefore, the loss landscape on which optimization is performed depends directly on the training data. The aim of pre-processing the data is to make it more amenable to efficient training.

We usually normalize the data before using it, making the data cloud zero-centered and ensuring that all the features have the same variance. This makes it easier to optimize as it becomes less sensitive to small changes in weight. Note that we would use the same exact mean and standard deviation (that we used on the training set) to normalize the test set.

Some standard practices of data pre-processing with image data include:

  • Subtract the mean image (used in AlexNet)
  • Subtract per-channel mean (mean along each channel, used in VGGNet)
  • Subtract per-channel mean and divide by per-channel std (used in ResNet)


Data Augmentation

It is a commonly used technique where we perform image transformations on the training set for increasing both the size and the diversity of labeled training data. Not only is data augmentation used to increase the number of training samples, but it also acts as a form of regularization by adding noise to the training data.

Image augmentations have become a common implicit regularization technique in computer vision tasks to combat overfitting in deep convolutional neural networks. Basic image transformations: flipping, rotating, scaling, blurring, jittering and cropping. It should be noted that only the image transformations that make sense are applied. For example, performing a vertical flip on a car image is never done (as upside-down car images are not found) but the same transformation can be applied to a white blood cell image in a blood smear.

Comments

Post a Comment

Popular posts from this blog

The move_base ROS node

Image
move_base package provides the move_base node which is a major element of ROS Navigation stack. It moves the robot from its current position to a goal position. When path planning is done, the user gives uses the 2D Nav Goal in RVIZ to specify the goal position and orientation. This position (x,y,z) and orientation(x,y,z,w) are actually published in move_base/goal topic. The move_base node links together a global and local planner to accomplish its global navigation task. The Global Planner It is in charge of calculating the safe path in order to arrive at goal pose. Note that the path is calculated before the robot starts moving so it will not take into account the laser readings while moving i.e. unscanned obstacles that are not in the map. Navfn Planner The Navfn planner is the most commonly used global planner for ROS Navigation. It uses Dijkstra's algorithm in order to calculate the shortest path between the initial pose and the goal pose. Carrot Planner This plann...
Read more

Three Wheeled Omnidirectional Robot : Motion Analysis

Image
A Three-wheeled omnidirectional robot is capable to moving in any given direction without rotating by giving velocities to the three wheels in a certain ratio. All motors are situated at an angle of 120 degrees from each other. f B ,  f L ,  f R  are for back , left and right wheel respectively. a x  and a y  are accelerations in X R  and Y R  axis in Robot frame and w is the angular velocity in Robot frame. Hence putting the values of alpha and making it into a matrix form we get : As we need to obtain the f values, we will invert the matrix. We need  a x  , a y  and w is the world frame, so we apply transformation. The transformation angle is    α 1  = 30 degrees.  Putting this in the previous equation, Multiplying the two matrices, we get the final relation as : For example, if you want to move the robot along X axis(world frame), put  a x W  = 1 , ...
Read more

Overview of ATmega328P

Image
While reading about the ATmega328, I came a across a lot of stuff which I would like to share here. There is a humongous amount of information available in its datasheet. Introduction ATmega328 is a single chip micro-controller created by Atmel. It has a In-System Self-programmable Flash memory of 32 KB, SRAM of 2 KB (2048 bytes) and EEPROM of 1 KB (1024 bytes). It has a 8-bit AVR CPU type which means they can process data in 8-bit chunk as its data bus is 8-bit wide.   It contains 23 general purpose I/O pins and 32 general purpose working registers. It has 32 single-byte registers and 8-bit RISC (reduced instruction set computer) devices and hence the name 328. The P is for ' picoPower '. It has 3 flexible timer/counter with compare modes, internal and external interupts , 6-channel 10-bit A/D converter , programmable watch dog timer with internal oscillator and 5 software selectable power saving modes. Digital communication peripherals include 1-USART, 2-SPI and 1-...
Read more