Deep Learning with Apache SINGA & R
Course Overview
Gain firsthand expertise on installing, configuring, and deploying Apache SINGA and make use of the Python APIs for running synchronous, asynchronous and hybrid training frameworks. Through practical guided exercises leverage the stochastic gradient descent (SGD) to find optimized parameters for many popular feed-forward models such as convolutional neural networks (CNN) and recurrent neural networks (RNN).
In cloud labs, practice job submission, training linear regression models, multi-layer perceptron models, convolutional neural network models, and a recurrent neural network model.
Be the expert of deploying complex deep learning models using Apache SINGA and Python.
At the end of the training, participants will be able to:
- Install and configure Apache SINGA through conda packages
- Gain a deep understanding of major components of the software stack of SINGA
- Access implementation and modules of SINGA stack through Python APIs
- Train a model on a single node or in a cluster Submit a job for training
- Train a linear regression model, multi-layer perceptron model, CNN, RNN
- Deliver scalable models that can work on the complex and large datasets
Pre-requisite
Required: Working knowledge of Python, Machine Learning, Deep Learning Models
Duration
2 days
Course Outline
- From Conda
- From Source in Linux with CPU Only
- Major Components of Software Stack
- Core: Tensor and Devices Abstactions for storing model variable and operation
- Model: Higher Level Classes for Machine Learning
- IO: Data Loading, Processing, Message Passing
- Python API for optimization of memory and executio
- Tensor Implementation for storing variables
- Linear algebraic operation with Tensor
- Python API
- Layer-Wrapping C++ Layers for simpler construction of APIs
- Layer-Python API
- Feedforward Net-Neural net classes for constructing nets using Layers
- Initializer-Popular initialization methods for parameters values
- Initializer-Python API
- Loss-Training loss implementation using python tensor
- Metric-Evaluating model’s performance using Python
- Optimizer-Optimizing model parameters
- Data-Loading and fetching data batches
- Image Tool-Utility model for image augmentation
- Quick Start
- Training on a Single Node
- Preparing data and job configuration
- Training without parallelism
- Asynchronous parallel training
- Synchronous parallel training
- Training in a Cluster
- Starting Zookeeper
- Training with GPUS
- NeuralNet
- TrainOneBatch
- Updater
- Cluster Topology
- Using Tensor module of PySINGA
- Generating Training Data
- Training via SGD
- Importing PySINGA modules
- Generating Training Data
- Creating MLP Model
- Training the model
- Data Preparation
- Creating CNN Model
- Initializing the Parameters
- Setting up the optimizer and tensors
- Conducting SGD
- Saving and Loading the model
- Prediction & Dubugging
- Dataset Pre-processing
- Creating the network
- Conducting SGD
- Checkpoint
- Sample