[2018: Internship at CSIRO] Developing an End to End Pipeline for Machine Learning in Robotics

The Internship

In May 2018, I was chosen for a 6 month paid internship program at DATA61 CSIRO (The national science agency of Australia), in Brisbane, Australia. I was supervised by Nicholas Hudson (who previously worked as a Senior Roboticist in NASA Jet Propulsion Laboratory, Boston Dynamics and Google) and Dr. Navinda Kottege.

The main task assigned to me was to develop an end-to-end pipeline for robotics-related machine learning for future use in DATA61 CSIRO and demonstrate it by extending NVIDIA's trailnet (Resnet-18 based) neural network for autonomous navigation of indoor hallways and hill-climbing. For this I did the following tasks:

1. Built (with a partner) a 6-wheeled robot platform for data collection
2. Collected data (with a partner) into ROS bags
3. Wrote scripts to convert ROSbags into TF records on Bracewell (CSIRO's supercomputer)
   I needed to modify and build custom singularity docker containers to run this.
4. Create a 20-layer residual convolutional neural network with both Keras-Tensorflow and pure Tensorflow.
5. Build a pipeline to thoroughly and efficiently shuffle and prefetch data from TF records and train the model.
6. Optimize the trained model for embedded devices (Jetson TX2) with TensorRT (both pure C++ and python tf-trt)
7. Implement the inference engines as ROS nodes (C++ and python both)
8. Modify the motor controller ROS package of Roboclaw to avoid thread racing and deadlock issues and make the robot autonomously navigate hallways
9. Build models for hill climbing and experiment with them.

Following is a presentation I did to the audience of scientists from Robotics and Automation Group, DATA61, CSIRO about my pipeline.

Hallway Navigation