Hi I'm Harshal Bhat!

Technology Consultant: Software Development, PwC India LLP

Worked extensively on AWS cloud migration and pipeline for 50 procedures Environment setup and end-to-end migration of 50 Qlik applications and formulated data visualizations.

Contributed to Spark scripts in AWS Atena and optimized AWS Redshift performance by 60% faster query response. Gained familiarity with Docker, Bitbucket, JIRA, Confluence, and creating clean, modular, scalable software.

Research and Development Intern - IIT Ropar, India

Mentored by Dr. Dhiraj Mahajan and Dr. Pushpendra Singh. Designed and implemented an autonomous harvester capable of navigating in unknown environments and harvesting 15 different crop vegetables, utilizing AMCL and odometry data. Enabled navigation in unknown environments using IMU, GPS, & encoder data, integrated with ROS packages RVizSatellite & mapviz connected to Google Maps API. Published 1 Patent at The Indian Patent Office Journal.

Machine Learning/ Product Intern - Magna International, Lowell MA

Engineered a simulator that accurately emulated robot-software HTTP and gRPC API communication protocols, enabling offshore team to efficiently develop UI codebase without physical robot access

Increased overall development productivity by 10% through improved testing capabilities and reduced dependency on hardware

Designed and Implemented a collision avoidance system for forklift collision avoindance by fusing BLE sensors and Depth Camera feed and deployed on Jetson Nano

Currently working on Novel SLAM to optimize ORB-SLAM3 by adding lightweight relocalization pipeline using structure graph

3D Stereo Vision for Pseudo Lidar - Github

I developed a 3D Computer Vision pipeline, implementing disparity maps, 3D obstacle estimation, and 3D Point Cloud creation from Stereo Images. Utilized Stereo SGBM algorithms from OpenCV and integrated camera calibration for depth mapping. Calculated disparity to depth and reprojected Image to 3D for obstacle detection along with distances.

Road Segmentation Model for Autonomous Driving - Github

I trained a Deep Learning model having encoder decoder and skip connections architecture for road segmentation for video feed. I used the Berkeley BDK100 dataset for training and validation. The test set was a custom video from actual roads in Paris. The vision system was able to segment drivable area with a 95% accuracy rate.

Bicycle Tracking Using Unscented Kalman Filter - Github

This project focuses on utilizing the Kalman filter to track a bicycle's movement in a surveillance video captured from a stationary camera. The video contains several challenges, including linear movement, abrupt turns, occlusion by objects (e.g., light posts), and occasional issues with object detection. I aimed to enhance the tracking accuracy and robustness by implementing the Kalman filter.

Zero-Shot Semantic Neural Style Transfer - Github

We implemented a novel architecture for Neural Style Transfer based on Unseen content images. The content and style image are encoded using a VGG19 backbone, pretrained on ImageNet dataset. The default content-style attention map is generated using AdaAttN. The attention map generated is then fused with controlling signals from the segmentation mask of the content image. The stylized feature is computed given the updated attention map, and the final stylized image is generated by passing through a VGG decoder followed by new encoder and perpetual loss is calculated.

Robust Interception and Tracking for Quadrotor UAV - Github

This project aimed at designing a controller for the quadrotor to intercept and catch an alien UAV in defined space. First, I added a Linear Quadratic controller for hovering and interception, tuned the controller using state-of-the-art techniques and simulated in MATLAB

Adaptive Cruise Control System using MATLAB - Github

I designed an Adaptive cruise control in MATLAB/Simulink using a linear Model Predictive Controller(MPC) using transfer functions to regulate the accelerator pedal sequence. The behavior of the follower vehicle was regulated by the set-point reference velocity and intervehicular distance, 3 different states were modeled in the stateflow designer

DynaPRM: Efficient UAV Exploration in Dynamic Environments through Incremental Sampling PRM -Github

We, present DynaPRM, an incremental sampling-based probabilistic roadmap planner for UAV exploration in dynamic environments. Our approach addresses two key exploration challenges: efficient navigation of narrow passages and adaptive sampling density based on environment structure. DynaPRM detects and prioritizes exploration of confined spaces while employing sparse roadmap generation in open areas to optimize computational resources. The system integrates three core components: incremental PRM construction for viewpoint selection, multi-query planning for dynamic obstacle avoidance, and ESDF-based trajectory optimization for safe navigation. We evaluate the planner across diverse environments including confined spaces, maze-like structures, and open areas with moving obstacles. The results demonstrate the system's ability to maintain exploration coverage while adapting to different environment types. DynaPRM is particularly applicable to real-world scenarios such as search and rescue operations, where thorough exploration of complex, dynamic environments is essential

Vibration Analysis of Hydrodynamic Conical Journal Bearing using Machine Learning - Undergrad Thesis

We performed conditional monitoring for a conical journal bearing using C++, MATLAB and various machine learning methods. We give a comparative analysis of CNNs, SVMs, RF, KNN methods for prediction of the Bearing health. A dynamic model of the lubrication system was also proposed in this thesis to explain the vibration signal peaks in the FFT for vibration data. The research was presented at International conference for Industrial Tribology.

AutoPano - Automatic Panorama Stitching

Implemented an automatic panorama stitching algorithm from scratch that seamlessly combines 10 and more images into a single panoramic view. The project utilizes feature detection, matching, and homography estimation to align and blend images effectively.

Structure from Motion (SfM) - 3D Reconstruction of Unity Hall

Implemented a comprehensive Structure from Motion pipeline for 3D reconstruction from multiple images. The project includes feature matching, fundamental matrix estimation, camera pose estimation, and triangulation. A key focus was on implementing bundle adjustment to refine camera poses and 3D points simultaneously.

Bundle adjustment was formulated as an optimization problem minimizing the reprojection error across all cameras and 3D points: min∑∑V_ij(||x_j - P_i X_j||^2)

The implementation used a sparse Jacobian matrix for efficiency and the Trust Region Reflective algorithm for optimization. The final reconstruction of Unity Hall at WPI demonstrated significant improvement in accuracy, with the reprojection error reducing from 1.751347 to 1.015014 after bundle adjustment.

Neural Radiance Fields (NeRF) - Novel View Synthesis

Implemented a Neural Radiance Fields (NeRF) system for novel view synthesis. This project uses a fully connected deep network to represent a continuous 5D scene (3D spatial location and 2D viewing direction) and generate photorealistic images from new viewpoints.

Key aspects of the implementation include:

• Volume rendering with radiance fields using stratified sampling
• Network architecture: MLP with ReLU activation, width 256, and skip connections
• Positional encoding for improved fine detail representation
• Training on synthetic (Lego, Ship) and real-world (Coffee Cup) datasets

Results demonstrated significant improvement with positional encoding, achieving PSNR of 31.267 and SSIM of 0.9882 for the Lego model. The GIF above shows novel view synthesis for the Lego model after 200,000 iterations with positional encoding.

Einstein Vision: Advanced Computer Vision Pipeline

Developed a comprehensive computer vision system integrating multiple detection and processing modules:

• 3D Object Detection:
  • Implemented YOLOv8-Seg for primary object detection (vehicles, pedestrians, road signs, traffic lights)
  • Integrated DETIC for secondary object detection (brake lights, cones, poles, dustbins)
  • Used RAFT for optical flow analysis to determine object status (parked/moving)
• Depth Processing:
  • Utilized ZoeDepth for accurate depth map generation
  • Implemented IDEA-OSX for advanced object processing
  • Integrated YOLo3D for 3D pose estimation of detected objects
• 3D Lane Detection:
  • Employed YOLOPv2 for lane detection
  • Implemented 3D projection onto Blender world
  • Added polynomial curve fitting for smooth lane representation
• Scene Processing:
  • Developed Blender converter and parser for scene reconstruction
  • Created pipeline for generating side-by-side videos from sequential frames

The system outputs detailed scene information including object positions, classifications, and 3D poses, making it suitable for autonomous driving applications and advanced scene understanding.

Deep VIO: Visual-Inertial Odometry with Deep Learning - Github

Implemented a deep learning-based Visual-Inertial Odometry (VIO) system that fuses visual and inertial data for accurate robot localization. This project enhances traditional VIO methods with neural networks for improved performance in challenging environments.