Zheyuan Xu

iOS | Android ∙ VR | AR ∙ DL | RL ∙ IoT ∙ Robotics ∙ Autopilot Systems

If you'll indulge me, I'd like to change the world

ABOUT

Software engineer and inventor with deep interest in machine learning, VR/AR, natural language processing, computer vision, reinforcement learning and robotics.

DOWNLOAD RESUME

EDUCATION

University of Washington
Master of Science - Computer Science and Systems

SEP 2020 - DEC 2021

Georgia Institute of Technology
Bachelor of Science - Computer Science
Concentration in Intelligence and Theory

JAN 2016 - MAY 2020

Georgia Institute of Technology
Bachelor of Science - Electrical Engineering

AUG 2015 - MAY 2020

PERSONAL PROJECTS HIGHLIGH

A.R.M.S.

Mixed reality warehouse manegement system

MORE

Unity MLagents Python C++
Real-time sensor stream from robot to headset Automatic grabbing agent by using PPO (Proximal Policy Optimization)

GYM-DOFBOT

A vision-friendly simulation environment for Yahboom DOFBOT based on bullet physics engine and OpenAI Gym

pip install gym-dofbot==0.1.1

MORE

pybullet gym
vision-control reinforcement learning Python multi-agent support GPL-v3

GAIA System(ver. 1.0, 1.5 and 2.0)

An integrated tool for visualizing and managing natural disasters

Frontend in Three.js and HTML, CSS

Backend in Azure Cloud, Azure SQL, Google BigQuery

MORE

Resource Management WebGL
HTML CSS Python Javascript Azure SQL

FERM(Firefighting-Emergency-Resource-Management)

A resource managing tool for allocating firefighting units

Frontend in Three.js and HTML, CSS

Backend in Azure Cloud, Azure SQL

MORE

Resource Management WebGL
HTML CSS Python Javascript Azure SQL

AdaEye

A next-generation navigation and cognitive package for visually impaired

An audio companion for blind people 🔊 Gimbal device as the “eye” 👁 and the iOS App the “companion” 👯‍♀️ GPT-3 based AI agent for chatting and complex problem solving🧠

Voice only login, registration and navigation

Winner for Best Use of Google Cloud in MakeHarvard 2021

Built on Azure cloud VM with RabbitMQ, and Google Cloud Vision API for object detection and obstacle avoidance

MORE

Assistive Technology Voice Control IoT
iOS Swift Python RabbitMQ Azure Google Cloud Vision OpenAI

Neomap

Share you new year resolutions, relive your memories, and create new ones with Neomap!

Take a picture or upload a screen shot and post it to view later

Use hand gestures to like, share or comment on a post

Firebase for user authentication and post retrieval

Winner for Best Use of Google Cloud and Radar.io most creative award in MLH New Year New Hack

MORE

New Year Resolution Mixed Reality
iOS Swift firebase Reality Kit

Lunar Olympics

Future Olympics Game Hosted on the Moon

Arduino Nano Sense for gesture control via BLE (Bluetooth Low Energy)

Outdoor games hosted near Apollo landing site

Indoor games hosted inside Space-X Starship

Winner for The Open Innovation University Hackathon

MORE

IoT BLE Gesture Detection
iOS Swift Reality Kit SketchFab

AlphaFly

A flight assistant that helps improve flight experience

Increasing transparency of flight information

Luggage tracking by camera/GPS/Bluetooth

MORE

flight assistant TAMUhack 2021
Swift Reality Kit Google Cloud Vision

VCart

A mixed reality remote shopping experience on your cell phone

Virtual shop experience with hand gesture

Add items to cart with grabbing actions

Order history synchronized to cloud

Check out by tapping

MORE

Virtual Shopping Augmented Reality
iOS Swift Firebase

GAIA System

Countering Forest Fire with Aerospace Technology and Artificial Intelligence

Next-generation, voice-controlled GCS (ground control software)

Satellites and GPS information visualization on the radar panel

Allows automatic lock of fire region by mouse clicks

YOLO network for real-time fire detection

MORE

ground control station voice recognition automation
x86 desktop C# AWS Firebase Realtime DB

X-Waves

An Augmented Reality(AR) solution for Tourette Syndrome using Magic leap

Neutralize tics in Tourette Syndrome by audio and visual distraction

MORE

XR for medical MITReality Hack 2020
magic leap IMU sensor

Go to my devpost page for more details on each project.

DEVPOST

SKILLS

Languages
C/C++, Java, Python, C#, JavaScript, CSS, HTML, SQL, Swift, Dart, Objective-C

Tools & Frameworks
Visual Studio, Android Studio, Unity, Github, MySQL, Azure SQL, Firebase, Matlab, Xcode, Eagle CAD, Altium, Solidworks, Three.js, Cannon.js

Communication
English (Proficient), Chinese (Native)

WORK/RESEARCH EXPERIENCE

Plus AI - Senior Software Engineer
AD/ADAS
• Perception and data infra team

Sep 2024 - Present

Mercedes-Benz R&D North America - Software Engineer (Contract)
AD/ADAS
• Worked on cutting-edge OTA algorithm development
• Ported core open source libraries to AUTOSAR stack running on QNX

Nov 2023 - Sep 2024

U Power Robotics - ADAS/AD Software Engineer
AD/ADAS
• Setup simulation pipeline based on open source simulator, enabled verification of lane detection network and planning stacks
• Developed and maintained CUDA-based point cloud preprocessing package, reduced pipeline latency by more than 10 times
• Developed and maintained ROS2 driver package for Continental Radars and point cloud segmentation package for perception stack, enabled close-range obstacle detection (verified on recorded vehicle data)
• Developed multi-threaded ROS2 mission management package for urban autonomy demo, enabled visualization of high-level mission command and vehicle status

Jan 2022 - Sep 2023

Mavenir - Machine Learning Intern
Telecommunication and Robotics
• Worked on factory robotics project, developed a YOLO-based ROS package for water bottle detection and motion planning of affordable robotic arms, achieved more than 60% success rate

Oct 2021 - Dec 2021

RATLab LLC - Machine Learning Engineer
Consumer Electronics
• Developed neural network based algorithms for the consumer electronic product with Pytorch in Python, deployed on ARM-based microcontroller with Tensorflow Lite, achieved more than 85% classification accuracy in breathing detection

July 2021 - September 2021

Georgia Institute of Technology Research Assistant
Robotics and Automation | GTSR Lab
• Mechatronics and firmware development for GT-MAB 2.0
• Improved communication link between blimp and ground station, lowered latency by more than 150 times
• Helped on achieving 6DOF stabilization on a blimp
• Co-authored a paper that got best Student Paper Award for AIM2021 and co-invented two patents
Swing-Reducing Flight Control System for an Underactuated Indoor Miniature Autonomous Blimp AIM2021--Best Student Paper Award Flight Control System for Miniature Aerial Robots
Highly Effective Motion Capture Marker for Small Aerial Robots

MAY 2020 - AUG 2020

Georgia Institute of Technology - Undergraduate Research Assistant
Robotics and Automation | GTSR Lab
• Refurbished and modified the OSV (omni-directional surface vehicle), by using Solidworks, and Mavlink in C++ and Python
• Integrated RTK (real-time kinematic) GPS, enabling centimeter-accuracy in localization and heading measurement
• Helped secure NSF funding for the project

JAN 2020 - MAY 2020

HARDWARE PROJECTS

Multi-purpose Scientific Drone

Multi-purpose Quadcopter for Surveillence, Security, and Scientific Research

Raspberry Pi 3B as on-board computer, with Pixhawk 2.4.8 as co-processor

4G LTE/WIFI controlled, able to cruise for 20-25 under mild weather conditions

Detachable gripper for additional payload/emergency pickup

Intel NCS stick enabling real time object recognition

Test Flight

Aerospace Quadcopter Artificial Intelligence
Linux Python C/C++ ArduPilot AWS VPN

Glove Bot

Radio-controlled electric car by hand gesture

Xbee Radio Module sending command and data

ARM Cortex M3 microcontroller as ground station and transceiver

LED panel for information display

Flex sensors for mapping hand gesture to actual commands

Details

IoT Xbee Radio Flex Sensors
iOS Swift Reality Kit SketchFab