My personal webset

Yue Fan

Yue Fan

Ph.D. student
University of California, Santa Cruz

I am currently in my second year as a Ph.D. student in the Computer Science and Engineering (CSE) department at the University of California, Santa Cruz, advised by Professor Xin Eirc Wang. I earned my Bachelor's degree in Automation from Shandong University, followed by a Master's degree in Robotics from Johns Hopkins University. My research interests predominantly lie in the fields of Embodied AI, Computer Vision, and Natural Language Processing.


- [June 7 2023] Our SlugJARVIS Team won the third place in the Amazon Alexa Simbot Challenge.
- [May 9 2023] Our Athena teamTeam Athena, to which I have the honor of serving as team leader, has advanced to the semi-finals of Amazon Alexa Socialbot Grand Challenge 5.
- [May 2 2023] Our AVDN paper is accepted by ACL2023. AVDN challenge is released.


R2H: Building Multimodal Navigation Helpers that Respond to Help
Under review
Author: Yue Fan, Kaizhi Zheng, Jing Gu, Xin Eric Wang

Aerial Vision-and-Dialog Navigation
ACL 2023
Author: Yue Fan, Winson Chen, Tongzhou Jiang, Chun Zhou, Yi Zhang, Xin Eric Wang

JARVIS: A Neuro-Symbolic Commonsense Reasoning Framework for Conversational Embodied Agents
Preprint 2022
Author: Kaizhi Zheng*, Kaiwen Zhou*, Jing Gu*, Yue Fan*, Jialu Wang*, Zonglin Di, Xuehai He, Xin Eric Wang

Learn by Observation: Imitation Learning for Drone Patrolling from Videos of A Human Navigator
IROS 2020
Author: Yue Fan, Shilei Chu, Wei Zhang, Ran Song, and Yibin Li

My projects

Amazon Alexa Prize competition: Socialbot Grand Challenge 5 (on-going)

- The challenge aims at advancing conversational AI. University teams are tasked with developing a "socialbot", an AI chatbot that can interact naturally and intelligently with humans on a variety of topics through Amazon's Alexa platform.
- I serve as the team leader of our Athena3 team.
- Our Athena team has advanced to the semi-final

Alexa Prize Socialbot Grand Challenge

Athen3 Team

Students from the ERIC Lab and the Natural Language and Dialogue Systems Lab are making the fifth appearance in the competition. The goal of the team is to leverage advance algorithms and AI models to build a smart chat bot.

Location: Santa Cruz, California
Faculty advisor: Xin Wang
Team lead: Yue Fan

Amazon Alexa Prize competition: Simbot Challenge

- The challenge is focused on helping advance development of next-generation virtual assistants that will assist humans in completing real-world tasks by continuously learning, and gaining the ability to perform commonsense reasoning.
- Our SlugJARVIS Team won the third place in the Simbot Challenge.
- Our SlugJARVIS Team won the Public Benchmark Challenge.

Alexa Prize SimBot Challenge Public Benchmark Challenge


UC Santa Cruz is one of America's Public Ivy universities and a member of the prestigious Association of American Universities (AAU). The ERIC Lab is led by Prof. Xin Eric Wang and stands for Embodiment, Reasoning, Intelligence, and language Communication. The ERIC Lab’s research topics include natural language processing, computer vision, and machine learning, with an emphasis on building embodied AI agents that can communicate with humans in natural language to perform real-world multimodal tasks.

Location: Santa Cruz, California
Faculty advisor: Xin Wang

Unspuervised Adrenomyeloneuropathy disease data analysis

- Apply feature selection to find dominant factors among the disease progression.
- Design the extra-data-dimension heatmap toolkit for visualization the patient clusters. - Use Bayesian Neural Network to classify the progressor with uncertainty.

Heatmap Toolkit.

Learn by Observation: Imitation Learning for Drone Patrol from Raw Videos of A Human Navigator

- Design a data auto-labeling method using inter-frame geometric consistency.
- Bring up a DNN called UAVPatrolNet for Detecting Road.
- Make a dataset for drone autonomous Navigation.

IROS 2020 accepted

Training Quadruped Robot with Reinforcement Learning

- Use Unity3D ML-Agents.
- Train the quadruped robot to crawl.

RL for Robot

Object Detection in Aerial Image
- I contributed to the teamwork by reproducing existing mature algorithms, e.g. RPN, Faster R-CNN.
- I conducted simulated experiments and adjusted the parameters to realize the optimal training effect; improved the object detection performance on aerial images.

ECCV Workshop - Visdrone2018

Carbon-free Car

After testing nowadays' state-of-the-art object detection networks, we followed the Faster R-CNN algorithm. However, we made a few adjustments on it to adapt to VisDroneDet dataset. The dataset given consists of many variant-sized proposals which lead to a multi-scale object detection problem. In order to mitigate the impact of relatively rapid changes in sizes of bounding boxes, we added more anchors with large sizes to fit those larger objects and keep small anchors unchanged for detecting tiny objects such as people and cars in long distance. Moreover, the VisDroneDet dataset has an unbalanced object dis- tribution. When testing on validation dataset, we found that classification performance for car is much better than others for the reason that the appearance of cars is more frequent. To alleviate this problem, we masked out some car bounding boxes by hand for pursuing better classification performance.

Control and Monitoring System of DJI Drones through PC
- Designed the control interface on PC with varies functions like “vehicle detection”.
- Developed a system to transmit data between UVA and PC using Qt and DJI SDK.
- Applied the system in city traffic to successfully improve the management efficiency.


Drone with ROS

Control interface on PC.

DJI M100 Drone.

Binocular distance measurement with one camera on UVA.

Control of Carbon-free Car
- Developed a circuit board and selected the proper sensor by studying the control system of the carbon-free car.
- Conducted OOP of the machine by designing and applying the control algorithm.
- Awarded the First Prize in Engineering Training Integration Ability Competition of Shandong Province.

Carbon-free Car

-realize autonomous obstacle avoidance

-Since the car is powered by the hammer block‘s gravitational potential energy, it is called Carbon-free Car.

-Win the First Prize in Engineering Training Integration Ability Competition of Shandong Province

Assembling and Programming Drones with ROS
- Assembled drones from scratch.
- Designed programs for STM32 flight controller(Pixhawk) running Robot operating system.
- Won the municipal First Prize in China RoboWork Competition.


Drone with ROS


A Method to Precisely Apply Screen Protector
(Patent No.: ZL 2015 1 0853152. 0)

Extracurricular Experience

Career Exploration Program
Team Leader of 10, The University of Hong Kong, 2016

Filming in Support of College Entrance Examination Takers
Initiator, Shandong Experimental High School, 2015
video >>

Honor & Award

2nd-class Scholarship
2018, 2017 & 2016 • Issued by Shandong University

1st Prize, Quadrotor Aircraft Group & Hexacopter Aircraft Group of China RoboWork Competition
2018 & 2016 • Issued by The International Federation of Robotics

3rd Prize, National Undergraduate Engineering Training Integration Ability Competition
2017 • Issued by Department of Higher Education, Ministry of Education of PRC

1st Prize, National Olympiad in Informatics in Shandong Province
2014• Issued by China Computer Federation

Copyright 2023 by YueFan. All rights reserved.
All designs are the property of the owner.