About me
I am a Ph.D. candidate in Computer Science at Dartmouth College, specializing in Human-Computer Interaction, body sonification, and AI-generated music. My primary research objective is to develop interactive systems that leverage machine learning algorithms to provide acoustic feedback, such as musical feedback during physical rehabilitation, movement sonification, or music creation through conversation. I am deeply intrigued by the beauty and complexity of music and sound, with a particular focus on bridging the gap between human expression and musical creation. I am advised by Prof. Elizabeth L. Murnane and Prof. Michael A. Casey, and a member of the Empower Lab, where we design interactive technologies to empower people and promote social good.
Prior to my studies at Dartmouth, I earned my master's degree from Cornell University, where my research focused on novel interaction technologies and body reconstruction. During my time at Cornell, I was a member of the Sci-Fi Lab, under the guidance of Pro. Cheng Zhang and Prof. Francois Guimbretiere.
Publications
Augmenting memory care practices and insights by automating patient monitoring through multimodal machine learning (under review)
MusicalPT: Augmenting physical therapy by integrating adaptive musical guidance to enhance exercise quality and patient experience (IMWUT'24) Songyun Tao, Yong Hoon Chung, Yijing Feng, Shirin Amouei, Michael Cassey, Viola S. Stoermer, Elizabeth L. Murnane
NeckFace: Continuously Tracking Full Facial Expressions on Neck-mounted Wearables (IMWUT '21) Songyun Tao*, Tuochao Chen*, Yaxuan Li*, Hyunchul Lim, Mose Sakashita, Ruidong Zhang, Francois Guimbretiere, Cheng Zhang
C-Face: continuously reconstructing facial expressions by deep learning contours of the face with ear-mounted miniature cameras (UIST ’20) Tuochao Chen, Benjamin Steeper, Kinan Alsheikh, Songyun Tao, Francois Guimbretiere, Cheng Zhang
SpeeChin: A Smart Necklace for Silent Speech Recognition (IMWUT)/UbiComp ’21 Ruidong Zhang, Mingyang Chen, Benjamin Steeper,Yaxuan Li, Zihan Yan, Yizhuo Chen, Songyun Tao, Tuochao Chen, Hyunchul Lim, Cheng Zhang
ThumbTrak: Recognizing Micro-finger Poses Using a Ring with Proximity Sensing (MobileHCI '21）Wei Sun, Franklin Mingzhe Li, Congshu Huang, Zhenyu Lei, Benjamin Steeper, Songyun Tao, Feng Tian, Cheng Zhang
FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by Miniature Thermal Cameras on Wrist. (Acknowledged) (IMWUT)/Ubicomp’20Fang Hu, Peng He, Songlin Xu, Yin Li, Cheng Zhang