Title: Embracing NLP for Mobile Computing Innovations
Speaker: Prof. Sung-Ju Lee @ KAIST
Time : 15:00 ~ 16:00, Nov 26th, 2024
Location: Hybrid
- In-person: 26513
- Online: https://us02web.zoom.us/j/83979965009?pwd=7y5UZkqNwnGBlJOfDywWPt8DyvOGDs.1
Meeting ID: 839 7996 5009
Passcode: 112533
Language: English speech & English slides
Abstract:
After briefly introducing various projects of the Networking & Mobile Systems Lab (NMSL) at KAIST, we will delve into two projects that utilize NLP for mobile computing. First, we discuss FedTherapist, a mobile mental health monitoring system that utilizes continuous speech and keyboard input in a privacy-preserving way via federated learning. We explore multiple model designs by comparing their performance and overhead for FedTherapist to overcome the complex nature of on-device language model training on smartphones. We propose a Context-Aware Language Learning (CALL) methodology to effectively utilize smartphones' large and noisy text for mental health signal sensing. Next, we introduce "By My Eyes," that grounds Multimodal Large Language Models (MLLMs) with Sensor Data via Visual Prompting. Despite LLMs remarkable performance across diverse domains, the application to sensor data in ubiquitous applications (e.g., healthcare and environmental monitoring) remains limited due to the diversity and heterogeneity of sensor data, which complicates the development of a generalized foundational model. Existing approaches typically rely on text-based prompts, often requiring domain-specific knowledge or leading to poor performance and high computational costs. To overcome these challenges, we leverage the visual interpretation capabilities of MLLMs by converting sensor data into visual prompts, enhancing their ability to interpret and analyze complex sensory inputs more effectively.
Bio:
Sung-Ju Lee earned his Ph.D. in Computer Science from the University of California, Los Angeles (UCLA) in 2000. Following his doctoral studies, he started his industry career at the Hewlett-Packard Company, serving as a Principal Research Scientist and Distinguished Mobility Architect. Subsequently, he held the position of Principal Member of Technical Staff at the CTO Office of Narus, Inc. (now part of Symantec). In 2015, Dr. Lee transitioned to KAIST, where he currently holds the KAIST Endowed Chair Professorship. His research spans the area of mobile computing, wireless networking, ubiquitous sensing, mobile AI, network security, and human-computer interactions. His research has been well cited (over 17,000 times) and presented in esteemed conference venues including MobiCom, MobiSys, NeurIPS, CVPR, EMNLP, CHI, INFOCOM, CoNext, SenSys, UbiComp, UIST, CSCW, ICDCS, NDSS, and IMC. Additionally, he holds 51 granted US patents. Dr. Lee received the HP CEO Innovation Award in 2010 for his pivotal role in bringing innovative products to market. He has also been honored with prestigious awards, including the test-of-time paper award at ACM WiNTECH 2016, the best paper awards at IEEE ICDCS 2015 and ACM CSCW 2021, and the methods recognition award at ACM CSCW 2021. Additionally, he received the Technology Innovations Award from KAIST. Dr. Lee's leadership in the academic community is exemplified by his roles as General Chair of ACM MobiCom 2014 and co-TPC Chair of IEEE INFOCOM 2016 and ACM MobiCom 2021. He is an IEEE Fellow and an ACM Distinguished Scientist.
