I'm a Filipino-American Computer Scientist, and a Senior Research and Engineering Manager at Apple. I lead the Sensing Technologies Group under Machine Intelligence Neural Design at AIML. Our mission is to innovate and build differentiating experiences that make a profound impact on people through research and development in sensing and on-device AI. Applications of our work include health technologies, context-driven systems, and future device interactions.
I completed my Ph.D. at Carnegie Mellon University's School of Computer Science, where I was a founding member of the Future Interfaces Group. Prior to Apple, I was a Google Ph.D. Fellow, a recipient of the Fast Company Innovation by Design Award, a Disney Imagineer, and a recipient of nine Best Paper Awards and Nominations at premier venues in human-computer interaction.
I'm also an Affiliate Professor at Carnegie Mellon University's School of Computer Science and Human-Computer Interaction Institute. Last Fall (SY22-23), I gave guest lectures for 05-430 - Programming User Interfaces and 11-776 - On-Device Machine Learning at CMU, along with Debates in Design at UC-Berkeley (with Dr. Bjoern Hartmann). My CMU email is: first.lastname@cs.cmu.edu.My team is hiring. I have positions available for full-time senior research scientists, engineers, post-docs, and interns (all-year). I strongly encourage candidates from non-traditional and underrepresented backgrounds to apply.
Dr. Gierad Laput
Senior Manager
Dr. Jun Gong
Senior Research Scientist
Richard Kang
Senior Research Engineer
Dr. Kareem Bedri
Research Scientist
Mike Ralph
Firmware Engineer
Soroush Sahi
Research Associate
Rebecca Adaimi
Research Associate
Lisa Elkin
Research Associate
(now Ph.D. at UW)
Orson Xu
Research Associate
(now Ph.D. at UW)
Dr. Carolina Brum
Senior Research Scientist
Dr. Ke-Yu Chen
Senior Research Scientist
Cathy Fang
Research Associate
(now Ph.D. at MIT)
Fereshteh Shahmiri
Research Associate
(now at Apple HID)
Gabrielle Cohn
ML Engineer
(now at Apple AIML)
Dr. Scott Hudson
Senior Visiting Professor
We work closely with teams across Apple. Here are select features and experiences, used by millions of people, where we've made significant contributions:
In collaboration with Home Hardware, Home Software, Thermal Design, Hardware QA, Sensing Hardware: machine learning models and self-heating correction algorithms on the HomePod and HomePod Mini for temperature and humidity sensing.
In collaboration with Core Motion: machine learning models and algorithms on the watch and phone for vehicular detection, serving as an early stage classifier for Crash Detection. Apple Watch Series 8+, Apple Watch Ultra, iPhone 14+.
In collaboration with iPhone Hardware, Core OS, Core Motion, and Core Bluetooth: machine learning models and algorithms to support the Always-On Display, specifically power optimizations that levarage the watch-phone ecosystem. iPhone 14 Pro/Max.
In collaboration with Core Motion and Health Technologies: sensor fusion models for multimodal workout experiences. Automatic detection of indoor/outdoor swimming, cycling, and running in any sequence, including transitions. watchOS 9, Apple Watch Ultra action button integration.
In collaboration with Core Motion and Health Technologies: shipped algorithms and models for cycling workout detection and auto- pause/resume, powering Fitness and Fall Detection features. Apple Watch Series 6+
In collaboration with Accessibility and Human Interface Design. Assisted in feature productization.
Apple Watch Series 6, SE.
In collaboration with watchOS Software and Health Technologies. Developed pre-COVID. Conceptualized, prototyped, built sensing + machine learning algorithms, and shipped the feature into production. Apple Watch Series 4+.
Mac Studio— prior to joining our team, Mike was an integral part in shipping Mac Studio while at Hardware Engineering (Firmware and Diags).
Spatial Audio— prior to joining our team, Keyu was an integral part in shipping Spatial Audio (AirPods Pro) while at Core Motion.
Prior to Apple, I have published over 30 papers in interactive sensing and applied machine learning. My dissertation is titled: Context-Driven Implicit Interactions. The map below illustrates the breadth and depth of topics I have explored. My dissertation (purple track) is a culmination of several projects on practical, high-fidelity, and privacy-sensitive contextual sensing.
In my academic life, I publish at premier venues in interactive systems and human-computer interaction. Below are highlights (note: projects have no connections with Apple unless specified). See CV for a full list.
Enabling Hand Gesture Customization on Wrist-Worn Devices. 2022.
PaperProject
Automatic Class Discovery and One-Class Interactions for Acoustic Activity Recognition. 2020.
Paper Video Project
Sensing Fine-Grained Hand Activity Sensing with Wearables. 2019.
Paper Video Project
Cheap, Small, Low-Power and Clickable Fixed-Icon Laser Projections. 2014.
Paper Video Project
Expanding the Input Expressivity of Smartwatches Using Mechanical Pan, Twist, Tilt, and Click. CHI 2014.
Paper Video Project
Plug-and-Play Acoustic Activity Recognition. 2018.
Paper Video Project
LiDAR-Enabled Touch, User, and Object Sensing for IoT Experiences. 2019.
Paper Video Project
Long-Range Vibrometry for Smart Environment Sensing. 2018.
Paper Video
On-Touch Contextual Functionality for Smart IoT Appliances. 2017.
Paper Video
Fully Untethered, Stretchable, and Highly-Customizable Electronic Bandages. CHI 2019.
Exploring Data Tracking and Sharing Preferences of Wheelchair Athletes. 2018.
Paper
Early Concept, Interaction Framework, and Sensing Techniques for Microsoft Surface Duo.
Work Done at Microsoft in 2017.
Please see Curriculum Vitae for a full list of publications.
AppleSenior Research and Engineering Manager,
2019 - Present
Microsoft ResearchSensing + ML Research, 2017
GoogleSensing + ML Research, 2015
Disney ImagineeringSensing + ML Research, 2014
2022 Best Paper Nomination ACM CHI 2020 Best Paper Nomination ACM CHI 2018 Google PhD Fellowship 2018 Swartz Entrepreneurial Fellowship 2018 McGinnis Venture Capital Award 2017 Fast Company Innovation by Design Award 2016 Best Paper Award ACM UIST 2015 Best Paper Award ACM CHI 2016 Best Talk Award ACM UIST 2015 Best Talk Award ACM UIST 2016 Adobe Research Fellowship 2014 Qualcomm Innovation Fellowship 2013 Disney Research Fellowship
2023 ACM UIST, Program Committee, Associate Chair (AC) 2023 ACM UbiComp / IMWUT, Associate Editor (AE) 2022 ACM UIST, Program Committee, Associate Chair (AC) 2021 ACM CHI, Session Chair: Engineering Interactive Systems 2021 ACM UIST Program Committee, Program Technology Chair 2020 ACM UIST Program Committee, Demos Co-Chair 2020 ACM CHI Program Committee, Associate Chair (AC) 2019 ACM UIST Program Committee, Demos Co-Chair 2019 ACM UIST Program Committee, Associate Chair (AC) 2019 ACM CHI Program Committee, Associate Chair (AC) 2018 ACM UIST Program Committee, Associate Chair (AC) 2018 ACM CHI Program Committee, Associate Chair (AC) 2018 ACM UIST Prog. Comm., Student Innovation Contest 2018 ACM CHI, Session Chair: Tangibles 2018 ACM UIST, Session Chair: Sensing in the Small Scale 2017 ACM UIST Prog. Comm., Student Innovation Contest 2017 Faculty Hiring Committee, Carnegie Mellon, HCII 2017 ACM CHI, Session Chair: Sensing and Input 2017 ACM MobileHCI Program Commitee, AC, Workshops 2017 ACM CHI Program Committee, AC, Late-Breaking Work 2017 ACM MobileHCI, Workshop Co-Organizer 2016 ACM CHI Program Committee, AC, Late-Breaking Work
2022 UC Berkeley EECS (Berkeley, CA), Invited Speaker
2022 CMU, On-Device Machine Learning, Guest Lecture
2022 CMU, Programming User Interfaces, Guest Lecture
2022 ICML 2022 (Baltimore, MD), Invited Speaker (Apple)
2022 BostonCHI, Invited Speaker
2021 Gadgets, Sensors, and Activity Recognition, Carnegie Mellon, Guest Lecture
2021 Special Topics in HCI, Carnegie Mellon, Guest Lecture
2021 Robotics Institure Meets, CMU, Panelist
2021 On-Device Machine Learning Workshop, Apple (Speaker)
2021 CMU Portugal (Invited Speaker)
2021 Carnegie Mellon University (Guest Panelist)
2021 Apple Machine Learning Summit (Presenter and Organizer)
2020 Northwestern University CS (Guest Lecture)
2020 ARM Dev Summit (Invited Panelist)
2020 ACM SIGCHI Asian Affinity Group (Invited Keynote)
2019 MIT EECS (Cambridge, MA)
2019 UC Berkeley EECS (Berkeley, CA)
2019 Yale University CS (New Haven, CT)
2019 Cornell CIS (Ithaca, NY)
2019 University of Michigan EECS (Ann Arbor, MI)
2019 University of Wisconsin CS (Madison, WI)
2019 University of Chicago CS (Chicago, IL)
2019 UCLA CS (Los Angeles, CA)
2019 Microsoft Research (Redmond, WA)
2019 Apple AI & Machine Learning (Seattle, WA)
2019 Apple Design Studio (Cupertino, CA)
2019 Apple CoreAudio (Cupertino, CA)
2018 Apple Hardware Technologies (Cupertino, CA)
2018 Snap Research, Invited Speaker (Venice, CA)
2018 UC San Diego, HCI/Design Lab
2018 Qualcomm, HCI Seminar (San Diego, CA)
2018 World Sensors Symposium (Anaheim, CA)
2018 University of Michigan, MISC/HCI Series
2017 Pittsburgh City Council, PGH Labs 2.0
2017 ACM SIGGRAPH 2017 (Los Angeles, CA)
2017 Magic Leap (Seattle, WA)
2017 Amazon Alexa Day (Pittsburgh, PA)
2017 US-China Innovation Summit (Pittsburgh, PA)
2015 Stanford HCI Lunch Seminar (Stanford, CA)
2015 Disney, Themed Entertainment Conference
2015 Google Daydream Demo Day (Mt. View, CA)
Carnegie Mellon University, On-Device Machine Learning
Carnegie Mellon University, Programming User Interfaces
Carnegie Mellon University, Designing Human-Centered Systems
Carnegie Mellon University, Signal Processing and Electronics Prototyping
Carnegie Mellon University, User-Centered Research and Evaluation
University of Michigan, Needs Assessment and Usability Evaluation
University of Michigan, Introduction to Programming
ACM CHI ’13 ’14 ’15 ‘16 ‘17 '18 '19 '20 '21 '22
ACM UIST ’14 ‘15 ‘16 '17 '18 '19 '20, '21 '22
ACM UbiComp ’15 ‘16, '17 '18 '19 '20 '21 '22
ACM SIGGRAPH '19
ACM MobileHCI ‘15 ‘16 '17 '18 '19
ACM DIS '16 ‘17 '18, '19 '20
ACM TOCHI ‘15,
ACM ICMI ‘15, '17 '18,
ACM IUI ‘16 ‘17,
ACM ISS ‘16 '17'
ACM Automotive UI ‘17, ACM TEI ‘16 ‘17
IEEE Potentials ‘16, IEEE Pervasive '16,
My group's research agenda at Apple, combined with unmatched access to many of the world's most innovative products, puts us in a unique position to impact millions of people. I am looking for driven, highly creative, and highly collaborative candidates. I have positions available for interns, post-docs, and full-time research scientists + engineers. I welcome candidates with backgrounds in HCI, machine learning, electrical engineering, and related fields. I strongly encourage candidates from non-traditional and underrepresented backgrounds to apply.
My group has positions available in Pittsburgh, Seattle, and Cupertino. If interested, email me directly.
My idea of a happy life is being with family and friends, travelling, learning, and pursuing creative projects. I'm currently into photography— here a few of my favorite shots. High-res versions are available upon request.
