I'm a Computer Scientist and an Engineering Manager at Apple. I lead the Perception and Interaction Engineering group under Machine Intelligence Neural Design (AIML). Our mission is to innovate and build differentiating experiences through research and development in multimodal machine learning with motion, vision, audio, and other sensors. Applications of our work include interactive technologies, health & fitness, and spatial computing experiences.
I completed my Ph.D. at Carnegie Mellon University's School of Computer Science, where I was a founding member of the Future Interfaces Group. Prior to Apple, I was a Google Ph.D. Fellow, a recipient of the Fast Company Innovation by Design Award, a Disney Imagineer, and a recipient of nine Best Paper Awards and Nominations at premier venues in human-computer interaction.
I'm also an Affiliate Professor at Carnegie Mellon University's School of Computer Science and Human-Computer Interaction Institute. This Spring (SY23-24) I am working with Moshe Mahler at the Entertainment Technology Center, specifically on ETC 53610 - Interdisciplinary Projects in Game Design. Last Fall (SY22-23), I gave guest lectures for 05-430 - Programming User Interfaces and 11-776 - On-Device Machine Learning at CMU, along with Debates in Design at UC-Berkeley (with Dr. Bjoern Hartmann). My CMU email is: first.lastname@cs.cmu.edu.My team is hiring. I currently have positions for interns (all-year), senior software engineers, and senior research scientists. I strongly encourage candidates from non-traditional and underrepresented backgrounds to apply.
Dr. Gierad Laput
Engineering Manager
Dr. Jun Gong
Senior Research Scientist
Richard Kang
Senior ML Engineer
Dr. Kareem Bedri
Senior Research Scientist
Erik Hornberger
Senior Software Engineer
George Haddad
Senior Software Engineer
Dr. Asaf Liberman
Senior Research Scientist
Baruch Zilber
Senior Software Engineer
Mike Ralph
Firmware Engineer
Shira Kadosh
ML Engineer
Oron Levy
Research Scientist
Dr. Daniel Rotman
Senior Research Scientist
Eyal Finkelshtein
ML Engineer
Saar Tal Mirkin
ML Engineer
Vimal Mollyn
Research Associate
Han Zhang
Research Associate
Moran Yanuka
Research Associate
Soroush Shahi
Research Associate
Rebecca Adaimi
Research Associate
(now Scientist at Oura)
Lisa Elkin
Research Associate
(now Ph.D. at UW)
Orson Xu
Research Associate
(now Ph.D. at UW)
Dr. Carolina Brum
Senior Research Scientist
Dr. Ke-Yu Chen
Senior Research Scientist
Cathy Fang
Research Associate
(now Ph.D. at MIT)
Fereshteh Shahmiri
Research Associate
(now at Apple HID)
Gabrielle Cohn
ML Engineer
(now at Apple AIML)
Dr. Scott Hudson
Senior Visiting Professor
We work closely with teams across Apple. Here are select features and experiences where we've made contributions:
In collaboration with CoreMotion, Siri, Intelligent System Experiences, and AirPods Firmware: machine learning models and algorithms for head gesture detection, enabling hands-free Siri Interactions on AirPods. AirPods 2nd gen+, iOS 18, watchOS 18, and macOS Sequioa.
In collaboration with SWE, Siri, and OpenAI: part of a cross-functional team enabling system-wide integration and API support for Apple Intelligence and ChatGPT, including on-device and private cloud compute interfaces. iOS 18, iPadOS 18, and macOS Sequioa.
In collaboration with SWE Input Intelligance: part of a large cross-functional team working on Apple Intelligence for Writing Tools and Personal Context Search, specifically on model safety integration. iOS 18, iPadOS 18, and macOS Sequioa.
In collaboration with SWE Xcode Teams: part of a large cross-functional team working on Apple Intelligence for Xcode, specifically on model safety integration for predictive code completion and Swift Assist. macOS Sequioa.
In collaboration with Video Engineering: part of a large cross-functional team making contributions to research and development in hand input and interactions, including hand-object interactions, synthetic data, gesture customization, and ML model optimizations.
In collaboration with the Design Team: early-stage explorations on sensing, perception, and interaction techniques for StandBy mode in iOS.
In collaboration with Health, Visual Experiences, and Core Motion: early explorations and algorithm support for time in daylight on the Watch for Vision Health. Recently announced at WWDC 2023.
In collaboration with Home Hardware, Home Software, Thermal Design, Hardware QA, Sensing Hardware: machine learning models and self-heating correction algorithms on the HomePod and HomePod Mini for temperature and humidity sensing.
In collaboration with Core Motion: machine learning models and algorithms on the watch and phone for vehicular detection, serving as an early stage classifier for Crash Detection. Apple Watch Series 8+, Apple Watch Ultra, iPhone 14+.
In collaboration with iPhone Hardware, Core OS, Core Motion, and Core Bluetooth: machine learning models and algorithms to support the Always-On Display, specifically power optimizations for suppression that levarage the watch-phone ecosystem. iPhone 14 Pro/Max.
In collaboration with Core Motion and Health Technologies: sensor fusion models for multimodal workout experiences. Automatic detection of indoor/outdoor swimming, cycling, and running in any sequence, including transitions. watchOS 9, Apple Watch Ultra action button integration.
In collaboration with Core Motion and Health Technologies: shipped algorithms and models for cycling workout detection and auto- pause/resume, powering Fitness and Fall Detection features. Apple Watch Series 6+
In collaboration with Accessibility and Human Interface Design. Gesture interactions for Accessibility on the Watch. Assisted in feature productization.
Apple Watch Series 6, SE.
In collaboration with watchOS Software and Health Technologies. Developed pre-COVID. Conceptualized, prototyped, built sensing + machine learning algorithms, and shipped the feature into production. Apple Watch Series 4+.
Mac Studio— prior to joining our team, Mike was an integral part in shipping Mac Studio while at Hardware Engineering (Firmware and Diags).
Spatial Audio— prior to joining our team, Keyu was an integral part in shipping Spatial Audio (AirPods Pro) while at Core Motion.
Prior to Apple, I have published over 35 papers in the intersection of sensing, interactive systems, and machine learning. My dissertation is titled: Context-Driven Implicit Interactions.
My team and I publish at premier venues in interactive systems, human-computer interaction, and machine learning. Below are highlights (note: projects have no connections with Apple unless specified). See CV for a full list.
Enabling Hand Gesture Customization on Wrist-Worn Devices. 2022.
PaperProject
Automatic Class Discovery and One-Class Interactions for Acoustic Activity Recognition. 2020.
Paper Video Project
Sensing Fine-Grained Hand Activity Sensing with Wearables. 2019.
Paper Video Project
Cheap, Small, Low-Power and Clickable Fixed-Icon Laser Projections. 2014.
Paper Video Project
Expanding the Input Expressivity of Smartwatches Using Mechanical Pan, Twist, Tilt, and Click. CHI 2014.
Paper Video Project
Plug-and-Play Acoustic Activity Recognition. 2018.
Paper Video Project
LiDAR-Enabled Touch, User, and Object Sensing for IoT Experiences. 2019.
Paper Video Project
Long-Range Vibrometry for Smart Environment Sensing. 2018.
Paper Video
On-Touch Contextual Functionality for Smart IoT Appliances. 2017.
Paper Video
Fully Untethered, Stretchable, and Highly-Customizable Electronic Bandages. CHI 2019.
Exploring Data Tracking and Sharing Preferences of Wheelchair Athletes. 2018.
Paper
Early Concept, Interaction Framework, and Sensing Techniques for Microsoft Surface Duo.
Work Done at Microsoft in 2017.
Please see Curriculum Vitae for a full list of publications.
AppleResearch and Engineering Manager,
2019 - Present
Microsoft ResearchSensing + ML Research, 2017
GoogleSensing + ML Research, 2015
Disney ImagineeringSensing + ML Research, 2014
2022 Best Paper Nomination ACM CHI 2020 Best Paper Nomination ACM CHI 2018 Google PhD Fellowship 2018 Swartz Entrepreneurial Fellowship 2018 McGinnis Venture Capital Award 2017 Fast Company Innovation by Design Award 2016 Best Paper Award ACM UIST 2015 Best Paper Award ACM CHI 2016 Best Talk Award ACM UIST 2015 Best Talk Award ACM UIST 2016 Adobe Research Fellowship 2014 Qualcomm Innovation Fellowship 2013 Disney Research Fellowship
2024 ACM CHI, Program Committee, Associate Chair (AC) 2023 ACM UIST, Program Committee, Associate Chair (AC) 2023 ACM UbiComp / IMWUT, Associate Editor (AE) 2022 ACM UIST, Program Committee, Associate Chair (AC) 2021 ACM CHI, Session Chair: Engineering Interactive Systems 2021 ACM UIST Program Committee, Program Technology Chair 2020 ACM UIST Program Committee, Demos Co-Chair 2020 ACM CHI Program Committee, Associate Chair (AC) 2019 ACM UIST Program Committee, Demos Co-Chair 2019 ACM UIST Program Committee, Associate Chair (AC) 2019 ACM CHI Program Committee, Associate Chair (AC) 2018 ACM UIST Program Committee, Associate Chair (AC) 2018 ACM CHI Program Committee, Associate Chair (AC) 2018 ACM UIST Prog. Comm., Student Innovation Contest 2018 ACM CHI, Session Chair: Tangibles 2018 ACM UIST, Session Chair: Sensing in the Small Scale 2017 ACM UIST Prog. Comm., Student Innovation Contest 2017 Faculty Hiring Committee, Carnegie Mellon, HCII 2017 ACM CHI, Session Chair: Sensing and Input 2017 ACM MobileHCI Program Commitee, AC, Workshops 2017 ACM CHI Program Committee, AC, Late-Breaking Work 2017 ACM MobileHCI, Workshop Co-Organizer 2016 ACM CHI Program Committee, AC, Late-Breaking Work
2023 Princeton University, Invited Speaker
2022 UC Berkeley EECS (Berkeley, CA), Invited Speaker
2022 CMU, On-Device Machine Learning, Guest Lecture
2022 CMU, Programming User Interfaces, Guest Lecture
2022 ICML 2022 (Baltimore, MD), Invited Speaker (Apple)
2022 BostonCHI, Invited Speaker
2021 Gadgets, Sensors, and Activity Recognition, Carnegie Mellon, Guest Lecture
2021 Special Topics in HCI, Carnegie Mellon, Guest Lecture
2021 Robotics Institure Meets, CMU, Panelist
2021 On-Device Machine Learning Workshop, Apple (Speaker)
2021 CMU Portugal (Invited Speaker)
2021 Carnegie Mellon University (Guest Panelist)
2021 Apple Machine Learning Summit (Presenter and Organizer)
2020 Northwestern University CS (Guest Lecture)
2020 ARM Dev Summit (Invited Panelist)
2020 ACM SIGCHI Asian Affinity Group (Invited Keynote)
2019 MIT EECS (Cambridge, MA)
2019 UC Berkeley EECS (Berkeley, CA)
2019 Yale University CS (New Haven, CT)
2019 Cornell CIS (Ithaca, NY)
2019 University of Michigan EECS (Ann Arbor, MI)
2019 University of Wisconsin CS (Madison, WI)
2019 University of Chicago CS (Chicago, IL)
2019 UCLA CS (Los Angeles, CA)
2019 Microsoft Research (Redmond, WA)
2019 Apple AI & Machine Learning (Seattle, WA)
2019 Apple Design Studio (Cupertino, CA)
2019 Apple CoreAudio (Cupertino, CA)
2018 Apple Hardware Technologies (Cupertino, CA)
2018 Snap Research, Invited Speaker (Venice, CA)
2018 UC San Diego, HCI/Design Lab
2018 Qualcomm, HCI Seminar (San Diego, CA)
2018 World Sensors Symposium (Anaheim, CA)
2018 University of Michigan, MISC/HCI Series
2017 Pittsburgh City Council, PGH Labs 2.0
2017 ACM SIGGRAPH 2017 (Los Angeles, CA)
2017 Magic Leap (Seattle, WA)
2017 Amazon Alexa Day (Pittsburgh, PA)
2017 US-China Innovation Summit (Pittsburgh, PA)
2015 Stanford HCI Lunch Seminar (Stanford, CA)
2015 Disney, Themed Entertainment Conference
2015 Google Daydream Demo Day (Mt. View, CA)
Carnegie Mellon University, On-Device Machine Learning
Carnegie Mellon University, Programming User Interfaces
Carnegie Mellon University, Designing Human-Centered Systems
Carnegie Mellon University, Signal Processing and Electronics Prototyping
Carnegie Mellon University, User-Centered Research and Evaluation
University of Michigan, Needs Assessment and Usability Evaluation
University of Michigan, Introduction to Programming
ACM CHI ’13 ’14 ’15 ‘16 ‘17 '18 '19 '20 '21 '22
ACM UIST ’14 ‘15 ‘16 '17 '18 '19 '20, '21 '22
ACM UbiComp ’15 ‘16, '17 '18 '19 '20 '21 '22
ACM SIGGRAPH '19
ACM MobileHCI ‘15 ‘16 '17 '18 '19
ACM DIS '16 ‘17 '18, '19 '20
ACM TOCHI ‘15,
ACM ICMI ‘15, '17 '18,
ACM IUI ‘16 ‘17,
ACM ISS ‘16 '17'
ACM Automotive UI ‘17, ACM TEI ‘16 ‘17
IEEE Potentials ‘16, IEEE Pervasive '16,
My group's research agenda at Apple, combined with unmatched access to many of the world's most innovative products, puts us in a unique position to impact millions of people. I am looking for driven, highly creative, and highly collaborative candidates. I have positions available for interns, post-docs, and full-time research scientists + engineers. I welcome candidates with backgrounds in HCI, machine learning, electrical engineering, and related fields. I strongly encourage candidates from non-traditional and underrepresented backgrounds to apply.
My group has positions available in Pittsburgh, Seattle, Cupertino, and Israel. If interested, email me directly.
My idea of a happy life is being with family and friends, travelling, learning, and pursuing creative projects. I'm currently into photography— here a few of my favorite shots. High-res versions are available upon request.
