Andrew Richardson

visionOS UI Frameworks

Apple

2019 - 2024

I had the opportunity to help build visionOS from the early days of Apple Vision Pro, working with some amazing people to figure out how people should build apps in a 3D spatial environment. I got to work on a lot of fun and very challenging projects, like figuring out how SwiftUI should become a fully 3D-capable UI framework. It was a thrill to get to see the Vision Pro, one of Apple's most innovative and ambitious new products in years, through to shipping! Below are some of the projects I worked on.

WWDC23: Meet SwiftUI for spatial computing

I had the honor of presenting a headline talk for visionOS at WWDC23, “Meet SwiftUI for spatial computing”, that introduced the core design concepts for SwiftUI on visionOS and some of its new capabilities. Presenting a WWDC talk was a dream come true for me!

Privacy-preserving hover effects

Privacy is a core tenet of the Vision Pro, and your gaze patterns are extremely sensitive – they can be used to tell all kinds of things about you. I was tasked with the challenge of figuring out how to display UI feedback when looking at elements of an app without revealing to the app where you’re looking. I helped design and architect the solution for this, known as “hover effects”, from the core system behavior up to the visual appearance and the public APIs, which also received a patent. This was featured in the announce keynote for the Vision Pro, which was a thrill for me!

Home app

Apple

2015 - 2019

I worked on the Home app team for almost 5 years, from the beginning of the first version of the app in iOS 10. I had the opportunity to work on many different parts of the app, from the core appearance to accessory controls, home cameras, and automations.

HomeKit Secure Video

HomeKit Secure Video enables HomeKit cameras to automatically record footage based on events like when a person is detected at the door and save it to the cloud, securely and privately. I helped build the Home app UI and design the architecture for the feature. I focused in particular on the data model and streaming camera clips via HTTP Live Streaming (HLS), but had a hand in many aspects of the feature, as well as project management.

Home Presence Automations

Home presence automations allow actions to be run automatically when people in the home arrive or leave, using a secure "check-in" system with a home hub (e.g., a HomePod) to determine who's home. I was able to help design, plan, and build the feature, from the core architecture to the entire UI flow. We also received a patent for the design!

iOS Intern

Apple

2013 - 2014

I first joined Apple as an intern in 2013 on the iOS Apps & Frameworks team, where I was able to help fix a variety of bugs for the release of iOS 7. Across my two internships in 2013 and 2014, I was able to work on a number of different projects and prototypes and demo them to senior leadership. Here are two of them that wound up shipping:

Search in Settings

I built the first version of the search feature in the Settings app on iOS, which later shipped in iOS 9. This meant building both the UI and the search index across all settings, which required building new APIs and sprawling adoption across hundreds of settings pages.

Interactive Calculator

I rewrote large parts of the Calculator app in Swift, making it one of the first Apple apps using the language in iOS 9. I also built a new design that supported visualizing and editing expressions, and viewing a calculation history. While these did not ship in iOS 9, they did eventually get resurrected with the Calculator redesign in iOS 18!

iOS Jailbreak Developer

Jailbreak

2011 - 2013

I got my start in iOS development while in university by getting involved in the iOS jailbreak community. I worked on building “tweaks”, or extensions that modified the OS itself, for jailbroken devices. I learned how to reverse engineer the internals of the OS, hook into the right functions, and use it to build cool tweaks. Some of these were eventually added as native iOS features by Apple!

Abstergo

Meaning “to clean away” in Latin, Abstergo provided tools for easily managing notifications for iOS 5-6. It added the ability to clear a single notification or all notifications with a swipe (before these existed natively), and a reminder system to postpone notifications until later.

Merge

Messages on iOS had a longstanding annoying behavior in iOS 5-6: conversations with different numbers or emails for the same contact were kept separate. This was especially bad when iMessage had a habit of changing which address you were messaging, causing conversations to fork off and get very confusing. Merge fixed that problem, automatically merging these conversations together in the UI, and providing an easy way to select which address your message would be sent to.

Jukebox

Jukebox provided a beautiful music widget with rich controls built into Notification Center on iOS 5-6 (before widgets existed natively). It even integrated with other tweaks to support being used directly on the Home Screen.