My Work

My Work

Using LLMs to Quickly Adapt Hate Speech Enforcement at Meta

From 2018–2022, I was working in Facebook/Meta’s Infra and Integrity groups, working to build back-end AI-powered systems, picking up Facebook-scale Infra, Data Science & Data Engineering skills. This isn’t as photogenic work as some of my older projects, but I learned a lot about building things at scale, in a highly regulated environment, and getting buy-in across an enormous cross-functional organization.

In 2022, I led a project to productionize the use of multi-modal LLMs to adapt quickly to new kinds of hate speech around the globe.

Social Vouching: Age Verification on Instagram

Here is an article on work I was part of in the Integrity group, on age verification at Instagram. In particular, I drove the social vouching work across Meta/Facebook platforms, and implemented the core common APIs for all the major apps to use.

The Social Vouching Flow as implemented in Instagram Age Verification

Generative Art through NN Style Transfer

This Forbes article talks about the Camera work I was part of at Facebook around 2016–2018, including the Neural-Network based video Style Transfer, which was a type of generative content that re-styles images based on other artists. This was the first generative content shipped at Facebook, and also the first use of client-side deep learning there.

Person Segmentation

We also shipped person segmentation, which was used for all sorts of background & foreground effects, including background blurring. Since this was the second neural-network-based effect we shipped, we made sure there was a common runtime engine shared by all of them, to save space in the app.

Mobile Phone SLAM AR

We also worked with Oculus to adapt the SLAM system used for inside-out tracking on the Quest VR headsets for mobile use, and shipped it into Facebook mobile apps. This is a blog post that we published about how we did it, and about working with a SF-based artist to create one of the first AR virtual art installations in 2016.

iDAvatars Health Care Avatar

For this project, we combined speech recognition, NLP, facial emotion detection, and an animated avatar, in collaboration with IBM Watson, to build an mobile app that could collect health care information before or after medical visits.

“A Vision Quest” AR Game

I founded a company focused on using phone-based GPU computation on the edge. We created a game that we pitched on-stage at the GamesBeat 2013 Innovation Showdown, NVIDIA’s GPU Technology Conference, and competed at the Independent Games Festival 2014. The game used the edges and colors detected by the phone’s camera to let the user’s character cross obstacles and solve puzzles.

Smule’s Android Magic Piano

In 2012, I led the port of Smule’s Magic Piano music game to Android, which was my introduction to casual gaming. This was a top-ten Music app in the Play store for over a year after launching, had over 50 million installs.

Dropbox Android App

In 2010, I joined Dropbox as the 16th employee, built the Android app from 0 lines of code, and supported its growth to a few million users before we hired a second engineer. This was a tremendous opportunity to build a major consumer app from the ground up, and I learned a lot along the way. It now has over 1 Billion installs.

Snibbe Interactive “Magic Mirror”

I worked as the CTO at Snibbe Interactive in 2008–2009. One of the projects that I built was a video screen that would track the user’s posture and movement, and the displayed avatar would track you, trying to mirror your movements. The original version I worked on was for the Denver Museum of Nature & Science, but I can’t find a video of it online, so please find below a similar installation that was done after I left.

Older Work

Before this, I had a bunch of other projects I worked on:

  • Steganography in audio, image, and video codecs
  • Sold tom.com domain
  • First Java-based ad banners on the Internet
  • Part of the ISO standards committee for VRML (part of MPEG-4).
  • Technical director of WAXweb, the first feature-length film on the Internet.
  • Worked on the VR Virtual Windtunnel at NASA Ames.