Jack McCauley understands the interplay between video game hardware and human interaction like few others in the industry. He designed the guitar and drums for Red Octane’s (later Activision’s) Guitar Hero video game series. As co-founder and chief engineer of Oculus VR, he designed the Oculus DK1 and DK2 virtual reality (VR) headsets and helped guide the company through its acquisition by Facebook in 2014. Now active in automotive technology, Jack McCauley builds cars at Black Lab, his private R&D facility and hardware incubator in Livermore, Calif. And, in no small feat, he thinks he’s solved the head-tracking problems in augmented reality (AR)/VR headsets – which he’ll demonstrate during his keynote presentation, MEMS Applications in Augmented Reality, October 6 at MSEC 2020, SEMI’s first virtual MEMS & Sensors Executive Congress. The event is October 6-8 and 13-15, 2020, and registration is open. SEMI interviewed McCauley to preview his presentation. Register now for MSEC 2020.
SEMI: What inspired you to become the first person to use a MEMS sensor in a gaming device?
McCauley: When I started designing the Guitar Hero peripherals, I had intermittent problems with motion tracking. I switched to a Freescale single-axis accelerometer, developed some IP around it, and that fixed the problem. That’s how I became an early customer of MEMS.
SEMI: When you pioneered immersive VR gaming experiences at Oculus VR, tech industry analysts predicted the widespread adoption of VR for gaming. What do you think happened?
McCauley: There are a lot of reasons why VR hasn’t become the standard-bearer for gaming. Gaming used to be a solitary activity, but as companies like Microsoft and Sony got behind multiplayer gaming, we realized many gamers found the social aspect more important than the visual aspect. Many gamers are content to play on a 2D screen or on multiple monitors because they’re playing against many people. The proliferation of internet connections worldwide has also promoted the kinship and social aspect of gaming.
SEMI: Do you think VR has a place in other applications?
McCauley: I think it has a lot of potential in real estate, VR movies, and engineering and design, among other areas. The automotive designer Henrik Fisker, for example, created whole vehicles in a game-engine model. If you wanted to buy one of his cars, let’s say, you could change the color and upholstery, for example, and then view it in a VR environment.
SEMI: One of the biggest obstacles to VR adoption is the motion sickness some people experience during gameplay. What would you do to fix that?
McCauley: The vestibular system in the brain, which uses the inner ear, is crucial to helping you balance. If there’s a mismatch between what your eyes see and your brain is perceiving, you’re likely to feel dizzy.
I’ve built a VR headset that uses a MEMS pico projector with micromirrors and a small laser for position tracking as well as for facial tracking and modeling. But the platform’s not for sale.
Still, many of the technical advances that we’ve made in VR are helping us with AR development. The increasing power of mobile chipsets and GPUs, the decreasing geometry for individual transistors and the way specific chips are processed, screen interfaces that will drive a 4K panel at a high frame rate, plus MEMS devices inside the eyewear for rotations and tracking are all helpful innovations.
SEMI: When designing cars in your own lab, you’re doing a lot of work with AR. What do you think of AR’s commercial viability?
McCauley: I know there are well-funded AR programs in place at major companies. That’s because mobile-device companies want an omnipresent phone in front of your face. I thought Google Glass, for example, was brilliant, but it was way too early for that product, and there was too much hype behind it.
AR is appealing because it lets people see through a screen – and have objects appear on that screen – while they are moving through space. My son actually came up with one of the ideas I’m implementing in a car I’m designing. We were driving in Spain, and he suggested that instead of using Google Maps to show me driving directions – which would force me to look down at an infotainment display – a sign could appear on AR glasses that would instruct me how to drive to Italy. That’s just an example of how we’ll use AR.
SEMI: After you sold Oculus VR to Facebook, you began investing time and resources into engineering education. Why did you make that choice?
McCauley: I’m originally from a blue-collar family, and then I got an education at Berkeley. That made a major difference in my life. When I sold Oculus, I donated to education-focused charities primarily, because education can lift an entire family out of poverty. Let’s say your family is farmworkers, but you get a degree in engineering and land a job at Apple. That could produce a ripple effect. As other members of your family and people in your community see the benefits of your education, they’re more likely to get an education, too.
SEMI: What would you like MSEC attendees to take away from your presentation?
McCauley: I appreciate what the MEMS industry has done for VR because if Oculus didn’t have nine degrees of freedom (9DoF) IMU, no one would have bought our company.
A new application will come along sooner or later that will require a different type of MEMS technology, and I have total confidence that the MEMS industry will deliver what’s needed.
For more information on McCauley’s R&D projects or on his position as Innovator in Residence at UC Berkeley’s Jacobs Institute for Design Innovation, visit his website.
MEMS & Sensors Industry Group® (MSIG), a SEMI technology community that connects the MEMS and sensors supply network in established and emerging markets, enables members to grow and prosper. Visit us today.
Maria Vetrano is a PR consultant at SEMI.
