A Bridge to the Metaverse: GDC 2023 Shows Off Tech to Make Games More Immersive and More Real

The Game Developers Conference returned to San Francisco this year with attendance back at pre-pandemic levels, and hundreds of exhibits lining the underground halls of the Moscone Center.

As players lined up to try the latest VR demos from big sponsors including Meta and PlayStation, one of them summed up a major theme of the show: "We're building games that interact with our actual world."

Back in the 1980s and 1990s, players could only enter the virtual worlds of video games through a handheld controller and a living room TV.

"It was the golden age," quipped a developer playing Super Mario World on a Super Nintendo at the conference. "We thought these graphics looked like real life."

The Super Nintendo console was part of an exhibit of 8-bit and 16-bit video games set up by the Museum of Art and Digital Entertainment in Oakland.

"We hope to kind of inspire that next generation by helping them look back in a way that's useful," said Rob Curl, the museum's historian. "People had to be creative in a different way when they were making games for these older systems."

Video games of the 1980s didn't have music recorded with acoustic instruments, or lifelike 3D characters. But this year's GDC featured an entire section of the Moscone South convention hall devoted to game audio, and an eye-catching display of skateboarders doing tricks on a ramp as high-speed infrared cameras recorded them from numerous angles.

"To measure their position in 3D, in real time, so that we can capture their exact movements," explained OptiTrack sales director Thierry Chevalier. "We would then be able to get pro-level skaters to have their own signature tricks within the game."

3D motion capture is also playing a role in developing video games based on a hip hop dance show called "Fray" about the life and challenges faced by two brothers who build video games.

"It just started as, 'Hey, we should put a video game in the show — that would be interesting creative storytelling,'" said Kate Duhamel, the founder and creative director of Candy Bomber Productions. "And then it was like, 'No, we should actually make a real game.'"

They wound up making two games — including the mobile rhythm-based game "Fray Jam," now available on Apple's App Store, and a more intricate PC game that's now in the final stages of production, Duhamel said. The games follow the storyline of the dance show, and share some of the choreography too.

"You turn on music and you ask them to freestyle," Duhamel said of the motion capture process. "So you create a whole library of movement ... for the avatars in the game."

Across the street from the conference, Epic Games showed off its latest tech for facial performance capture and photo-realistic human avatars. It was Epic's first time giving a keynote at GDC in four years, and the presentation focused on the latest in 3D rendering tech, and ever-expanding metaverses including the virtual world of Epic's own hit game Fortnite.

With all the focus on emerging technologies like VR and AR headsets and NFTs, Epic founder and CEO Tim Sweeney reminded the audience that millions of people already immerse themselves in wide-open metaverses every day using only their smartphones and desktop computers.

"(There are) over 600 million active users in these virtual worlds," Sweeney said. "This revolution is happening right now."

But the new technologies are evolving quickly. At GDC, the latest virtual reality hardware on display featured eye and hand tracking, gloves and vests that thump and vibrate when a player gets hit, and a new motorized swivel chair that lets players move around in virtual space without bumping into real-life walls and coffee tables.

"You're stationary, but you have infinite movement," explained PNI sales director Won Cho as he demonstrated the new chair. "When you press the pedal down, (your game character) goes forward. When you press it backwards, it would go backwards."

Down the hall, new and experimental forms of game control were on display at GDC's alternative controllers exhibition, aptly named "Alt.Ctrl."

"We created this whole installation just to play our game," explained Sara Brugioli, gesturing toward a contraption that resembled a life-sized grocery cart stuck in a doorway, made entirely out of plywood.

The wooden shopping cart is part of a two-player video game called "Grocery Trip" that began as a student project. One player pushes and steers the cart, while a second player sits in the cart, grabbing items from shelves and punching other customers who get in the way. Brugioli said it was inspired by real-life events.

"A guy at the shopping center actually stopped in front of me," in the middle of the aisle to check his phone, Brugioli explained. "And I was like ... this is so annoying. I wish I could punch you right now."

Brugioli said that's when the proverbial light bulb went on.

"And I was like, 'Oh my God, that would actually be a good game,'" she said.

Some alternative control games could end up being made into arcade machines, while others serve to experiment with new ways of getting players to interact with each other.

"Haber Dasher" is a research project by Ph.D. candidate Erin Truesdell that requires two players to jointly steer a single avatar by placing their heads inside a giant shared hat. Nearly the size of a small car, the hat hovered high above the other exhibits, dangling from the ceiling on heavy duty steel cables. Players shouted directions to each other as they attempted to tilt the hat brim in different directions to navigate.

"It's been really great watching people make friends in line as they find someone to play with," Truesdell said. "It's a hat that's also a networking opportunity."

Like most of the alternative control games on display, Haber Dasher is built to get players engaged and immersed in the game through how they control it — not through photorealistic graphics or surround sound. Rob Curl was quick to point out that games of the 1980s and 1990s didn't have that sort of realism either — and yet, today's kids still enjoy playing them.

"These games are still fun," he said. "They still provide that similar sort of emotional reaction. No matter your age."