During the final home game for the San Francisco 49ers this season, I was somewhat amazed to see a replay appear on the Levi’s Stadium app even before the next play had concluded. Clearly, the VenueNext team behind the app had progressed significantly from the early season, when the app’s replay function struggled somewhat.

During the Dec. 28 game against the Arizona Cardinals, the VenueNext team invited Mobile Sports Report to Levi’s Stadium for a behind-the-scenes look at how the replays are generated; here’s a quick take on how the video moves from playing field cameras to fans’ phones in a matter of seconds.

Step 1: Capturing what the cameras see

How do you make sure you have replays ready? You start by basically capturing all the action from all the cameras in the building. Those views are shuttled back to the main Levi’s Stadium video room, where 15 people have the somewhat enviable job of watching views from all the live cameras in the stadium. (We still haven’t heard a response to our offer of bringing a cooler in exchange for a spare seat next season.)

Inside the video room at Levi’s Stadium. (Click on any photo for larger image) Credit all photos: Paul Kapustka, MSR

According to VenueNext CEO John Paul, who led our tour at Levi’s, VenueNext has its own servers to cache video — here is a look at the Elemental gear used to store the video for the app’s replay function. Paul said that so far, VenueNext has only needed a little more than half of the capacity of the four servers pictured.

Elemental gear in data room next to video room at Levi’s Stadium

Step 2: Labeling and thumbnailing the replay

While most of this kind of gear is not new to broadcast operations, the interesting part from an app perspective is the next layer, the choosing and human editing. One of my bigger questions was about how you exactly get a replay set up, labeled with a headline and a thumbnail, and into the app within seconds? The answer is: You write another app, and have two Android-based tablets with football-savvy operators in the press box so they can see the plays live and quickly attach the appropriate info to the replay clips.

In the two pictures below, you can see an over-the-shoulder view of the two tablet apps; the first one is a sort of play-by-play generator, which the operator uses to label the next replay clip as either a run, pass, punt, incomplete, etc., so that a fan looking at the app can quickly figure out what the play might be (“45-yard TD pass,” or something like that). On the second tablet app, another operator sees instant thumbnails gleaned from the video room from all the different camera angles, and when the play is over, that operator picks a thumbnail with a click and the replay is on its way to the app.

First replay tablet app, which adds info about the play

Second replay tablet app, which adds a thumbnail to the replay

Since we only had a few minutes to watch the replay team in action (and since Paul was simultaneously hosting a group of representatives from a prospective future sports-team client), I didn’t get to ask detailed questions about how this all works but it does give you some idea. What’s impressive about the Levi’s Stadium app is that it offers four different camera angles of the same play, as well as a multi-cam view that shows all different angles at once. Though I’m not a fan of the all-at-once option I did find it enjoyable to watch a reply from multiple different perspectives, like watching a pass being caught from both a sideline and end-zone view.

Step 3: Educating fans how to use replay

Initially one of the most-hyped features of the planned stadium app, the replay function has seen only limited use this season, making it somewhat of a disappointment to Niners CEO Jed York. From our perspective, there were two big things holding back extended use of the replay function: The first was the ever-changing appearance of the app itself, which felt like a beta project for most of the season, with a different interface almost every single game; the second thing holding people back from using the app to view replays are the two excellent humungous video boards at Levi’s, which not only show live game action but also quickly show replays, sometimes from those multiple angle views as well.

There might even be a third thing holding fans back from trying out the replay function — that would be just getting fans to use the stadium app in the first place. While usage of the cellular and Wi-Fi networks started at record levels and remained strong from game to game, most fans at Levi’s Stadium were using their devices for other apps, like Twitter, Instagram, Facebook and good old email. As the season progressed, the team started promoting the app and the network more aggressively, adding more-frequent messages to the main video boards and dressing its stadium tech support crew (the “NiNerds”) in neon vests for easier recognition.

What’s next for next season? My guess is that with a year under their belt, the Niners and VenueNext will have a more-aggressive campaign in 2015 to steer more fans to the replay function, which is truly a wonderful enhancement to the game-day experience.