February 10, 2016

UPDATE: Top 4 carriers combine for 15.9 TB of cellular data use at Super Bowl 50

New Verizon Wireless under-seat DAS antenna placement at Levi's Stadium. Photo: Verizon Wireless

New Verizon Wireless under-seat DAS antenna placement at Levi’s Stadium. Photo: Verizon Wireless

UPDATE, 2/8/16, 1:50 p.m. — We now have data totals in from all four of the major U.S. cellular carriers, and at Sunday’s Super Bowl 50, fans combined to use 15.9 terabytes of data on the networks in and directly around Levi’s Stadium in Santa Clara, Calif.

Leading the way in usage was Verizon Wireless with a claim of 7 TB used by its customers; AT&T was next with 5.2 TB of claimed usage, followed by T-Mobile with a report of 2.1 TB, and Sprint with 1.6 TB. All the carriers’ numbers are well above figures from last year’s Super Bowl, where by our reporting Sprint, AT&T and Verizon had a combined 6.56 TB of cellular data consumed during the big game. (We did not have any T-Mobile reports from last year.)

For all the carriers, the data apparently includes both traffic on the in-stadium distributed antenna system (DAS) network was well as any macro deployments outside the stadium in parking lot areas. The final total was well over double the 6.56 TB of cellular traffic seen at last year’s big game in Glendale, Ariz. We are still waiting for Wi-Fi numbers from the Levi’s Stadium networking crew but it’s a good bet the 6.23 TB number from last year’s game will be eclipsed and we will have a new single-game Wi-Fi record as well so stay tuned.

Though we did hear and see some scattered reports of network connectivity issues during the Denver Broncos’ 24-10 victory over the Carolina Panthers it appears the upgrade of the DAS Group Professionals DAS install at Levi’s Stadium with its gear mainly provided by JMA Wireless stood up to the biggest-ever test of traffic. Congrats to all involved.

Thanks also to the Verizon and AT&T crews who supplied us with tweet reports and emails Sunday night, it made for some entertaining in-game stats. Some tweets embedded below.

Commentary: Super Bowl 50 and the pursuit of stadium network statistics

Dan Williams talks Wi-Fi while the Levi's Stadium new turf grows silently behind him in this 2014 photo. All photos: Paul Kapustka, MSR

Dan Williams talks Wi-Fi while the Levi’s Stadium new turf grows silently behind him in this 2014 photo. All photos: Paul Kapustka, MSR

If there’s one person who didn’t get enough recognition in the run-up to today’s Super Bowl 50 at Levi’s Stadium, it’s Dan Williams, the former director of information technology for the Niners and the man who was largely behind the venue’s celebrated network design.

From our standpoint, Williams deserves credit not just for leading the deployment of a stadium network that set the standard for the future of connected venues, but also for being among the first to openly talk about the network’s performance, sharing statistics both good and bad during his short stay at Levi’s Stadium. His honesty helped open the door wider on a potentially rich stream of information that we think could greatly assist stadium network professionals everywhere, especially if more teams followed his lead and shared stats openly and honestly.

But the pursuit of meaningful network statistics remains a challenging process on many levels, and it’s one that is ironically likely to get more confusing today as multiple parties are set to release live network statistics from Super Bowl 50 at Levi’s Stadium. Both Verizon Wireless and AT&T are planning to release live game-day performance figures for their cellular networks in and around the venue today, and we also expect to get Wi-Fi network statistics from the San Francisco 49ers, if not during the game then shortly thereafter.

Niners VP of technology Dan Williams attempts to fix my Droid 4 Wi-Fi issues (while trying not to laugh at the fact that I actually have and use a Droid 4) during the first preseason game at Levi's Stadium, July 2014

Niners VP of technology Dan Williams attempts to fix my Droid 4 Wi-Fi issues (while trying not to laugh at the fact that I actually have and use a Droid 4) during the first preseason game at Levi’s Stadium, July 2014

While we salute the change of heart (especially from Verizon) to make more information available, we’d also like to make a plea here for more independent access to venues on game day, so there can be an independent, objective voice on hand to counter the “sanitization” or simple non-reporting of less-impressive data that is almost sure to occur when interested parties are talking about themselves. But unless team, league or venue IT representatives band together and craft recognized standard methods of reporting the most-interesting data, we’re going to be left to sift through an ever-increasing mix of figures provided by those with their own agendas on what’s important and what should remain hidden. We’ll do our best on that front, but it’d be better if we had some industry-wide help.

Out of school behavior provides the most interesting info

Like many other stadium network professionals we’ve been fortunate enough to meet and spend time with — a list here that includes folks like the San Francisco Giant’s Bill Schlough, Madison Square Garden’s Katee Panter, the Dallas Cowboys’ John Winborn, and Chip Foley, formerly with the Barclays Center, among many others — Williams from the start showed a passion for making the network better, so it could better serve the fans who used it. Our first face to face meeting came during the first preseason football game at Levi’s Stadium, when I tweeted that I was having problems connecting to the Wi-Fi network. William’s response? An email asking me where I was sitting, followed quickly by an in-person visit. Now that’s customer support.

The outcome of that interaction was a revelation that the Levi’s Stadium Wi-Fi network hadn’t prepared adequately for older devices that were still using the 2.4 GHz unlicensed bands; what was cool to me was not just that I kind-of discovered it by accident, but that Williams and his team admitted it publicly, then went about fixing it. The combination of objective reporting and honest confirmation, I think, worked to the benefit of all. But not everybody sees such things in the same light.

Season opener issues: Picture of app late in the first half.

Season opener issues: Picture of app late in the first half.

For instance, during the rest of Levi’s Stadium’s first year of operation, Williams shared with us not just raw “tonnage” numbers of Wi-Fi use but also interesting stats from the app, like exact numbers of replays watched and in-seat food delivery orders. Since those numbers were kind of low — in the low thousands for food deliveries and a decline in replay use after the season opener — they weren’t necessarily flattering to the owners of the system, even if you took into account the fact that some of this (like delivering to any seat in a 68,500-seat stadium) had never been tried before. In the end, even Niners CEO Jed York admitted that some elements of the stadium hadn’t performed as expected, as the stats we’d been seeing had borne out.

After Williams left the team, the Niners took control of the network stats reporting and hugely trimmed down the list of things they were willing to report, though we would also note that Roger Hacker and the Niners communications team are still at the top of the list when it comes to consistent, repeated reporting of the network stats they feel comfortable talking about. When there are big numbers to report, all goes well because everyone loves to talk about systems that deliver for fans as promised. But I can also tell you that when things go south, so does a lot of the honest communication, an unfortunate if understandable situation.

Who will tell you when things don’t work?

For Levi’s Stadium, the darkest night for the network came on Feb. 21, 2015, at the NHL’s Coors Light Stadium Series outdoor hockey game between the San Jose Sharks and the Los Angeles Kings. In what has easily been our highest-traffic post ever, we described in detail some of the big problems that surfaced that evening, which included breakdowns of the delivery service, the Wi-Fi network, the light rail boarding process, and part of Verizon’s cellular network. What’s funny about our reporting is that I hadn’t planned on “working” that night — I had bought full-price tickets for myself and my brother, and as longtime hockey nuts we proudly wore our Blackhawks jerseys to represent our hometown team.

Screen shot of Levi's Stadium app during hockey game issues

Screen shot of Levi’s Stadium app during hockey game issues

My plan was just to relax, take CalTrain and VTA to the stadium and show off the network by ordering us food and drink to be delivered to our seats, but when some obvious and incomprehensible network breakdowns were evident, I went into reporting mode, and eventually found out mostly what went wrong. My point here is that, without independent, objective reporting live from the scene, it’s doubtful that parties with vested interests — like cellular providers or owners of the stadium network — are going to honestly and quickly talk about breakdowns or failures. Yet it’s always the lessons learned from problem times that help the most; my worry is that with more “sponsored” or in-house statistics flooding the zone, the more valuable objective data will get lost or devalued.

How can this be fixed? For starters, teams and leagues that spend an inordinate amount of time and money promoting technical features of stadiums should at the very least credential some reporters who are interested in following up to see how said deployments perform. While we’re grateful for the few forward-thinking organizations that have given us the same access that sports reporters get, more often than not our requests for media access to events are denied.

Since we can’t be at every stadium, we also rely on teams, venues and network operators to provide stats when possible, and we encourage more participants to join the growing list who do provide us with network performance figures, like the IT teams at Nebraska and Texas A&M, among others. While we have also drawn some industry criticism for our headline focus on such stats, our response is that the more data we get and the more opinions on which data matter, the better we will get over time. We also encourage any and all interested parties to attend the SEAT Conference this summer in Las Vegas, where you can be sure we will have discussions about this topic and how to make stats reporting more useful for those to whom it matters most — the professionals who install and operate these networks so that fans can stay connected at events.

MSR on the scene at Sports Authority Field at Mile High in January

MSR on the scene at Sports Authority Field at Mile High in January

Niners: All (tech) systems go at Levi’s Stadium for Super Bowl 50

Levi's Stadium, ready for the Super Bowl. All stadium photos: Levi's Stadium (click on any photo for a larger image)

Levi’s Stadium, ready for the Super Bowl. All stadium photos: Levi’s Stadium (click on any photo for a larger image)

As far as the technology at Levi’s Stadium is concerned, it’s all systems go for Sunday’s Super Bowl 50, according to San Francisco 49ers chief operating officer Al Guido.

In a phone interview with Mobile Sports Report, Guido said the 2-year-old stadium’s vaunted technology underpinnings — especially the wireless connectivity for fans — is ready to go for the NFL’s biggest yearly event, after a second season spent mainly fine-tuning the different components.

“We couldn’t feel more confident, hosting the game,” said Guido, speaking specifically about the technology infrastructure at Levi’s Stadium. As he stated before the regular season began, the Niners didn’t do anything radical to the stadium’s Wi-Fi network, which uses gear from Aruba, an HP Enterprise company, to bring the main wireless bandwidth to fans.

And while the stadium’s distributed antenna system (DAS) got a complete replacement over the summer, the new capabilities including under-seat DAS antennas for Verizon Wireless should only lead to better reception than the year before. According to Guido, representatives from Aruba as well as from “all the carriers” will be on hand Super Sunday just in case anything needs close attention.

“Everybody’s going to be at a high tech [support] level” on game day, Guido said.

Drink delivery order page on Super Bowl stadium app, including the $13 Bud Light.

Drink delivery order page on Super Bowl stadium app, including the $13 Bud Light.


No food, but in-seat beverage delivery as part of stadium app

The Super Bowl 50 stadium app, designed for the NFL by the Niners’ in-house app development company VenueNext, will have some but not all of the features Niners fans have available during the regular season. The most obvious omission is the lack of food delivery to all seats, something that makes Levi’s Stadium stand apart from any other large public sporting venue. Instead, the stadium app will only allow fans to order beverages for in-seat delivery, with the option to order food, beverage and merchandise that can be claimed at “express pickup” concession windows.

According to Guido, the decision to only have beverage deliveries at Super Bowl 50 was one reached jointly by the NFL, the Niners and VenueNext, and the catering company for the stadium, Centerplate. Guido said that the potential “amount of education” for all the fans new to the stadium and new to the app led the league, the Niners and the caterers toward a path of greater simplicity, namely just having beverages available for in-seat delivery.

“It was a risk-reward decision about the amount of fan education needed,” Guido said. “There’s so much going on at a Super Bowl and so many people new to the stadium that it didn’t seem worth it to us to risk someone not getting an order delivered because of their error, or our error.” Guido added that with all the extra breaks in action for a Super Bowl, and additional concessions stands, “there’s enough time to get around” to get food.

View of the temporary media towers on the Dignity Health concourse

View of the temporary media towers on the Dignity Health concourse

Michelle McKenna-Doyle, senior vice president and chief information officer for the NFL, told Sports Business Journal that the league was also concerned about game-day delivery traffic patterns being disrupted by the new media towers that have been built for the game in the corner plaza areas of the stadium. “We were worried about having to keep up with demand … and we need to keep the aisles clear, which is important to the security team,” McKenna-Doyle said in a story by SBJ’s Don Muret.

The app will, however, include its normal live wayfinding capabilities, which should prove useful to new visitors to Levi’s Stadium since they can watch themselves walk through a map of the facility as a familiar moving blue dot. Like it does for Niners games, the app will also have instant replays from multiple camera angles available, as well as Super Bowl extras like a “celebrity cam” and the ability to watch Super Bowl commercials right after they are broadcast on TV.

Guido said the Levi’s Stadium app performed well all season, with an average of about “2,000 to 2,500″ in-seat delivery orders per game. What was especially pleasing to the team was the number of fans who used the app’s ability to support digital ticketing, a feature that makes life somewhat simpler for fans but exponentially better for the team, which can gain valuable marketing insight from digital ticket-use statistics. According to Guido almost 35 percent of fans used digital ticketing during the past season.

Media towers save seats for fans

Niners fans watching Sunday’s game on TV might be surprised by the media towers, which Guido said were built in the Intel and Dignity Health concourse areas, which during regular-season games are simply open spaces. Guido said the decision to build temporary facilities for media means that the regular stadium seats will be saved for fans. At many other pro championship or playoff events, the overflow media are often housed in regular seating areas.

“The NFL made a great decision there” to put the media in the pavilions, Guido said.

If there is one thing that can’t really be controlled, it’s the traffic and transportation issues of bringing fans to the game. On Sunday fans coming to the game will confront Levi’s Stadium’s unique location in the middle of many Silicon Valley corporate headquarters buildings, which presents challenges that stadiums like AT&T Stadium in Dallas or the University of Phoenix Stadium in Glendale, Ariz. — which are surrounded by acres of stadium-controlled parking lots — simply don’t have. To help with Super Bowl traffic the planners are using multiple methods, including using Google employee buses as shuttles as well as signing Uber as a sponsor with its own dedicated pickup and dropoff lot. There is also light rail service which stops right outside the stadium, which intially in the past experienced lengthy delays especially after games, but has improved over time.

“Traffic and transportation is our largest concern,” Guido said.

Bring on the players and fans!

Bring on the players and fans!

Stadium Tech Report: Connectivity soars at Denver Broncos’ Sports Authority Field at Mile High

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Panoramic view of Sports Authority Field at Mile High from the top seats. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

On most of our stadium visits, we have to wait until we get inside the venue to start testing the network. At Sports Authority Field at Mile High, however, we barely got out of the car before the Wi-Fi auto-connected — at superb speeds. Nothing like a network that announces itself before you get in the door.

The parking-lot connection — at a download speed of 45.48 Mbps and an upload speed of 53.35 — was the first clue that football fan connectivity is taken seriously in Denver, especially so if you have Verizon service. While the stadium’s Wi-Fi network is currently only available to Verizon customers — more on this later — full DAS participation by the three other major U.S. wireless carriers means that pretty much any visitor to the venue is going to have good, if not great, connectivity for their mobile device, no matter which service they use.

Inside the stadium, a trained eye can spot many different types of DAS and Wi-Fi antenna placements, under overhangs, on towers, on ceilings and on walls; and thanks to a first-person stadium tech tour conducted by Russ Trainor, vice president of information systems for the Denver Broncos, we got to learn about a wide range of not-so-noticable antenna deployments, including in railing enclosures and on field-level walls, all part of an ongoing plan to try to stay ahead of the still-growing demand for mobile data from sports fans who come to the games.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The day we visited, during the last regular-season game on Jan. 3, was important for the Broncos as a team since their 27-20 victory over the San Diego Chargers gave Denver home-field advantage through the playoffs, an edge that helped the team reach its eighth Super Bowl. But even as he celebrated his team’s win, Trainor was happy for another reason: the bye week gave him and his team more time to light up some new Wi-Fi and DAS antenna placements, to better handle the expected and eventual playoff data crush.

“You can never have enough APs,” Trainor said.

Good Wi-Fi, but still only for Verizon customers

Opened on Aug. 11, 2001, with a concert by the Eagles, the then-named Invesco Field at Mile High replaced the old Mile High Stadium in basically the same spot, sitting at 5,280 feet above sea level. Seen by many on TV when it hosted the 2008 Democratic National Convention and the acceptance speech of then-Sen. Barack Obama, the “new” Mile High has seen more than 12 million fans come through its doors since it opened for a variety of sports and entertainment events.

But true high-speed wireless for fans didn’t take root until 2012, when a revamp led by Verizon Wireless and the Broncos’ IT staff added a Cisco-based Wi-Fi network to the stadium with 500 access points, designed to serve 25,000 concurrent users and also designed to be “open,” allowing any other carrier to provide access to its customers by negotiating a deal with Verizon. While Trainor said the option still remains open and talks with some of the other carriers are underway, none have yet signed on — making the Wi-Fi network a fast playground for Verizon customers, who apparently are in the vast majority in the Denver region.

Sorry, AT&T customer, no soup for you

Sorry, AT&T customer, no soup for you

We don’t have any exact proof of that thinking, but statistics from the recent AFC Championship game at Sports Authority Field — a 20-18 Denver victory over the New England Patriots — seem to show Verizon customers in a bit of a majority. According to Verizon, its customers at the game used a total of 2.87 terabytes of data, with 1.7 TB on the Wi-Fi network and another 1.17 TB on the Verizon LTE DAS network. AT&T, by comparison, said its customers used 819 GB on the AT&T DAS network that day. So either there are more Verizon customers at the stadium on game days, or Verizon customers use more data because they have more network options; take your pick.

With our Verizon iPhone 6 Plus in hand, we found great connectivity on Wi-Fi pretty much everywhere we roamed. After finding our way from the parking lot to the press box, we got a signal of 46.46 Mbps down and 46.90 up, this from the regular fan network in the stands and not from the press-only Wi-Fi network.

While roaming through the plush United Club we got a speed test of 33.36/35.19, a figure that Trainor said could change on any given game day — “when it gets cold outside, this place fills up,” he noted — and then later when we walked up to the top, 5th-level concourse, we still got a Wi-Fi signal of 34.96/30.40 on the walkways behind the seats. During second-quarter action we even sneaked up to the nosebleed seats in section 501, one of the ski-slope steep sections near the stadium’s top edge — and still got a Wi-Fi signal of 10.28 Mbps/5.00 Mbps.

According to Trainor, the upper seats are among the toughest challenges for Wi-Fi reception, especially those in the “bulge” areas in the middle of the stadium where on both sides the sections curve upwards, adding more seats. Though the light structures that wind all the way around the stadium do provide good spots for antenna mounts, the bulge areas are harder to reach, and in the near future Trainor and his team will keep experimenting with other methods of deployment, like railing enclosures and row-end mounts they have used successfully for both Wi-Fi and DAS in other areas of the stadium.

Lots of antennas visible in this overhang area

Lots of antennas visible in this overhang area

One interesting architectural quirk of the stadium — its use of metal decking instead of concrete — actually helps the wireless deployment team, Trainor said. Installed to mimic the metal upper deck at the old Mile High Stadium — where Broncos fans would do the “Denver Stomp” to produce thunderous noise — the metal construction acts as a barrier to keep Wi-Fi signals from the bowl from interfering with those from antennas inside suites and concourses, Trainor said.

While most of the stadium has favorable locations for overhead antennas — there are three main levels of seating, providing two expansive overhangs covering about 80 percent of the seating area — some typical problem places like seats near field level and in the no-overhang South stands have required some creative thinking, an excercise that never really ends.

“We started with 500 Wi-Fi APs, and we’re now at 640, and by the time we get it [the current plan] all built out we’ll have about 850 to 900 total,” Trainor said.

DAS deployments a mix of connectivity

On the DAS side, Trainor said that the four major carriers — Verizon Wireless, AT&T, Sprint and T-Mobile — are all present inside the stadium, with different antenna placements in different numbers. In some instances, all the carriers use “neutral” antennas, mainly in areas where there isn’t enough room for exclusive deployments. But in other areas, the carriers have installed their own antennas, an arrangement that allows them to replace and upgrade them as necessary at their own discretion, Trainor said.

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

Field-level Wi-Fi AP (small white box next to right leg of Peyton Manning fan)

We didn’t have a Sprint or T-Mobile device on hand, but our AT&T Android phone had good connectivity everywhere we measured, including a 4G LTE signal of 27.94 Mbps down and 6.86 up in the press box, and signals of 47.83/6.37 on the same 5th-level concourse area where we tested the Verizon Wi-Fi.

All the carrier back-end gear is housed in a brick building built outside the southeast side of the stadium, Trainor said, since there wasn’t room inside the stadium structure itself. DAS and Wi-Fi antennas also exist in great number in the vast parking lots that directly surround the stadium, as well as in the “fan zone” gathering area outside the South stands.

Like with the Wi-Fi, Trainor and his team are always planning for more DAS capacity, even if contracts aren’t signed yet. On the new railing enclosures they are installing, the Denver IT team builds in enough space for both DAS and Wi-Fi, even if only one network is using the deployment to start with. Again, you can never have enough antennas — or enough places to put them.

YinzCam app and Cisco SportsVision

Rounding out the mobile-device offerings is not one but two YinzCam team apps, one for use at outside the stadium and the other one for live game-day offerings, with a geocache feature that allows the team to provide content it has stadium rights to, like the NFL’s RedZone channel. Both apps have live links to the Broncos radio coverage from KOA Radio, and the in-stadium instant replay feature worked superbly during our visit, showing plays in seconds and often before they appeared on the stadium’s big screens.

In the concourses we recognized the split-screen capabilities of Cisco’s StadiumVision technology, which can direct programming to all the TV screens inside a stadium. Another nice touch in the United Club was a circular charging station, with tabletop space so fans could have a place to put food and drink while waiting for their devices to juice back up. “We are always looking for ways and configurations to allow fans to recharge their devices,” Trainor said.

With all its different parts, the wireless deployment at Sports Authority Field at Mile High adds up to a favorable fan experience, one that clearly has the ability to keep getting better on an incremental basis. But like their Super Bowl team, Denver fans should be happy with what they have right now.

MORE PHOTOS BELOW

mihi7

Railing antenna enclosure. Some of these have both Wi-Fi and DAS.

mihi11

App promo on the scoreboard

mihi13

Panoramic view of the stadium and the city

mihi4

South stands have a horse and Wi-Fi antennas on the top of the scoreboard

mihi20

Cisco SportsVision in action on 6-panel display

mihi22

DAS antennas on end-of-row railing area

mihi24

Game on, phones out!

mihi25

Antennas covering the concourse area on second level

mihi26

More SportsVision and Wi-Fi deployment in the United Club

mihi14

Only accept on the scene reporting!

Verizon: Denver fans used 2.87 TB of wireless data during AFC championship; AT&T also sets DAS traffic records at both Sunday games

Sports Authority Field at Mile High, during Jan. 3 game vs. San Diego. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Sports Authority Field at Mile High, during Jan. 3 game vs. San Diego. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

During their team’s exciting 20-18 victory over the New England Patriots Sunday, Denver Broncos fans who are Verizon Wireless customers used 2.87 terabytes of wireless data, according to Verizon. That total includes 1.7 TB used on the Verizon-only Wi-Fi network at Sports Authority Field at Mile High, and another 1.17 TB of data on the Verizon LTE DAS network at the stadium.

Also on Sunday, at the Bank of America Stadium in Charlotte, N.C., where the Carolina Panthers defeated the Arizona Cardinals, Verizon Wireless customers there used 1.3 TB of data on the Verizon LTE DAS network, according to figures sent to us by Verizon.

According to AT&T, its wireless customers set DAS traffic records at both stadiums Sunday, with 819 GB used on AT&T’s DAS network at Sports Authority Field and 739 GB used by AT&T DAS customers at Bank of America Stadium. Both totals are the highest-ever marks seen by AT&T at the respective stadiums, according to AT&T; the Denver total Sunday was 34 percent higher than the total used in the team’s first playoff game this season against Pittsburgh, and was 52 percent more than the average data used during regular-season games. In Charlotte, the DAS traffic total Sunday was 16 percent higher than the number hit during the playoff game a week previous against Seattle, and 50 percent higher than the average regular-season game, according to AT&T.

On the AT&T Wi-Fi network at Bank of America Stadium, fans used 740 GB of data, which AT&T said is also the highest-ever mark for that network, 19 percent higher than the previous playoff game vs. Seattle and 36 percent more than the network saw for average regular season games. The Wi-Fi network at Bank of America Stadium will be replaced this offseason, with a new network built by AmpThink and Aruba.

In Denver, the network situation is somewhat unique since Verizon built the Wi-Fi network inside Sports Authority Field at Mile High, but so far only Verizon customers are allowed access to it. While we’ll describe the situation in more detail in a stadium-visit profile coming very soon, the word from stadium IT types is that the Wi-Fi is open to other carriers but none have yet signed on to allow their customers access to it. There are, however, separate DAS networks for AT&T, Sprint and T-Mobile inside the stadium as well, so at least from a DAS perspective most fans at Mile High are pretty well connected.

Stay tuned for more soon on the networks at Sports Authority Field at Mile High! In the meantime, some pictures from our Jan. 3 visit below.

mihi3

Count the antennas! See if you can spot the AT&T DAS antennas (slightly rounded) and the T-Mobile antenna (big square) among others in this overhang shot at Sports Authority Field at Mile High.

mihi4

Denver’s south end zone scoreboard is topped by a horse… and two Wi-Fi antennas

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

Your Levi’s Stadium technology primer: Everything you need to know about wireless technology at the site of Super Bowl 50!

Scoreboard promo for the Levi's Wi-Fi network, from 2014 season. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Scoreboard promo for the Levi’s Wi-Fi network, from 2014 season. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

With Super Bowl 50 two weeks away there is going to be increased interest about whether or not Levi’s Stadium in Santa Clara, Calif., host site of the event, is “the most technologically advanced” athletic venue. Here at Mobile Sports Report, we have spent the better part of the last two years researching and reporting on Levi’s Stadium and all its technical components, attending multiple events and compiling all the known statistics we can find, to present as complete and as honest an assessment as possible, from a completely objective perspective, about the technology found at Levi’s Stadium.

So what’s our verdict? As we see it, there are three main features that set Levi’s Stadium aside from most others, and qualify it for consideration as one of the most technologically advanced large public venues: The stadium’s Wi-Fi network, its distributed antenna system, or DAS, and the integrated Levi’s Stadium app, which takes advantage of a large network of beacons to provide wayfinding and other location-based features. Though some of the components, like the Wi-Fi network, may not be the fastest or largest around, it’s our opinion that the sum of the parts puts Levi’s Stadium at or near the top of any well-connected stadium list; but the 2-year-old venue’s real test won’t come until Super Sunday, when we’ll all see if the networks, apps and personnel performance can live up to the stress of one of sport’s biggest events.

For anyone who wants to know the exhaustive details behind the technology, we’ve included in this story links to all of our Levi’s Stadium stories we think are pertinent, to help other writers or interested sports-tech types get a grip on what’s really going to be technologically available to the 72,000 or so fans who show up on Feb. 7 to watch the NFL’s 50th annual big game.

For starters, here is the first part of a feature we did at the start of the season about how Levi’s Stadium was getting ready for Super Bowl 50. Though we expect some more news next week about late additions, this article pretty much sums up the first-year performance and the tweaks the San Francisco 49ers made to their home-stadium’s wireless infrastructure. And here is the second part of the feature, which focuses more on the stadium’s excellent app, which we’ll talk more about later.

Fans take pictures at Levi's Stadium, opening day 2014 season.

Fans take pictures at Levi’s Stadium, opening day 2014 season.


Wi-Fi: It’s good, but is it the best?

The Wi-Fi in the stadium is pretty good, among the best out there anywhere, but probably not the biggest or fastest network in all the land. Though the Aruba-gear network was innovative for its heavy use of under-seat Wi-Fi APs and the 1,200 APs it had for its first year, other stadiums are meeting or beating those numbers, and under-seat deployments are now becoming quite trendy for venues that want fast, wide connectivity. With slightly more than 1,200 APs now, the Levi’s Stadium Wi-Fi network has seen some big-traffic days for Wi-Fi, including the stadium’s NFL regular-season opener and a WrestleMania event last year.

Among stadiums we’ve seen, AT&T Stadium in Arlington, Texas, has more Wi-Fi APs and is a bigger place (by about 30,000 in capacity for football) so it had more overall Wi-Fi traffic than Levi’s the past couple years. And Kyle Field’s new network down at Texas A&M is the fastest we’ve seen anywhere, and already has had a bigger Wi-Fi traffic day than Levi’s Stadium. And we haven’t yet visited Miami’s Sun Life Stadium but they get a lot of wireless traffic there too. So while Levi’s Stadium may be among the best, we’re not quite sure it is at the top of the list, at least when it comes to sheer Wi-Fi connectivity.

We might change our tune if the Super Bowl 50 crowd can top last year’s Super Bowl Wi-Fi traffic total of 6.23 terabytes, recorded at the University of Phoenix Stadium in Glendale, Ariz. But so far the biggest total recorded at Levi’s Stadium was 4.5 TB seen at the WrestleMania 31 event last March. From our unofficial observations, the “top 5″ list of most single-day Wi-Fi events we know of are:

1) 6.23 TB — Super Bowl XLIX, University of Phoenix Stadium, Glendale, Ariz., Feb. 1, 2015
2) 5.7 TB — Alabama vs. Texas A&M, Kyle Field, College Station, Texas, Oct. 17, 2015
3) 4.93 TB — College Football Playoff championship game, AT&T Stadium, Arlington, Texas, Jan. 12, 2015
4) 4.9 TB — College Football Playoff championship game, University of Phoenix Stadium, Glendale, Ariz., Jan. 11, 2016
5) 4.5 TB — WrestleMania 31, Levi’s Stadium, Santa Clara, Calif., March 31, 2015

For what it’s worth, Super Bowl XLVIII in MetLife Stadium used only 3.2 TB of Wi-Fi, so it will be interesting to see what happens with the growth curve at SB50. In addition to total data “tonnage” there is also an interesting observation about how much data is used per fan, on average. When we get the stats back from Super Bowl Sunday it will be interesting to see if the smaller crowd at Levi’s Stadium will have used more data per connected person, a good reflection of both the carrying capacity of the network and the ease of connecting and staying connected to the Wi-Fi.

Replacing the entire DAS for better cellular connectivity

What is often confusing to non-tech types who try to write about stadium wireless is realizing that there are often two separate networks, Wi-Fi and cellular, operating in the same venue. While many fans actively seek out Wi-Fi, many game-day attendees either don’t bother or don’t know how to connect to Wi-Fi, and so just use their phones like they do anywhere else. To make sure they still have a strong signal, wireless carriers and venues often team up to deploy a distributed antenna system, or DAS, which is basically a bunch of small antennas located inside the venue that act just like a big cell tower, connecting phones to the nearest antenna.

Close-up of new DAS antennas (from mid-July, before the wires were connected)

Close-up of new DAS antennas (from mid-July, before the wires were connected)

At Levi’s Stadium, integrator DAS Group Professionals (DGP) built a “neutral host” DAS for the stadium, which means the team owns the infrastructure and rents out space to carriers so they can connect to customers inside the building. One of the more interesting twists this past offseason was that DGP ripped out and replaced the entire DAS network it built the year before, at the behest of its customers, the major cellular providers. Why? According to DGP, the cell providers — who paid for the upgrade — are expecting as much as 2.5 times more cellular data at this year’s Super Bowl compared to last year, huge numbers that they were afraid might overwhelm the system installed in 2014.

During a stadium tour this summer, MSR saw that the main Levi’s Stadium head end (where the telecom gear that connects the stadium to the outside networks lives) was being doubled in size, so by any stretch cell connectivity should be good if not great during the big game. DGP was also supposed to be increasing cell coverage outside the stadium in the parking lots, but so far we haven’t heard any reports if reception was better this year than last.

At big events like the Super Bowl, the big wireless carriers will spend like crazy to make sure there are no reports of “phones not working,” so the DAS upgrades have become somewhat par for the course. AT&T said that it spent $25 million on wireless infrastructure improvements in the greater Bay area, including expanding its DAS operations inside Levi’s Stadium to allow them to handle 150 percent more traffic. You can expect that Verizon was spending some similar dollars, so rest assured, if you are there your phone will more likely than not find a signal.

What will the app let you do?

The biggest question remaining about the technological underpinnings of Super Bowl 50 — at least as of Sunday night — is whether or not all the features from the regular-season Levi’s Stadium app will make it into the mix for the Super Bowl, especially the one that really sets Levi’s Stadium apart, the ability to order food to be delivered to any seat in the stadium.

Though we’ve been given a “head nod” that the service will be available for Super Sunday, we haven’t yet received any official notice of what’s going to be in the game-day app either from app provider VenueNext or the NFL. This season Niners fans at home games could not only order food and drinks for themselves, they could order and pay for food to be delivered to friends in the stadium, something we noted in our season preview of changes to the groundbreaking Levi’s Stadium app.

App showing ability to buy pricey parking ticket for your RV

App showing ability to buy pricey parking ticket for your RV

If there is some doubt whether the league and the stadium might not make food-delivery available for the Super Bowl, it might have to do with the fact that at one of last year’s “big events” at Levi’s Stadium, the NHL’s Stadium Series outdoor hockey game, the food-delivery service melted down in the face of a massive amount of orders and a too-low level of human staffing. But our guess is that eventually (maybe this week?) we will hear that the Super Bowl app will embrace all the features of the regular Levi’s Stadium game-day app, including in-seat delivery.

What many fans at the game may find even more useful is the app’s ability to provide wayfinding capabilities through a mapping feature that uses the 2,000+ Bluetooth beacons installed throughout the venue to provide live wayfinding, just like how Google Maps shows your car as a blue dot driving down the highway. With many attendees most likely visiting the stadium for the first time, having the ability to find your way around via your device may be the most welcome reason to download the app. Fans should also be able to watch in-stadium replays seconds after plays happen, and may also be able to watch Super Bowl broadcast commercials via their mobile device. Stay tuned for more “official” announcements of app capabilities as we hear them.

In case you haven’t heard enough, here are a few more links from our in-person visits to Levi’s Stadium for Niners home games during the 2014 season.

Niners’ home opener tops Super Bowl for Wi-Fi data traffic with 3.3 Terabytes (Sept. 16, 2014)

Levi’s Stadium ‘NiNerds’ get high-visibility wardrobe upgrade (Nov. 23, 2014)

Stadium Tech Report: Network finishes season strong at Niners’ Levi’s Stadium (Jan. 12, 2015)