Vikings hit peak of 4.32 TB for Wi-Fi use at U.S. Bank Stadium, with average 43 percent take rate

Game day at U.S. Bank Stadium. Credit all photos: Vikings.com (click on any photo for a larger image)

Game day at U.S. Bank Stadium. Credit all photos: Vikings.com (click on any photo for a larger image)

While the football season may not have gone exactly to Vikings’ fans wishes, the Wi-Fi network at U.S. Bank Stadium performed well during its inaugural NFL season, with a peak single-game data total of 4.32 terabytes used, part of a season average of 2.89 TB used during Vikings games.

According to statistics provided to MSR by Tod Caflisch, vice president and chief technical officer for the Vikings, the biggest data-use day was Sept. 18, 2016, during the regular-season home opener for the Vikings against the rival Green Bay Packers, a 17-14 Vikings victory. That contest also saw season highs for unique Wi-Fi users, with 31,668 fans connecting to the Wi-Fi at some point of the game day, and for most concurrent users, with 17,556 users connected at the same time. The 31,668 number represented a 49 percent take rate, with the game’s reported attendance of 64,786.

Even though Caflisch said the Vikings didn’t heavily promote the AmpThink-designed Wi-Fi network — which uses Cisco Wi-Fi gear in mostly handrail-mounted AP locations to serve the main bowl seating areas — the average take rate during the season was at the high end of numbers we’ve seen, with a 43 percent average over the two preseason and eight regular-season Vikings games.

Screen Shot 2017-01-12 at 11.41.21 AMAnd even though the total data-used number only crested 3 TB one other time in the season — a 3.16 TB mark during a 30-24 Vikings win over the Arizona Cardinals on Nov. 20, 2016 — the average mark of 2.89 TB per game showed solid, consistent use.

Caflisch said that the Vikings and U.S. Bank Stadium were also able to correct the train-snafu issue that arose at some of the early events at the new venue, which has a light-rail station right outside the stadium doors. While some of the first events had big lines of riders and not enough trains, Caflisch said that during the season extra trains were held in reserve at the transit station that is close to Target Field (a few stops down the line from U.S. Bank) and then filtered in as Vikings games neared their end.

“We were able to clear the [train] platform in 40 minutes after the last game,” Caflisch said. “The fans really loved the trains.” (More U.S. Bank Stadium images below)

Screen Shot 2017-01-12 at 11.39.38 AM

Vikings fans gather outside the stadium for pregame activites.

Screen Shot 2017-01-12 at 11.40.04 AM

Great nighttime view with city skyline visible through windows.

vik5

A look at the handrail Wi-Fi antenna mounts (this photo credit: Paul Kapustka, MSR)

NRG Stadium Wi-Fi ready for Super Bowl LI

NRG Stadium. Credit: Houston Texans Instagram

NRG Stadium. Credit: Houston Texans Instagram

Nonexistent a year ago, the new Wi-Fi network at Houston’s NRG Stadium has at least one more live game that administrators can use as a final tune-up before the venue and its wireless infrastructure host Super Bowl LI on Feb. 5.

Live since the start of the current NFL season, the Wi-Fi network deployed by integrator 5 Bars using Extreme Networks Wi-Fi gear has seen growth in fan usage for each subsequent game, according to David Moore, manager of information services for NRG Park.

“The first few games [of the season] it wasn’t heavily promoted, but as we went on usage shot up,” said Moore in a recent phone interview. Though he wouldn’t release specific figures on data use, Moore said that game-day totals near the end of the season saw in the range of 25,000 unique users per game, with data totals in the “4-5 terabyte” range. The stadium’s main tenants, the Houston Texans, will have at least one more home game this weekend when they host the Oakland Raiders in the first round of the NFL playoffs.

Under seat APs visible down seating row. Credit: 5 Bars

Under seat APs visible down seating row. Credit: 5 Bars

While it’s possible that the Texans could be hosting the AFC Championship game if all the higher-seed teams lose en route, this weekend’s game is most likely the last chance the NRG Stadium tech team will have as a dress rehearsal for the Super Bowl. Since the Super Bowl is historically the biggest single-day data-usage event — and has gotten bigger every year — all technical eyes will be on the NRG Stadium network, which only started becoming a reality after the stadium hosted last year’s men’s NCAA basketball Final Four. With a base seating capacity of 72,220, NRG’s Super Bowl crowd should roughly be the same as last year’s at Levi’s Stadium in Santa Clara, Calif.

Fast deployment schedule met

“It was a challenge [to deploy so quickly] and we are still working out the tuning,” said Michelle McKenna-Doyle, the NFL’s chief information officer, in an email interview. “We were fortunate to have the regular season to work on it, but hope to have it Super Bowl ready within next couple weeks. 5 Bars, Extreme Networks, The Texans, Harris County and the host committee all worked hard to make it a reality for our fans.”

At one point in recent history, it seemed like NRG might never get a fan-facing Wi-Fi network since the county-owned facility couldn’t find the budget necessary to bring wireless connectivity to the venue. But with the Super Bowl approaching, a consensus finally pushed through, with 5 Bars winning a deployment bid that still needed a change in equipment from Ruckus (the original supplier in the 5 Bars bid) to Extreme. While no official breakdown of funding shares has been supplied, the reported $6 million-plus cost of the Wi-Fi deployment was likely shared in some fashion by the Texans, the NFL, Verizon Wireless and Extreme.

Ready for the playoffs and Super Bowl! Credit: 5 Bars

Ready for the playoffs and Super Bowl! Credit: 5 Bars

According to Moore, Verizon chipped in to secure guaranteed offload for its cellular customers, as well as its own SSID that will automatically sign on Verizon customers in the facility, a tactic used by Verizon in several other NFL stadiums. Non-Verizon customer fans can use a free xfinitywifi SSID, as Comcast is the backbone supplier for the Wi-Fi network services.

According to Moore there are 1,250 Extreme Wi-Fi APs in the venue now, with 550 of those deployed under seats in the main bowl. While Moore said the under-seat location allows for much denser deployment and better network tuning, the method also caused the most headaches during deployment, beyond the usual cost and struggle of drilling through concrete floors.

First of all, the installers had to bring all the necessary power and cabling infrastructure in, since there was nothing underneath the concrete seat floors, Moore said. In fact, because there are offices underneath some of the seating areas, contractors had to negotiate a “drip pan” that kept moisture from seating power-washing away from the office roofing.

Another “big hiccup” emerged when the original equipment used for the under-seat locations didn’t lock out all the moisture, leading to a full replacement of all the 550 under-seat APs. Moore said the under-seat locations now use Extreme’s highest-grade outdoor-rated AP, the 3965i.

New DAS installed last year

Since Super Bowls also typically set records for cellular DAS usage, it’s no surprise that Verizon also recently updated the DAS at NRG Stadium, reportedly putting $12 million into a new system installed before the 2015 season. According to Moore, the DAS and the Wi-Fi also cover parking areas outside the venue, including a plaza where the Texans typically see 6,000 or more fans gathering before games.

“There’s great coverage” in the parking lots with the new DAS, Moore said. Announcements have not yet been made public, but you can expect that both AT&T and Verizon are busy beefing up the cellular systems in and around the stadium, as typical before any big public event.

With the dust finally settled, Texans fans and the soon-to-arrive Super Bowl fans will have good connectivity for the big day, due in no small part to the efforts of 5 Bars, which Moore complimented repeatedly for the company’s persistence and effort.

According to Moore 5 Bars had brought semi-trailers full of equipment to town ahead of the Final Four, and even used the floor of the old Houston Astrodome to roll out network inventory.

“They were working under the gun, with a limited amount of time to get it done,” said Moore of 5 Bars.

Washington upgrades Wi-Fi backbone at FedExField

Screen Shot 2016-11-30 at 11.07.38 AMFans at FedExField now have more bandwidth to use on the stadium Wi-Fi network thanks to a recent upgrade that moved the backbone capacity to 10 gigabits per second, according to a release from the team.

Though it’s never been previously publicly announced by the Washington, D.C. NFL franchise, according to sources close to the team the reported network built by Verizon Wireless using Cisco Wi-Fi gear has been in full operation since last season, albeit under a sort of “soft launch” mode done without any promotion.

With more fans now using the network, the team saw a need to increase bandwidth from the 1 Gbps pipe that had been supplying all the Wi-Fi network needs. For many football-stadium Wi-Fi networks, 10-Gbps pipes are a popular choice, with several stadiums opting for multiple such connections. Levi’s Stadium in Santa Clara, Calif., home of the San Francisco 49ers, has four 10 Gbps pipes as part of its backbone configuration, while US Bank Stadium in Minneapolis, the new home of the Minnesota Vikings, has six 10 Gbps links.

For those who are new to stadium Wi-Fi (or who don’t know the difference between backhaul and actual end-user network speeds), the backbone bandwidth is shared by the entire network, which may have multiple tens of thousands of client devices running off any of hundreds of Wi-Fi access points. Typical user bandwidth speeds on Wi-Fi in large networks can range from single-digit megabits per second anywhere up to 20, 40 or even 60 Mbps, depending upon network configuration and density of users. Any fans attending games at FedExField who want to send us a speedtest please do!

Seahawks see 4.1 TB of Wi-Fi usage for Monday Night game vs. Bills

Seahawks fans at CenturyLink Field during Nov. 7 game. Credit: Seahawks.com

Seahawks fans at CenturyLink Field during Nov. 7 game. Credit: Seahawks.com

With the Seahawks starting to gain momentum for another playoff push, the Wi-Fi network at their home of CenturyLink Field is getting a good workout as well, with 4.10 terabytes of data used during Seattle’s 31-25 win over the Buffalo Bills on Nov. 7.

The Monday Night Football game, an exciting, close contest, also saw a total of 35,808 unique clients and a peak concurrent user number of 23,281 on the Extreme Networks-powered Wi-Fi network at the stadium, according to Seahawks IT officials. The game set an attendance record for CenturyLink Seahawks games, with 69,084 in the seats, breaking the old record of 69,080 from last season.

According to information from all the games so far this season (see below), the CenturyLink Field Wi-Fi has been seeing strong and growing usage, with unique numbers starting out in the high 20,000-user level and edging up each week. Upload totals also seem to be growing, with maybe the team’s improved record as the impetus.

image001

AT&T: College and pro football crowds have used 37 TB of cellular data this year, almost double from 2015

mihi24At about the halfway point for the football season, AT&T said it has seen 37 terabytes of traffic on its stadium wireless networks so far, for both college and professional games combined.

If the total tonnage doesn’t mean that much to you, consider a related data point that should give stadium network managers pause: According to AT&T, so far this year it has seen 85 percent more data used at college games and 90 percent more data used at pro games as compared to last year. If you’re wondering why stadiums are already upgrading networks they installed just a few years ago, the data growth is likely the biggest cause. And there’s still no end in sight to the upward usage curve. Even though these numbers represent only AT&T customer traffic on AT&T stadium DAS installations, the totals and the growth are still pretty staggering.

AT&T also enclosed some weekly highlights for pro and college stadiums, enclosed below.

Screen Shot 2016-11-09 at 10.40.41 AM

Screen Shot 2016-11-09 at 10.40.51 AM

Stadium Tech Report: Carolina Panthers take ownership of DAS, Wi-Fi at Bank of America Stadium

James Hammond, director  of IT for the Panthers, poses next to an under-seat Wi-Fi AP. Credit all photos: Carolina Panthers

James Hammond, director of IT for the Panthers, poses next to an under-seat Wi-Fi AP. Credit all photos: Carolina Panthers

“The fan is the most valuable member of our team,” Jerry Richardson, owner of the Carolina Panthers, is fond of saying.

And it’s become the virtual mission statement for the Charlotte, N.C.-based National Football League franchise. So even though its home field, the Bank of America Stadium, was built relatively recently (1996), technology has come a long way in two decades. And as the Panthers began a four-phase renovation in 2014, they did it with fans’ MVP status in mind, according to James Hammond, director of IT for the Panthers. “It was time for some changes,” he said.

While Carolina was among the first NFL stadiums to install fan-facing Wi-Fi and enhanced cellular networks, the previous DAS and Wi-Fi systems weren’t keeping up with demand and that was starting to adversely impact the Panthers fan experience, Hammond said.

“We chose to perform a rip-and-replace on both DAS and Wi-Fi and take ownership in-house,” Hammond explained. Because the Panthers own and operate BofA Stadium, making those moves was a lot easier than if they were tenants.

Time for an upgrade

Editor’s note: This profile is part of our latest STADIUM TECHNOLOGY REPORT, which includes more stadium profiles as well as looks at Wi-Fi at the Mall of America, and analytics software being used by the Cleveland Browns. DOWNLOAD YOUR FREE COPY of the report today!

The first “fan-centric improvements,” as Hammond called them, came in 2014 in the form of escalators, video big boards and a distributed audio system. As part of the second phase of upgrades, the Panthers then used the 2015 offseason to renovate the club-level suites and tore out the old DAS system while they were at it. And after a careful evaluation of different DAS solutions, they shortlisted two vendors: CommScope and Corning.

CommScope ultimately got the nod; the Panthers then had to decide between the vendor’s ION-B and ION-U DAS systems. “We went with the ION-U, which was quite new and cutting edge at that point, since it had NEMA-rated remotes,” Hammond said. Other systems lacked that kind of weatherproofing and would require additional enclosures – and expense.

CommScope's ION-U powers the new DAS at Bank of America Stadium.

CommScope’s ION-U powers the new DAS at Bank of America Stadium.

“We started over with all new fiber and coax. We did the decommissioning and construction in 90 days, which was pretty quick for a ground-up project,” he said. Beam Wireless Inc. did the design, integration and optimization and is handling the ongoing maintenance of the DAS system; Optical Telecom installed the DAS gear. BofA Stadium now has 256 DAS remotes and more than 600 DAS indoor and outdoor antennas.

AT&T, Verizon and Sprint are the participating DAS carriers; T-Mobile is weighing whether to join the mix during the 2017 off-season.

The Panthers have divided BofA Stadium into 48 DAS zones: 16 zones for the upper bowl, 16 in the lower bowl, and another 16 for concourses, suites, clubs, and offices. Not all zones are used exclusively; carriers choose simulcast patterns that place multiple zones into sectors, and can change them as capacity requirements dictate, Hammond told Mobile Sports Report.

“With some minor design changes to the interior areas, we can accommodate nearly 70 zones,” he explained. “At present the most sectors in use by a carrier is 32. This means the carrier simulcasts across a mix of our 48 zones in order to match them up to 32 carrier sectors.”

Once the new DAS was built and the first couple of events were analyzed, carriers began asking for more frequencies and additional DAS sectors to continue meeting ever-growing demand. In response to the new carrier requests, the first round of DAS upgrades were implemented in the spring of 2016, Hammond said. During the 2015 season, DAS bandwidth was running around 2 GB during games. Hammond said, “With these latest DAS upgrades, we expect the bandwidth numbers to be even higher.”

A DAS remote in a NEMA-rated enclosure.

A DAS remote in a NEMA-rated enclosure.

The impact of the new DAS system was felt immediately upon its debut in July 2015. “It was a much better experience for fans who noticed the improved cellular experience,” Hammond said. Another unexpected benefit: The upgraded DAS helped mitigate bottlenecks with the old Wi-Fi system, which Hammond characterized as “under-designed.”

Going under seat for Wi-Fi upgrade

Unfortunately, there wasn’t time to address any Wi-Fi upgrades before the 2015 football season began, but the Panthers issued an RFP for new Wi-Fi in August 2015 in preparation for Phase 3 renovations that would also include security upgrades and renovations to the upper concourse.

Interested vendors needed to ensure high bandwidth rates as well as high take-rates that allowed three different ISPs (Time Warner Cable/Charter, Level 3 Communications and Windstream) to deliver in excess of 10 GB, though Hammond said they’re starting at a 7-GB threshold.

The Wi-Fi award went to Aruba, now HP Enterprise, in December 2015, and construction began in January 2016 after the last postseason game, when the Panthers beat the Arizona Cardinals to win the NFC championship and a trip to Super Bowl 50.

Similar to Levi’s Stadium and the Dodgers Stadium, the Panthers chose underseat AP enclosures; BofA Stadium sports 770 AP enclosures in the upper and lower bowls out of a total of 1,225 APs, all to ensure maximum coverage and minimal dead spots. The Panthers selected AmpThink to do the Wi-Fi integration and construction; the turnkey contractor also designed and fabricated a custom enclosure for the APs.

Indoor access point inside the stadium.

Indoor access point inside the stadium.

One other innovation in the Panthers’ Wi-Fi installation is that the underseat enclosure is mounted to the riser — the vertical part of the step — but looks like it’s on the tread, the horizontal part, which is intended as a waterproofing measure. “The riser is easier to seal and isn’t affected by pressure washers, which you’re doing constantly with an outdoor stadium,” Hammond said. “And by running pipe through the riser, you don’t have gravity working against you,” which helps keep out water, he explained.

Panthers fans access the stadium Wi-Fi through a portal page after accepting the team’s terms and conditions. From there, they are whitelisted and can automatically join the Wi-Fi network for the rest of the season. Hammond said a fan’s email is requested but not required by the portal page, and there’s a small incentive offered to encourage fans’ email subscriptions.

The new Wi-Fi system got a workout with a soccer game at BofA Stadium at the end of July 2016, then with a Panthers’ Fan Fest the following week. “All the indicators were good, and fan feedback about the system was excellent,” Hammond said. But he cautioned that the two events were not “full bowl” events with smaller attendance numbers (~50,000) than a regular season football game (75,000+). “We will continue to optimize and tune settings as we learn more during events with higher attendance,” Hammond said.

Total budget so far for the technology upgrades totals about $16 million; the DAS build-out was just under $10 million; Wi-Fi was a little more than $6 million, which included additional wired infrastructure, according to the team.

Beacons coming next

And the Panthers aren’t done making technology improvements to their stadium. Phase 4 looks to add Bluetooth beacons and do some refinement of the Panthers app. “My goal during the upcoming season is to look at options for location-aware services,” Hammond said. Some APs have beacons built in; other may need to be added to get the granularity the Panthers want for location awareness.

Hammond also wants to give fans more things to do with the Panthers app and also optimize it for push notifications, even with something as basic and useful as restroom and concessions location information. “As we learn more about fans individually, we can direct them to things of particular interest to them,” he added.

“So far, we are very pleased with the performance of the Wi-Fi and DAS systems,” Hammond said, noting the Panthers will continue to tune frequencies, add zones and increase bandwidth where needed. It’s the sort of attention that smart sporting franchises pay to their most valued team members.

car1