From overhead to under seat: A short history of the hows and whys of stadium Wi-Fi network design

Wi-Fi handrail enclosures at U.S. Bank Stadium, Minneapolis, Minn. Credit: Paul Kapustka, MSR

Wi-Fi handrail enclosures at U.S. Bank Stadium, Minneapolis, Minn. Credit: Paul Kapustka, MSR

By Bill Anderson, AmpThink

The history of high density (HD) Wi-Fi deployments in stadiums and arenas is short. Yet the the amount of change that occurred is significant; both in terms of how these networks are deployed and why.

Venue operators, manufacturers, and integrators are still grappling with the particulars of HD Wi-Fi in large open environments, even though there are a substantial number of deployed high quality implementations. Below, I’ve shared our perspective on the evolution of HD Wi-Fi design in stadiums and arenas and put forth questions that venue operators should be asking to find a solution that fits their needs and their budget.

AmpThink’s background in this field

Over the past 5 years, our team has been involved in the deployment of more than 50 high-density Wi-Fi networks in stadiums throughout North America. In that same period, the best-practices for stadium HD Wi-Fi design have changed several times, resulting in multiple deployment methodologies.

Each major shift in deployment strategy was intended to increase total system capacity [1]. The largest gains have come from better antenna technology or deployment techinques that better isolated access point output resulting in gains in channel re-use.

What follows is a summary of what we’ve learned from the deployments we participated in and their significance for the future. Hopefully, this information will be useful to others as they embark on their journeys to purchase, deploy, or enhance their own HD Wi-Fi networks.

In the beginning: All about overhead

Editor’s note: This post is part of Mobile Sports Report’s new Voices of the Industry feature, in which industry representatives submit articles, commentary or other information to share with the greater stadium technology marketplace. These are NOT paid advertisements, or infomercials. See our explanation of the feature to understand how it works.


Designers of first generation of HD Wi-Fi networks were starting to develop the basic concepts that would come to define HD deployments in large, open environments. Their work was informed by prior deployments in auditoriums and convention centers and focused on using directional antennas. The stated goal of this approach was to reduce co-channel interference [2] by reducing the effective footprint of an individual access point’s [3] RF output.

However the greatest gains came from improving the quality of the link between clients and the access point. Better antennas allowed client devices to communicate at faster speeds which decreased the amount of time required to complete their communication, making room for more clients on each channel before a given channel became saturated or unstable.

Under seat Wi-Fi AP at Bank of America Stadium. Credit: Carolina Panthers

Under seat Wi-Fi AP at Bank of America Stadium. Credit: Carolina Panthers

The concept was simple, but limited by the fact that there were few antennas available that could do the job effectively. Creative technicians created hybrid assemblies that combined multiple antennas into arrays that rotated polarization and tightened the antenna beam to paint the smallest usable coverage pattern possible. In time, this gap was addressed and today there are antennas specifically developed for use in overhead HD deployments – Stadium Antennas.

Typically, Stadium Antennas are installed in the ceilings above seating and/or on the walls behind seating because those locations are relatively easy to cable and minimize cost. We categorize these deployments as Overhead Deployments.

From overhead to ‘front and back’

First generation overhead deployments generally suffer from a lack of overhead mounting locations to produce sufficient coverage across the entire venue. In football stadiums, the front rows of the lower bowl are typically not covered by an overhang that can be used for antenna placement.

These rows are often more than 100 feet from the nearest overhead mounting location. The result is that pure overhead deployments leave some of the most expensive seats in the venue with little or no coverage. Further, due to the length of these sections, antennas at the back of the section potentially service thousands of client devices [4].

As fans joined these networks, deployments quickly became over-loaded and generated service complaints for venue owners. The solution was simple — add antennas at the front of long sections to reduce the total client load on the access points at the back. It was an effective band-aid that prioritized serving the venues’ most important and often most demanding guests.

This approach increased the complexity of installation as it was often difficult to cable access points located at the front of a section.

And for the first time, antennas were placed where they were subject to damage by fans, direct exposure to weather, and pressure washing [5]. With increased complexity, came increased costs as measured by the average cost per installed access point across a venue.

Because these systems feature antennas at the front and rear of each seating section, we refer to these deployments as ‘Front-to-Back Deployments.’ While this approach solves specific problems, it is not a complete solution in larger venues.

‘Filling In’ the gaps

Data collected from Front-to-Back Deployments proved to designers that moving the antennas closer to end users:
— covered areas that were previously uncovered;
— increased average data rates throughout the bowl;
— used the available spectrum more effectively; and
— increased total system capacity.

The logical conclusion was that additional antennas installed between the front and rear antennas would further increase system capacity. In long sections these additional antennas would also provide coverage to fans that were seated too far forward of antennas at the rear of the section and too far back from antennas at the front of the section. The result was uniform coverage throughout the venue.

In response, system designers experimented with hand rail mounted access points. Using directional antennas, coverage could be directed across a section and in opposition to the forward-facing antennas at the rear of the section and rear-facing antennas at the front of a section. These placements filled in the gaps in a Front-to-Back Deployment, hence the name ‘In-Fill Deployment.’

While these new In-Fill Deployments did their job, they added expense to what was already an expensive endeavor. Mounting access points on handrails required that a hole be drilled in the stadia at each access point location to cable the installed equipment. With the access point and antenna now firmly embedded in the seating, devices were also exposed to more traffic and abuse. Creative integrators came to the table with hardened systems to protect the equipment – handrail enclosures. New costs included: using ground-penetrating radar to prepare for coring; enclosure fabrication costs; and more complex conduit and pathway considerations. A typical handrail placement could cost four times the cost of a typical overhead placement and a designer might call for 2 or 3 handrail placements for every overhead placement.

Getting closer, better, faster: Proximate Networks

In-Fill strategies substantially solved the coverage problem in large venues. Using a combination of back of section, front of section, and hand-rail mounted access points, wireless designers had a tool box to deliver full coverage.

But with that success came a new problem. As fans discovered these high density networks and found new uses for them, demands on those networks grew rapidly, especially where teams or venue owners pushed mobile-device content strategies that added to the network load. In-spite of well placed access points, fan devices did not attach to the in-fill devices at the same rate that they attached to the overhead placements [6]. In-fill equipment remained lightly used and overhead placements absorbed hundreds of clients. Gains in system capacity stalled.

Close-up look at U.S. Bank Stadium railing enclosure during final construction phase, summer 2016. Credit: Paul Kapustka, MSR

Close-up look at U.S. Bank Stadium railing enclosure during final construction phase, summer 2016. Credit: Paul Kapustka, MSR

To overcome uneven system loading, designers needed to create a more even distribution of RF energy within the deployment. That required a consistent approach to deployment, rather than a mix of deployment approaches. The result was the elimination of overhead antennas in favor of access points and antennas installed within the crowd, closest to the end use; hence the name ‘Proximate Networks.’

Proximate networks come in two variations: handrail only and under seat only. In the hand rail only model, the designer eliminates overhead and front of section placements in favor of a dense deployment of hand rail enclosures. In the under seat model, the designer places the access point and antenna underneath the actual seating (but above the steel or concrete decking). In both models, the crowd becomes an important part of the design. The crowd attenuates the signal as it passes through their bodies resulting in consistent signal degradation and even distribution of RF energy throughout the seating bowl. The result is even access point loading and increased system capacity.

An additional benefit of embedding the access points in the crowd is that the crowd effectively constrains the output of the access point much as a wall constrains the output of an access point in a typical building. Each radio therefore hears fewer of its neighbors, allowing each channel to be re-used more effectively. And because the crowd provides an effective mechanism for controlling the spread of RF energy, the radios can be operated at higher power levels which improves the link between the access point and the fan’s device. The result is more uniform system loading, higher average data rates, increased channel re-ue, and increases in total system capacity.

While Proximate Networks are still a relatively new concept, the early data (and a rapid number of fast followers) confirms that if you want the densest possible network with the largest possible capacity, then a Proximate Network is what you need.

The Financials: picking what’s right for you

From the foregoing essay, you might conclude that the author’s recommendation is to deploy a Proximate Network. However, that is not necessarily the case. If you want the densest possible network with the largest possible capacity, then a Proximate Network is a good choice. But there are merits to each approach described and a cost benefit analysis should be performed before a deployment approach is selected.

For many venues, Overhead Deployments remain the most cost effective way to provide coverage. For many smaller venues and in venues where system utilization is expected to be low, an Overhead deployment can be ideal.

Front-to-Back deployments work well in venues where system utilization is low and the available overhead mounting assets can’t cover all areas. The goal of these deployments is ensuring usable coverage, not maximizing total system capacity.

In-fill deployments are a good compromise between a coverage-centric high density approach and a capacity-centric approach. This approach is best suited to venues that need more total system capacity, but have budget constraints the prevent selecting a Proximate approach.

Proximate deployments provide the maximum possible wireless density for venues where connectivity is considered to be a critical part of the venue experience.

Conclusion

If your venue is contemplating deploying a high density network, ask your integrator to walk you through the expected system demand, the calculation of system capacity for each approach, and finally the cost of each approach. Make sure you understand their assumptions. Then, select the deployment model that meets your business requirements — there is no “one size fits all” when it comes to stadium Wi-Fi.

Bill Anderson, AmpThink

Bill Anderson, AmpThink

Bill Anderson has been involved in the design and construction of wireless networks for over 20 years, pre-dating Wi-Fi. His first experience with wireless networking was as a software developer building software for mobile computers communicating over 400 MHz and 900 MHz base stations developed by Aironet (now part of Cisco Systems).

His work with mobile computing and wireless networks in distribution and manufacturing afforded him a front row seat to the emergence of Wi-Fi and the transformation of Wi-Fi from a niche technology to a business critical system. Since 2011 at AmpThink Bill has been actively involved in constructing some of the largest single venue wireless networks in the world.

Footnotes

^ 1. A proxy for the calculation of overall system capacity is developed by multiplying the average speed of communication of all clients on a channel (avg data rate or speed) by the number of channels deployed in the system (available spectrum) by the number of times we can use each channel (channel re-use) or [speed x spectrum x re-use]. While there are many other parameters that come into play when designing a high density network (noise, attenuation, reflection, etc.), this simple equation helps us understand how we approach building networks that can support a large number of connected devices in an open environment, e.g. the bowl of a stadium or arena.

^ 2. Co-channel interference refers to a scenario where multiple access points are attepting to communicate with client devices using the same channel. If a client or access point hears competing communication on the channel they are attempting to use, they must wait until that communication is complete before they can send their message.

^ 3. Access Point is the term used in the Wi-Fi industry to describe the network endpoint that client devices communicate with over the air. Other terms used include radio, AP, or WAP. In most cases, each access point is equipped with 2 or more physical radios that communicate on one of two bands – 2.4 GHz or 5 GHz. HD Wi-Fi deployments are composed of several hundred to over 1,000 access points connected to a robust wired network that funnels guest traffic to and from the internet.

^ 4. While there is no hard and fast rule, most industry experts agree that a single access point can service between 50 and 100 client devices.

^ 5. Venues often use pressure washers to clean a stadium after a big event.

^ 6. Unlike cellular systems which can dictate which mobile device attaches to each network node, at what speed, and when they can communicate, Wi-Fi relies on the mobile device to make the same decisions. When presented with a handrail access point and an overhead access point, mobile devices often hear the overhead placement better and therefore prefer the overhead placement. In In-Fill deployments, this often results in a disproportionate number of client devices selecting overhead placements. The problem can be managed by lowering the power level on the overhead access point at the expense of degrading the experience of the devices that the designer intended to attach to the overhead access point.

Analysis: The year of the big stadium Wi-Fi upgrade

Carolina Panthers director of IT James Hammond shows off a new under-seat Wi-Fi AP at Bank of America Stadium. Credit: Carolina Panthers

Carolina Panthers director of IT James Hammond shows off a new under-seat Wi-Fi AP at Bank of America Stadium. Credit: Carolina Panthers

Even in the midst of several brand-new stadium debuts and the future-proofed wireless networks inside them, there is a separate, yet distinct trend emerging in the big-stadium, wireless connectivity world: Call it the year of the big upgrade.

Our profile in our latest STADIUM TECH REPORT of Bank of America Stadium in Charlotte, N.C., is a case in point: Thanks to the never-ending demand for more connectivity for fans, stadiums that deployed networks just a few years ago are now finding that those old systems already need upgrades or replacements, typically at a much higher cost than the original network. In addition to BofA Stadium, the New England Patriots’ home, Gillette Stadium, also got a Wi-Fi makeover this past summer, going from about 400 Wi-Fi APs to well over a thousand, with most of the new ones deployed under seats.

According to Fred Kirsch, who oversees the Gillette Stadium network, some of the under-seat placements there were especially tricky, since granite underneath the stands didn’t allow for the ability to drill through the concrete. A workaround involving an above-ground enclosure was envisioned and manufactured, underlining the custom complexity of network deployment found from stadium to stadium. No two are the same, and what works at one may or may not work at another.

But what is common across all these large venues is the ever-increasing need for bandwidth, a moving target that has yet to slow down or stabilize. Last year the story that turned everyone’s head was the need by carriers to upgrade their DAS infrastructure at Levi’s Stadium ahead of Super Bowl 50 – this coming just a year after the stadium had opened for business. While the demands of a Super Bowl (especially Super Bowl 50, which set records for DAS and Wi-Fi usage) are perhaps much different than everyday events, it’s still a safe bet that for many stadiums with Wi-Fi networks – especially the early movers – 2016 has become a year of reckoning, or biting the bullet and writing more checks for more coverage, perhaps seemingly too soon after the initial rollout.

Getting ready for Super Bowl LI

In Houston, NRG Stadium finally has Wi-Fi, and not a moment too soon, with Super Bowl LI on the near horizon. Since the venue didn’t have Wi-Fi prior to this season it’s not really an upgrade but it’s hard to understate the challenge of putting in a Super Bowl-ready network in just one summer, a construction calendar shortened by the fact that integrator 5 Bars and equipment vendor Extreme Networks had to wait until after the NCAA Men’s Final Four was over to begin installing cabling and APs. At of the start of the NFL season the Wi-Fi network is already live at NRG Stadium, and is sure to go through weekly tweaks as the league marches on toward its championship game.

Gillette Stadium before the Sept. 11 game vs. the Miami Dolphins. Credit: Steve Milne, AP, via Patriots.com

Gillette Stadium before the Sept. 11 game vs. the Miami Dolphins. Credit: Steve Milne, AP, via Patriots.com

And while attention-grabbing new stadiums like US Bank Stadium in Minneapolis and Mercedes-Benz Stadium in Atlanta are planning big network capacity from the get-go, some new stadiums like T-Mobile Arena in Las Vegas have upgrade thinking planned in from the start, with the idea that the network will never really be a finished product, at least until they stop making new phones or developing new apps. Of course, that future isn’t happening anytime soon, with the Apple iPhone 7 announcement with the new double-lens camera coming in just before the start of another football season.

New phones and new apps mean more bandwidth demands, leading even those who already have stadium networks to keep wondering if what they’ve installed is enough. We suspect this may be an ongoing story line for the foreseeable future, so – stay tuned here to Mobile Sports Report for the latest success stories and lessons learned from those who have already jumped in or jumped back in to the deployment fray.

Editor’s note: This column is from our latest STADIUM TECH REPORT, which is available for free download from our site. Read about Wi-Fi deployments at Bank of America Stadium, Mercedes-Benz Stadium and more!

Ookla shares Speedtest data from CenturyLink Field, other stadiums

Ookla ad banner being flown over CenturyLink Field in Seattle. Credit: Ookla

Ookla ad banner being flown over CenturyLink Field in Seattle. Credit: Ookla

Anyone who follows Mobile Sports Report knows that I use the Speedtest app from Ookla to measure stadium network performance whenever I visit a sporting venue. While my one-man tests do show some measure of network power, I always dreamed of harnessing the results from many fans at the same game to see a better picture of the network performance.

Well, Speedtest’s creators think along the same lines, and conducted an experiment during an Aug. 25 Seattle Seahawks preseason game at CenturyLink Field in Seattle. You can read their very thorough post and neat results here, with some interesting twists — for instance, the cellular networks are way faster than the CenturyLink Wi-Fi, according to the Ookla results.

UPDATE: Ookla responded to our email and let us know that on Aug. 25, there were 252 Speedtests at CenturyLink Field, a great sampling to draw results from. Ookla also talked about tests from 12 different events at CenturyLink Field, and said in the email that across those events it saw 1,143 tests conducted.

Ookla also published some test result totals from other stadiums as well, including Levi’s Stadium, AT&T Stadium and Bank of America Stadium, but didn’t say when those tests were recorded, or how many tests were taken.

What we really like, however, is that Ookla’s tests show what our stadium tech report surveys have been showing — that overall, in-stadium network performance is steadily improving. Over time, more data like this can help dispel the still-lingering rumor that stadium networks don’t deliver good connectivity. Now if we could only get Ookla to partner with us to do league-wide or college-comparison speedtests… anyone ready for that idea?

Verizon: Denver fans used 2.87 TB of wireless data during AFC championship; AT&T also sets DAS traffic records at both Sunday games

Sports Authority Field at Mile High, during Jan. 3 game vs. San Diego. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Sports Authority Field at Mile High, during Jan. 3 game vs. San Diego. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

During their team’s exciting 20-18 victory over the New England Patriots Sunday, Denver Broncos fans who are Verizon Wireless customers used 2.87 terabytes of wireless data, according to Verizon. That total includes 1.7 TB used on the Verizon-only Wi-Fi network at Sports Authority Field at Mile High, and another 1.17 TB of data on the Verizon LTE DAS network at the stadium.

Also on Sunday, at the Bank of America Stadium in Charlotte, N.C., where the Carolina Panthers defeated the Arizona Cardinals, Verizon Wireless customers there used 1.3 TB of data on the Verizon LTE DAS network, according to figures sent to us by Verizon.

According to AT&T, its wireless customers set DAS traffic records at both stadiums Sunday, with 819 GB used on AT&T’s DAS network at Sports Authority Field and 739 GB used by AT&T DAS customers at Bank of America Stadium. Both totals are the highest-ever marks seen by AT&T at the respective stadiums, according to AT&T; the Denver total Sunday was 34 percent higher than the total used in the team’s first playoff game this season against Pittsburgh, and was 52 percent more than the average data used during regular-season games. In Charlotte, the DAS traffic total Sunday was 16 percent higher than the number hit during the playoff game a week previous against Seattle, and 50 percent higher than the average regular-season game, according to AT&T.

On the AT&T Wi-Fi network at Bank of America Stadium, fans used 740 GB of data, which AT&T said is also the highest-ever mark for that network, 19 percent higher than the previous playoff game vs. Seattle and 36 percent more than the network saw for average regular season games. The Wi-Fi network at Bank of America Stadium will be replaced this offseason, with a new network built by AmpThink and Aruba.

In Denver, the network situation is somewhat unique since Verizon built the Wi-Fi network inside Sports Authority Field at Mile High, but so far only Verizon customers are allowed access to it. While we’ll describe the situation in more detail in a stadium-visit profile coming very soon, the word from stadium IT types is that the Wi-Fi is open to other carriers but none have yet signed on to allow their customers access to it. There are, however, separate DAS networks for AT&T, Sprint and T-Mobile inside the stadium as well, so at least from a DAS perspective most fans at Mile High are pretty well connected.

Stay tuned for more soon on the networks at Sports Authority Field at Mile High! In the meantime, some pictures from our Jan. 3 visit below.

mihi3

Count the antennas! See if you can spot the AT&T DAS antennas (slightly rounded) and the T-Mobile antenna (big square) among others in this overhang shot at Sports Authority Field at Mile High.

mihi4

Denver’s south end zone scoreboard is topped by a horse… and two Wi-Fi antennas

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

The parking lots just outside Sports Authority Field have good Wi-Fi coverage as this light pole shows.

AmpThink, Aruba win Wi-Fi deal for Carolina Panthers’ Bank of America Stadium

Screen Shot 2015-12-10 at 9.29.18 PMDetails are still yet to emerge, but AmpThink and Aruba have officially won the deal to put a new 1,200-AP Wi-Fi network in the Carolina Panthers’ Bank of America Stadium for next football season.

Of course, the undefeated Panthers still have a lot left going on this season, and fans at home games will still be able to use the AT&T Wi-Fi network that has been in the stadium for several years. But as we noted in our NFL Stadium Tech roundup this season, there had been some Panther-fan grumbling about the performance of the network and stadium IT execs had been busy vetting a new provider during this past year.

Earlier today James Hammond, director of information technology for the Carolina Panthers, sent us this brief note that confirmed the selection of AmpThink and Aruba. It also confirms another primarily under-seat AP design for AmpThink, following its help in deploying the new network at Kyle Field at Texas A&M (see report out very very very soon!); it’s also another big NFL deal for Aruba (a Hewlett Packard Enterprise company), whose gear is also used at the San Francisco 49ers’ Levi’s Stadium. According to Hammond:

The Carolina Panthers have issued a letter of intent to award the high density Wi-Fi project to AmpThink. The deployment will utilize approximately 1,200 Aruba access points in an under-seat configuration.

More on this deal as details become available.

NFL Stadium Tech Reviews — NFC South

Editor’s note: The following team-by-team capsule reports of NFL stadium technology deployments are an excerpt from our most recent Stadium Tech Report, THE PRO FOOTBALL ISSUE. To get all the capsules in one place as well as our featured reports, interviews and analysis, download your free copy of the full report today.

NFC SOUTH

Reporting by Paul Kapustka

Screen Shot 2015-12-07 at 4.57.13 PMNew Orleans Saints
Mercedes-Benz Superdome
Seating Capacity: 76,468
Wi-Fi – Yes
DAS – Yes

Ten years after Hurricane Katrina devastated New Orleans, the Superdome remains a signal of the region’s recovery. Inside the dome, the Wi-Fi installed before the most recent Super Bowl there is still going strong.

Atlanta Falcons
Georgia Dome
Seating Capacity: 71,280
Wi-Fi – Yes
DAS – Yes

Even as they wait for the technological wonder that is the coming next year at Mercedes-Benz Stadium (with fiber-based Wi-Fi from IBM and Corning), Falcons fans still have way better than average connectivity inside the Georgia Dome, where a Cisco-powered network provides access to mobile treats like instant replays from multiple camera angles.

Carolina Panthers
Bank of America Stadium
Seating Capacity: 74,455
Wi-Fi – Yes
DAS – Yes

Bank of America Stadium completely replaced its DAS solution during the offseason with a new Commscope ION-U system. The new DAS is owned by the Panthers directly and includes Verizon, AT&T, and Sprint as participants. Look for the stadium to completely replace its Wi-Fi system this coming offseason, with a new 1,200-AP network for the 2016 season.

Tampa Bay Buccaneers
Raymond James Stadium
Seating Capacity: 65,890
Wi-Fi – Yes
DAS – Yes

Raymond James Stadium got free fan-facing Wi-Fi for the 2012 season, but since then we haven’t heard anything about upgrades to the system. With the college playoff championship game coming in 2017, however, you can probably expect to see connectivity improvements coming soon.