Entry issues snarl start of WrestleMania 32 at AT&T Stadium; crowd uses 6.77 TB of Wi-Fi

While Sunday’s WrestleMania 32 eventually ended up with a record attendance for the WWE’s signature event with 101,763 fans in attendance, some snafus with the entry process had fans reporting as long as three-hour waits to get in, with some pegging a Wi-Fi outage that disabled ticket scanners as one of the roots of the problem.

We don’t as of yet have any on-the-record comments from AT&T Stadium so we can’t say exactly what the problem was. UPDATE, 4/4/16, 2:50 p.m.: According to John Winborn, chief information officer for the Dallas Cowboys Football Club (and the person who runs the network inside AT&T Stadium), despite the rumors that emerged there was no Wi-Fi outage at the stadium — in fact Winborn said the crowd used 6.77 TB of Wi-Fi data Sunday, a single-day stadium Wi-Fi total second only to the 10.1 TB used at Levi’s Stadium for Super Bowl 50. (More details on the Wi-Fi consumption coming soon.)

Wi-Fi rumors aside, on Sunday night Twitter and Instagram were full of photos from irate fans who were stuck outside the venue, with some reportedly unable to get inside or to their seats before the actual matches started. According to a blog post at the Fort Worth Star-Telegram, the WWE and the Stadium issued a joint press release halfway through the event that read:

“To ensure the safety of WWE fans, increased security measures were put in place tonight. We apologize that it may have taken some fans longer than usual to get into AT&T Stadium.”

Several other news outlets reported the problem, with all seeming to peg the issue on a Wi-Fi problem with the ticket scanners. Winborn, however, said there were no Wi-Fi outages in the stadium or with the ticket scanners.

According to the Fort Worth Star-Telegram report, the vitriol on Twitter and other social media outlets may not have been entirely correct; according to the report:

Do not believe the viral reports and unsightly online pictures that fans could not get to their seats by the time the early matches began — they could but they preferred to walk around, buy food, drink and merchandise. But the images and the complaints that flooded the Internet about #WrestleMania32 were so plentiful, and negative, joint press release was issued halfway through the event.

With 101,763 finally in their seats the event was the fifth-largest crowd inside AT&T Stadium, and as such most likely produced a large Wi-Fi and DAS traffic number once things finally got underway. We will update this post and have another when we get the wireless traffic stats from the stadium.

Betting the Under (Part 2): Putting Wi-Fi antennas under seats is the hot new trend in stadium wireless networks

Under-seat Wi-Fi AP at Levi's Stadium. Photo: Paul Kapustka, MSR (click on any photo for a larger image)

Under-seat Wi-Fi AP at Levi’s Stadium. Photo: Paul Kapustka, MSR (click on any photo for a larger image)

Part 2 of this story picks up with the decision to put Wi-Fi APs under seats at Levi’s Stadium. If you missed it, here is the link to Part 1.

According to Chuck Lukaszewski, now vice president of wireless strategy and standards at Hewlett Packard Enterprise (formerly very high density architect in the CTO Office of Aruba Networks), Aruba had been testing under-seat AP designs since around 2010, “in one form or another.” There were some initial tests of under-seat AP deployments at Turner Field in Atlanta and at American Airlines Arena in Dallas, but nothing on the scale of AT&T Park’s 2013 deployment, or on the scale Aruba planned to have at Levi’s Stadium when it opened in 2014.

Some of the first under-seat Wi-Fi deployments in other arenas were actually deployed completely under the stands, Lukaszewski said, with signals shooting up through the concrete. Though he said “you could get reasonably good throughput through concrete,” especially for 2.4 GHz frequencies, installing antennas above the concrete was “considerably better,” Lukaszewski said.

Curiously, one of the biggest problems in stadium Wi-Fi deployment — especially for those heavy on overhead antenna use — is negotiating interference between antennas; sometimes, clients can “see” antennas and APs that are across the stadium, and will try to connect to those instead of the AP closest to them, a problem that leads to inefficient bandwidth use. Interference also means you can’t place APs too closely together, making it somewhat of an art to find ways to increase coverage without increasing interference.

Dan Williams, former VP of technology for the San Francisco 49ers, talking networking at Levi's Stadium. Photo: Paul Kapustka, MSR

Dan Williams, former VP of technology for the San Francisco 49ers, talking networking at Levi’s Stadium. Photo: Paul Kapustka, MSR

What Aruba found in its testing, Lukaszewski said, was that under-seat Wi-Fi AP deployments could be far more dense than overhead-centric designs, mainly because the human bodies in the seats would provide beneficial “blocking” of signals, allowing network designers to place APs more closely together, and to be able to re-use the same Wi-Fi channels in more antennas.

“If you can use human bodies to contain signals, you can have much smaller cells,” Lukaszewski said. Under-seat deployments, he said, “allows us to re-use the same channel less than 100 yards away.”

With more channels available for each AP, the difference in the metric Lukaszewski calls “megabytes per fan” can be “profound” for an under-seat design versus an overhead design, he said.

“We do see trends [in stadium network data] of under-seat being able to deliver well over 100 MB per fan per event, while overhead designs [deliver] significantly under 100 MB per fan per event,” said Lukaszewski.

Dan Williams, the former vice president of technology for the San Francisco 49ers, said he and Lukaszewski were in agreement that under-seat was the best method to deploy at Levi’s Stadium.

Kyle Field at Texas A&M. White spots in stands are under-seat AP locations. Photo: Paul Kapustka, MSR

Kyle Field at Texas A&M. White spots in stands are under-seat AP locations. Photo: Paul Kapustka, MSR

“I just did not believe in overhead,” said Williams, who said he brainstormed with Aruba’s Lukaszewski on the under-seat idea, which they both brought to the Wi-Fi design at Levi’s. By using under-seat APs, Williams said, the Levi’s Stadium design looked to provide “cones [of bandwidth] around the audience, immersing [fans] in a signal.”

After beating the previous year’s Super Bowl Wi-Fi total at its NFL regular-season opener in 2014, Levi’s Stadium’s Wi-Fi network more than passed its biggest test ever this year, carrying a record 10.1 terabytes of Wi-Fi data during Super Bowl 50. Those numbers are proof of Lukaszewski’s claim: “By far, under seat is better.”

New deployments trending to under-seat

Editor’s note: This excerpt is from our latest STADIUM TECH REPORT, our long-form PDF publication that combines in-depth stadium tech reports with news and analysis of the hottest topics in the world of stadium and large public venue tech deployments. Enjoy this PART 1 of our lead feature, or DOWNLOAD THE REPORT and read the whole story right now!

Even though under-seat deployments can be considerably more expensive, especially in a retrofit situation where deployment requires coring through concrete, many stadiums are now seeming to agree with another Lukaszewski claim, that “the return absolutely justifies the investment.”

At AT&T Stadium in Arlington, Texas, the Cowboys quicked followed their sister park’s lead and installed under-seat APs in force ahead of that venue’s hosting of the inaugural College Football Playoff championship game in January of 2015. John Winborn, chief information officer for the Dallas Cowboys Football Club, said the team worked with AT&T’s “Foundry” innovation centers to produce a smaller, sleeker under-seat AP enclosure that fit well with the stadium’s commitment to aesthetics.

Back on the baseball side, the Giants now have 1,628 Wi-Fi APs in their park, with the vast majority of them under-seat, in all three decks of seating. And the Giants’ main rival to the south, the Los Angeles Dodgers, also used under-seat APs in a recent Wi-Fi upgrade.

Close-up of conduit running to under-seat AP at Kyle Field. Photo: Paul Kapustka, MSR

Close-up of conduit running to under-seat AP at Kyle Field. Photo: Paul Kapustka, MSR

And if Levi’s Stadium led the way for under-seat Wi-Fi, the new mainly under-seat network at the refurbished Kyle Field at Texas A&M might be the QED on the debate, with ultra-fast network speeds and big data-consumption numbers (including 5.7 TB of Wi-Fi at a game versus Alabama) adding measureable momentum to the under-seat trend. Bill Anderson, CEO of Wi-Fi deployment strategy firm AmpThink, said he was an early disbeliever in under-seat Wi-Fi — until he saw the numbers.

“At first we mocked it, made fun of it,” said Anderson, whose firm has been called in to produce Wi-Fi network designs for several recent Super Bowls, as well as for the Kyle Field design. But when Aruba showed AmpThink the data from under-seat tests and deployments, “that was the ‘a-ha’ moment for us,” Anderson said.

Working with Aruba at Kyle Field, AmpThink was able to collect its own data, which convinced Anderson that under-seat was the way to go if you wanted dense, high-performing networks.

“The really important thing is to get APs closer to the people,” said Anderson. “That’s the future.”

Anderson said some doubters may remain, especially those who try to mix a small amount of under-seat APs with existing overhead deployments, a recipe for lowered success due to the potential interference issues. At Texas A&M, Anderson said AmpThink was able to build a design with far less interference and much greater density than an overhead solution, producing numbers that people have to pay attention to.

“We only know what we’ve observed, but we’re evangelistic supporters” of under-seat designs, Anderson said. “If someone says to you under-seat is hocus-pocus, they’re not looking at the data.”

Not for everyone, but more are trying under-seat

Though proponents of under-seat Wi-Fi all agree on its ability to deliver denser, faster networks, they all also agree that under-seat can be considerably more costly than overhead Wi-Fi, especially in a retrofit situation.

In addition to having to core through concrete seating areas to get conduit to the under-seat APs, the devices themselves need to be sealed, to guard them from weather, drink spills, and the power-washing equipment employed by most stadiums to clean seating areas.

Aruba’s Lukaszewski also noted that under-seat deployments generally use more linear feet of cabling to connect the APs than overhead, which also drives up the cost. Then since under-seat designs tend to use more APs, that also means a higher budget to cover a higher number of devices.

A row shot of the under-seat APs at AT&T Stadium. Photo: Dallas Cowboys

A row shot of the under-seat APs at AT&T Stadium. Photo: Dallas Cowboys

For some stadiums, the construction materials used prohibit the under-seat option from even being tried. At the Green Bay Packers’ legendary Lambeau Field, a late-1950s construction design that used lots of concrete and rebar — as well as part of the stadium’s bottom sitting directly in the ground — meant that under-seat Wi-Fi wasn’t an option, according to Wayne Wichlacz, director of information technology for the Packers.

Other stadiums, like the University of Nebraska’s Memorial Stadium, don’t have enough space between the stadium’s bleacher seats and the floor for under-seat APs to be safely installed. And many schools or teams simply don’t have big IT budgets like the $20-million-plus available to Texas A&M that allowed the Kyle Field design to seek the best result possible.

But many of the new stadiums under construction, as well as existing venues that are planning for new best-of-breed networks, have already committed to under-seat Wi-Fi designs, including the Sacramento Kings’ Golden 1 Center, where Ruckus Wireless will implement its first under-seat stadium Wi-Fi network.

Steve Martin, senior vice president and general manager at Ruckus, said the Golden 1 Center design, planned to be the most dense anywhere, will “primarily be underseat,” a choice he said “helps in a lot of ways.”

Foremost is the performance, something Martin said Ruckus has been testing at the Kings’ current home, the Sleep Train Arena. “It [under seat] does give you the isolation for frequency re-use,” he said.

The under-seat design also makes sense in Golden 1 Center since the stadium’s overall design is very open, with lots of glass walls and unobstructed views.

And under-seat deployment is even making inroads into the distributed antenna system (DAS) world, with Verizon Wireless implementing more than 50 under-seat DAS antennas at Levi’s Stadium prior to Super Bowl 50. Mainly installed to cover the bottom-of-the-bowl rows, the under-seat APs helped Verizon manage a record day for DAS traffic, with 7 TB reported on its in-stadium cellular network during the game.

“To get a quality signal, we had to go under seat,” said Brian Mecum, vice president, network, for Verizon Wireless, who said that in that area of the stadium, under seat was the only way to get a quality signal close to the subscriber’s phone. Verizon, he said, helped design the under-seat DAS antenna, and is looking to deploy it in other stadiums soon.

“It’s the first of more,” he said.

END PART 2… HERE IS THE LINK TO PART 1… TO READ THE WHOLE STORY NOW, DOWNLOAD OUR REPORT!

AT&T: NFL fans used 55% more DAS data this year

dx1AT&T customers who visited NFL stadiums this season used 55 percent more cellular traffic this year than last, according to some year-end figures from AT&T.

In the 31 different NFL venues where there is an AT&T DAS AT&T customers used 132.8 terabytes of cellular data this NFL season, with the Dec. 14 Monday night game between the New York Jets and the Miami Dolphins topping the single-game charts with 1.6 TB of DAS data used, according to AT&T. It’s appropriate that Sun Life Stadium had the biggest data game, since Miami’s home also led the NFL for highest average DAS data used, with 1.4 TB per game. Close behind in second place for average DAS use was AT&T Stadium in Arlington, Texas, where the average hit 1.257 TB this season. Third was San Diego’s Qualcomm Stadium with 1.085 TB, and fourth was Levi’s Stadium in Santa Clara, Calif., with an average of 1.054 TB each game on the AT&T DAS.

New Report: Is Texas A&M’s $20 million, all-optical DAS and Wi-Fi the fastest stadium network out there?

One of the many maxed-out speed tests we took at Texas A&M's Kyle Field. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

One of the many maxed-out speed tests we took at Texas A&M’s Kyle Field. All photos: Paul Kapustka, MSR (click on any photo for a larger image)

Is there a combined stadium Wi-Fi and DAS deployment that is as fast as the one found at Texas A&M’s Kyle Field? If so we haven’t seen or heard of it.

In fact, after reviewing loads of live network-performance data of Kyle Field’s new Wi-Fi and DAS in action, and after maxing out the top levels on our speed tests time after time during an informal walk-around on a game day, we’ve come to the conclusion that Kyle Field has itself a Spinal Tap of a wireless deployment. Meaning, that if other stadium networks stop at 10, this one goes to 11.

Movie references aside, quite simply, by the numbers Kyle Field’s wireless network performance is unequaled by any other large public venue’s we’ve tested in terms of raw speed and the ability to deliver bandwidth. With DAS and Wi-Fi speed measurements ranging between 40 Mbps and 60+ Mbps pretty much everywhere we roamed inside the 102,512-seat venue, it’s a safe bet to say that the school’s desire to “build the best network” in a stadium hit its goal as best as it could. And since the school spent “north of $20 million” on the network, perhaps it’s no surprise that it’s the fastest anywhere.

Our full profile of our in-depth visit to College Station to see this network in action can be found in our latest STADIUM TECH REPORT, our COLLEGE FOOTBALL ISSUE for 2015. You can download the report for free, right now! In addition to the Texas A&M profile you will find in-depth looks at wireless deployments at Kansas State, Ole Miss, Oklahoma and the venerable Rose Bowl — so download your copy today!

See the white dots? Those are under-seat Wi-Fi APs

See the white dots? Those are under-seat Wi-Fi APs

Audible to optical made the difference

Inside our latest 40+ page report you will get a full breakdown on how the Texas A&M network came to be — in an exclusive interview with Phillip Ray, Vice Chancellor for Business Affairs at The Texas A&M University System, you hear for the first time how much the Kyle Field network cost — “north of $20 million” — as well as how much the top two wireless carriers paid to be a part of it. Want to know? Then download the report!

And while the Kyle Field story is our lead article, that’s not all you’ll find in our latest in-depth exploration of stadium technology deployments. Reporter Terry Sweeney checks out the new DAS deployment blanketing Pasadena’s Rose Bowl, perhaps one of the toughest old-style stadium construction challenges to try to bring in wireless coverage. We also have profiles of Wi-Fi deployments at Kansas State and at Ole Miss, and a feature about covering RV parking lots with Wi-Fi at the University of Oklahoma. To top it all off we have some Wi-Fi cost/benefit analysis from yours truly, and a bonus photo feature by photographer Phil Harvey, who accompanied MSR for a recent visit to AT&T Stadium.

We’d like to take a quick moment to thank our sponsors, which for this issue include Mobilitie, Crown Castle, SOLiD, CommScope, Aruba (a Hewlett Packard Enterprise company), JMA Wireless, Corning, 5 Bars, Extreme Networks, and ExteNet Systems. Their generous sponsorship makes it possible for us to offer this content free of charge to our readers. We’d also like to thank you for your interest and continued support. Thanks for reading and enjoy the COLLEGE FOOTBALL ISSUE!

Kyle Field at kickoff.

Kyle Field at kickoff.

NFL Stadium Tech Reviews — NFC East

Editor’s note: The following team-by-team capsule reports of NFL stadium technology deployments are an excerpt from our most recent Stadium Tech Report, THE PRO FOOTBALL ISSUE. To get all the capsules in one place as well as our featured reports, interviews and analysis, download your free copy of the full report today.

NFC EAST

Reporting by Paul Kapustka

Dallas fan in mobile action at AT&T Stadium. Photo: Phil Harvey, MSR

Dallas fan in mobile action at AT&T Stadium. Photo: Phil Harvey, MSR

Dallas Cowboys
AT&T Stadium
Seating Capacity: 105,121
Wi-Fi – Yes
DAS – Yes

AT&T Stadium continues to push the envelope when it comes to wireless connectivity, perhaps no surprise given the sponsor name on the building. For 2015 the venue will have up to 2,000 Wi-Fi APs at any given time (1,900 fixed and 100 flexible for deployment where needed), making it by far the stadium with the most access points.

New York Giants
MetLife Stadium
Seating Capacity: 82,500
Wi-Fi – Yes
DAS – Yes

Like with the Jets, the Giants and their fans are still benefitting from the technology installed for the Super Bowl two years ago, with superb Wi-Fi and DAS coverage throughout the facility.

Philadelphia Eagles
Lincoln Financial Field
Seating Capacity: 69,176
Wi-Fi – Yes
DAS – Yes

One of the early Extreme deployments, the “Linc” will continue to be well-connected with Wi-Fi available to all seating areas.

Washington Redskins
FedExField
Seating Capacity: 79,000
Wi-Fi – Yes
DAS – Yes

After briefly considering Chinese firm Huawei for Wi-Fi last season, the Redskins pulled a reverse and instead went for a deployment led by Cisco and Verizon, now apparently with United Airlines as a service sponsor.

Levi’s Stadium, AT&T Stadium see lots of Wi-Fi for Seahawks visits

Seahawks vs. Cowboys at AT&T Stadium, Nov. 1. Photo: Dallas Cowboys

Seahawks vs. Cowboys at AT&T Stadium, Nov. 1. Photo: Dallas Cowboys

So far this NFL season we’ve seen something that we call the Patriots effect, where games featuring the defending Super Bowl champs as visitors produce big numbers on the stadium Wi-Fi networks. There appears to be a similar trend following the Seattle Seahawks around, especially when they’re playing NFC rivals like the San Francisco 49ers and the Dallas Cowboys, as both those teams’ stadiums saw big Wi-Fi days during recent Seahawks visits.

The Niners were first to get a Seahawks effect, carding 2.2 terabytes of Wi-Fi network usage during Seattle’s 20-3 victory over San Francisco at Levi’s Stadium on Oct. 22. According to figures provided by Roger Hacker, senior manager of corporate communications for the Niners, out of the 70,799 in attendance for the Thursday-night game there 16,299 unique users on the Wi-Fi network at Levi’s Stadium, with a maximum concurrent user number of 10,306.

Screen Shot 2015-11-03 at 10.16.54 PM

Last Sunday at AT&T Stadium, the Seahawks escaped with a narrow 13-12 win in front of 91,486 fans. According to figures provided by Dallas Cowboys chief information officer John Winborn, there were 36,388 unique users on the Wi-Fi network, with a peak of 24,667 concurrent users. The total tonnage used Sunday at AT&T Stadium was 4.12 TB, perhaps proving once again that 4+ TB is becoming the “new normal” for high-fidelity networks in the largest stadiums.

https://duwit.ukdw.ac.id/document/pengadaan/slot777/

https://mtsnupakis.sch.id/wp-content/zeusslot/

https://insankamilsidoarjo.sch.id/wp-content/slot-zeus/

https://smpbhayangkari1sby.sch.id/wp-content/slot-zeus/

https://alhikamsurabaya.sch.id/wp-content/slot-thailand/

https://mtsnupakis.sch.id/wp-content/bonus-new-member/

https://smptagsby.sch.id/wp-content/slot-bet-200/

https://lookahindonesia.com/wp-content/bonus-new-member/

https://ponpesalkhairattanjungselor.sch.id/wp-content/mahjong-slot/

https://mtsnupakis.sch.id/wp-content/slot777/

https://sdlabum.sch.id/wp-content/slot777/

https://sdlabumblitar.sch.id/wp-content/bonus-new-member/

https://sdlabumblitar.sch.id/wp-content/spaceman/

https://paudlabumblitar.sch.id/wp-content/spaceman/