October 21, 2014

Wi-Fi Whispers: Standing fans and marching band interference; NFL sideline Wi-Fi fixed?

Baylor students standing at football game. Photo credit: Rod Aydelotte, WacoTrib.com

Baylor students standing at football game. Photo credit: Rod Aydelotte, WacoTrib.com

Welcome back to another posting of Wi-Fi Whispers, our once-active “notes” column that I think makes sense to start back up again. What I like most about this feature is that it gives me a chance to comment and “move the ball forward” on other stadium Wi-Fi (and DAS) stories I’ve seen out there, or to correct/comment/criticize as needs be.

STAND UP FOR WI-FI: First up is a story about the Wi-Fi network at Baylor University’s new McLane Stadium, and how it ran into some unexpected interference — namely, the fact that a lot of Baylor students like to stay standing during the game, something that messed with the initial access point and antenna deployment.

According to a story by Regina Dennis at WacoTrib.com, Baylor will be tweaking the Wi-Fi antennas to compensate for the standing fans, which is apparently a Baylor tradition (and something we’ve seen at other schools). One of the more interesting quirks of Wi-Fi reception is the fact that water is a big blocker of signals in the unlicensed bands used by Wi-Fi; and since human bodies are mostly water, a standing fan could become a sort of Wi-Fi shield. Here’s a quote from the story that tells more:

“There’s really no issue with the design, it works absolutely perfect — as long as people are either sitting in their seats or standing on the ground,” said Pattie Orr, Baylor’s vice president of information technology. “It doesn’t ruin the reception, but it makes the angles of the antenna not the best option.”

The story also notes that the Wi-Fi signals were degraded in the area where the Baylor band sits, in part because of the large instruments used. Extreme Networks, supplier of the Wi-Fi gear in McLane Stadium, will help Baylor fine-tune the system, according to the story. And we are sure we’ll hear some interesting stories about “tuba interference” at next year’s SEAT conference.

SIDELINE WI-FI IS FINE, SAYS NFL: While it’s a little tough getting tech information out of the NFL these days due to the public-relations siege the league is facing, we did get a source to comment on background about an article saying that the league’s sideline Wi-Fi isn’t working well. The article is basically a late follow-up to an issue that “surfaced” early in the season, when New England Patriots head coach Bill Belichick complained that the new system of Microsoft Surface tablets and league-provided Wi-Fi wasn’t up to snuff.

The follow-on article at TechTarget’s SearchNetworking site says in its headline that “NFL’s Cisco-based stadium Wi-Fi frustrates tablet-toting coaches,” but it only has one coach, Belichick, complaining, and it doesn’t have any new reactions or information other than the original Boston Globe story. What it does have is confirmation that Cisco equipment is used in the sideline Wi-Fi deployment, then several reactions from industry analysts — none of whom have any actual information about the system or any actual, proven or reported failures.

While it’s possible that there have been more glitches, our question is, why write an article about it if you can’t find more than the single episode, from someone you haven’t really talked to in order to find out what really happened? Our league source, who would only comment on background, said there were issues with network interference during the preseason, but that now the systems are working in all stadiums. Of course, if any coaching staff types beg to differ you can always let us know here at MSR, but our guess is that if the systems were failing you’d hear complaints since NFL head coaches aren’t the silent type.

Why can’t the sideline networks just tap into the stadium Wi-Fi, especially in places where the new networks are powerful? According to Dan Williams, the vice president of technology for the San Francisco 49ers, the sideline networks need to be separate for security and objectivity reasons — meaning, so you can’t accuse the home team of screwing up the network to gain advantage. As soon as we can, we’ll get a more thorough report together on the deployments, which we think are pretty interesting — and probably a lot better than running photos down a wire from the press box, like they used to in the old days.

LEVI’S STADIUM ANNOUNCES JMA AS DAS GEAR SUPPLIER: You may have read our feature about Levi’s Stadium DAS deployer DAS Group Professionals, but for a little deeper inside-baseball dive there is a press release from the Niners today announcing that JMA Wireless is one of the gear suppliers behind the DGP DAS. JMA, also known inside the industry for the Teko Telecom products it uses, is behind other stadium DAS deployments, including at the Anaheim Angels’ ballpark.

Stadium Tech Report: Wi-Fi + advanced stadium app helps Philadelphia Eagles ‘Linc’ with fans

Wi-Fi gear on the exterior of Lincoln Financial Field. Credit all photos: Philadelphia Eagles

Wi-Fi gear on the exterior of Lincoln Financial Field. Credit all photos: Philadelphia Eagles

With victories in their first three games, the Philadelphia Eagles are off to a fast start this NFL season. And from a networking standpoint, Eagles fans are keeping pace, with Wi-Fi connections at Lincoln Financial Field already surpassing last year’s totals.

Now in the second season of having full-stadium Wi-Fi available for fans, the Eagles’ technology team is pushing the needle forward, much like the high-powered offense head coach Chip Kelly runs on the field. An already advanced stadium app will soon get even more video features, including instant replay, to further enhance the game experience for the 69,176 fans who fill the “Linc” on home-game Sundays.

According to Anne Gordon, the Eagles’ senior vice president for media and communications, there were more than 21,000 fans using the in-stadium Wi-Fi network at the team’s Sept. 7 opening game, a total that surpasssed the 19,671 users on the network at the Eagles’ final game last season, a 26-24 loss to the New Orleans Saints in a wild card game on Jan. 4, 2014.

Anne Gordon, SVP Media and Communications, Philadelphia Eagles

Anne Gordon, SVP Media and Communications, Philadelphia Eagles

Along with the growth in user numbers is an even greater jump in the amount of data being used; according to Gordon, the Eagles’ Extreme Networks-powered Wi-Fi network carried 946 Gigabytes of data in the Sept. 7 game against the Jacksonville Jaguars, up from around 400 GB used during the playoff game in January. Unlike some other stadiums with Wi-Fi networks, Gordon said that the Linc network regularly sees fans download more data than they upload — a reflection of the team’s strategy to provide a wide range of custom content for fans to help improve the game-day experience.

“We want to help them become better fans, and improve their enjoyment of the game,” said Gordon in a recent phone interview. “That was our vision from the beginning.”

But to get to that vision, the Eagles first had to give fans a way to get the content. That meant using some of the $125 million in recent renovation fees to build out the stadium-wide Wi-Fi network, which Gordon said eliminated past connectivity headaches for Philadelphia fans.

Build it, and show them how to use it

Screen shot of Eagles' stadium app

Screen shot of Eagles’ stadium app

“Prior to the 2013 season, there were real issues trying to connect mobile devices in the stadium,” Gordon said. “You might get a signal, but then walk 20 steps and lose it. When we talked to fans, improving the wireless network [in the stadium] was high on their list.”

Working with partner Extreme Networks, which now runs Wi-Fi networks in four other NFL stadiums, the Eagles had stadium-wide coverage ready to go before the 2013 season kicked off. But unlike some venues, which do little to promote their in-stadium networks, the Eagles and Extreme brought some more NFL flavor to their wireless experience in the form of “Wi-Fi coaches,” network-savvy people who roamed the stands in readily identifiable clothing, offering personal assistance to fans trying to connect.

“We put notes in the cup holders the first few games of last season, and then had the coaches with jackets that said ‘ask me’ on them,” Gordon said. That little bit of assistance, she said, spreads quickly.

“If you connect one person, that fan shows six more people in the row [how to connect],” Gordon said. “Our fans took to the network immediately.”

And just in case fans need a refresher, the team’s website has perhaps the league’s best help pages, with simple screen shots showing how to install, open and use the most popular features on the team app.

Out front with app features

As we found out in our recent report on NFL stadium technology deployments, there are many teams with stadium Wi-Fi networks, but the level of application and content delivery varies from team to team. With a feature lineup that includes in-stadium access to the NFL’s popular RedZone channel alongside a live feed from the stadium’s large video boards, the Eagles’ app was clearly among the league leaders when it launched last year. And soon, Gordon said the Eagles will add more live camera views and replay choices to the menu, developments made possible in part by the team’s close relationship with app designer YinzCam Inc.

“We have a unique relationship with YinzCam, and work hand in hand with them [on new developments],” Gordon said. “We’re blessed in that we get a lot of things in our app first.” The Pittsburgh-based YinzCam, which has designed team and stadium apps for a long list of sports-team customers, is a preferred team-app partner of the NFL, which was an early investor in the company.

Can you find the Wi-Fi access point?

Can you find the Wi-Fi access point?

With a long history of full houses, Gordon said the team doesn’t need to use its network or app strategy to try to put people in seats.

“We are continuously sold out, so thankfully we don’t have to sell tickets [with the app],” said Gordon, noting that some fans have had season tickets in their families for several generations. That fact allows the Eagles’ tech team to make their digital strategy “100 percent about improving the game experience,” Gordon said. “We’re giving them a reason to download and consume.”

Not possible without the network

With more than 700,000 downloads of the stadium app so far, it appears as if the Eagles have a winning digital strategy to match the team’s recent on-field successes. Now the biggest challenge may be finding enough Internet bandwidth to keep the fans supplied with the in-game content.

“We are definitely bumping up against our [bandwidth] pipe threshold,” Gordon said. So far, it looks like the campaign to use content to improve the experience at the Linc is working — along with the network that links it all.

“If the [Wi-Fi] network doesn’t work, people get frustrated and don’t use the app,” Gordon said. “The network is what had to happen to make this vision possible.”

Stadium Tech Report: THE FOOTBALL ISSUE arrives, with extensive coverage of Levi’s Stadium launch and tech reports on all 31 NFL stadiums

STR3_ThumbMobile Sports Report is pleased to announce the arrival of THE FOOTBALL ISSUE, our third Stadium Tech Report for 2014. As the title suggests this long-form report focuses on technology deployments at U.S. football stadiums, with an extensive inside look at the technology inside Levi’s Stadium, the new facility for the San Francisco 49ers. The report is available for free download from our site.

In addition to our Levi’s coverage, the Q3 issue of Stadium Tech Report also includes team-by-team reports on all 31 NFL stadiums, with a focus on Wi-Fi and DAS deployments. Our research found that while there are still 10 stadiums without fan-facing Wi-Fi, there is a lot of innovation going around league-wide, including big new digital displays in Jacksonville and Dallas, and new Wi-Fi and app deployments in other facilities.

Included in the report is an exclusive MSR interview with Michelle McKenna-Doyle, the NFL’s chief information officer, who talks about how the league office acts as a guide to helping teams with their tech deployments. We also have additional insight, analysis, and more tech profiles, and the good news is it is all free to read! Simply head to our report download page and get your free copy today!

We’d like to take a quick moment to thank our report sponsors, without whom we wouldn’t be able to offer such extensive original reporting and analysis free of charge. Our list for the Q3 2014 report includes SOLiD, Crown Castle, TE Connectivity, Extreme Networks, Aruba Networks, Mobilitie and DAS Group Professionals.

Planning a DAS deployment? Check out these AT&T stadium stats from football opening weekend

There’s lots of shared belief out there that fans want to use mobile devices while they are attending sporting events. What we like even more here at Mobile Sports Report are hard numbers that tell us just how much fans are using mobile devices while they are in stadiums and arenas. Though sometimes hard to get, such statistics are great signposts for those who are planning to build their own stadium networks sometime soon, because it gives them a target to shoot for.

Courtesy of our friends over at AT&T, here are some traffic statistics gleaned from AT&T’s distributed antenna system (DAS) deployments in major professional and college football stadiums during last weekend’s games. According to AT&T, the average amount of cellular data used at a pro venue home opener in 2014 was 361 gigabytes, up 59 percent from season-opening games last year. The stats, remember, are only from AT&T DAS networks at the 16 NFL stadiums where AT&T has a DAS network presence.

The top stadium in terms of AT&T DAS traffic was Miami’s Sun Life Stadium with 1,035 GB (or 1.035 TB), followed by AT&T Stadium in Dallas with 889 GB and Atlanta’s Georgia Dome with 696 GB. It will be interesting to see whether or not any of those totals are surpassed by the San Francisco 49ers’ Levi’s Stadium, which hosts its first regular-season home game this Sunday.

On the collegiate side, the average opening-day traffic number was 288 GB, which is huge compared to the 2013 season average of 186 GB per game. The stats are an average taken from 25 Division 1 schools where AT&T has a DAS presence. And in case you were wondering where college football is king, the stats for schools in the South was an average of 343 GB, with the rest of the country checking in at 186 GB. The top three schools in terms of home-opener DAS traffic were Oklahoma with 866 GB, Georgia with 688, and the new stadium at Baylor with 686 GB.

More AT&T DAS stats follow. Remember, this is just a fraction of the actual traffic since many of these venues also have other carriers on DAS networks, as well as Wi-Fi networks in place. Other carriers and stadiums — send us your stats!

AT&T STADIUM DAS STATS

Year-over-Year Professional Mobile Data Increase

o 2014 home opener – 361GB average per venue

o 2013 season average – 227GB average per venue

· 2013 season average to 2014 season opener is a year-over-year increase of more than 59 percent

· 2013 season opener to 2014 season opener is a year-over-year increase of more than 58 percent

Opening Weekend Mobile Data Usage Professional vs. Collegiate

o 2014 professional season opener – 361GB average per venue

o 2014 collegiate season opener – 288GB average per venue

Top 5 Opening Weekend Professional Venues by Total Mobile Data

o Miami, FL – 1035GB

· Equivalent to more than 2.9 million social media post with photos

· Peak hour – 1-2pm ET – 146GB

o Dallas, TX – 889GB

· Equivalent to more than 2.5 million social media post with photos

· Peak hour – 2-3pm CT – 158GB

o Atlanta, GA – 696GB

· Equivalent to nearly 2 million social media post with photos

· Peak hour – 2-3pm ET – 159GB

o Chicago, IL – 452GB

· Equivalent to nearly 1.3 million social media post with photos

· Peak hour – 1-2pm CT – 107GB

o Pittsburgh, PA – 385GB

· Equivalent to more than 1.1 million social media post with photos

· Peak hour – 2-3pm ET – 83GB

Caveats:

· All figures include only data traffic seen on AT&T’s venue-specific mobile network.

· All data metrics come from only venues with a DAS where AT&T’s mobile network is on-air. These metrics are not comprehensive of every game played during the opening weekend for professional or college football.

· This data is compiled from 16 professional football stadiums and 25 division one college football stadiums that had opening week home games where AT&T is on-air on a DAS.

· All 2013 season average data is compiled from only venues with a DAS where AT&T’s mobile network was on-air and games where data was tracked and available.

ADDITIONAL AT&T COLLEGIATE DAS STATS

Regional Breakdown by Mobile Data Usage

o South – 343GB average per venue

o Rest of the US – 186GB average per venue

· Year-over-Year Mobile Data Increase

o 2014 home opener – 288GB average per venue

o 2013 season average – 155GB average per venue

· Top 5 Opening Weekend College Venues by Total Mobile Data

· Norman, OK – 866GB

o Equivalent to more than 2.4 million social media post with photos

· Athens, GA – 688GB

o Equivalent to more than 1.9 million social media post with photos

· Waco, TX – 686GB

o Equivalent to more than 1.9 million social media post with photos

· Auburn, AL – 506GB

o Equivalent to more than 1.4 million social media post with photos

· Los Angeles, CA – 469GB

o Equivalent to more than 1.3 million social media post with photos

Caveats:

· All figures include only data traffic seen on AT&T’s venue-specific mobile network.

· All data metrics come from only venues with a DAS where AT&T’s mobile network is on-air. These metrics are not comprehensive of every game played during the opening weekend for college football.

· This data is compiled from 25 division one college football stadiums that had opening week home games where AT&T is on-air on a DAS.

· All 2013 season average data is compiled from only venues with a DAS where AT&T’s mobile network was on-air and games where data was tracked and available.

· The “South” region is based off how it is qualified by the U.S. Census. It includes the following states: Delaware, District of Columbia, Florida, Georgia, North Carolina, South Carolina, Virginia, West Virginia, Alabama, Kentucky, Mississippi, Tennessee, Arkansas, Louisiana, Oklahoma and Texas.

Bengals tap Extreme for Wi-Fi, TE Connectivity for DAS at Paul Brown Stadium

Screen Shot 2014-09-10 at 10.42.52 AMEven though the NFL season is barely underway, there’s one organization that already has a multi-win streak going: Extreme Networks said it will provide a Wi-Fi deployment for the Cincinnati Bengals at Paul Brown Stadium, the fifth NFL team deal for Extreme and the third announced this year.

The Wi-Fi network at Paul Brown Stadium is scheduled to go live this Sunday for the Bengals’ home opener against the Atlanta Falcons. According to the team and Extreme, the Bengals will also utilize the Extreme Wi-Fi coaches strategy, where knowledgeable network types roam the stands to help fans connect. Prior to this year, Extreme won Wi-Fi deals for the New England Patriots’ Gillette Stadium and the Philadelphia Eagles’ Lincoln Financial Field. In addition to Cincinnati, this year Extreme has announced Wi-Fi projects for the Tennessee Titans’ LP Field, and a joint deal with SignalShare to bring Wi-Fi to EverBank Field, home of the Jacksonville Jaguars.

And while we haven’t seen a press release yet we also know that TE Connectivity is in charge of installing a DAS at Paul Brown Stadium, meaning that cellular reception should improve as well.

We haven’t yet spoken to the folks at Paul Brown Stadium, but here are some prepared quotes from the press release to tell you how happy they are to be joining the NFL Wi-Fi fraternity:

“Our fans are of the utmost importance to our organization, and as technology continues to provide new possibilities, it was crucial that we identify a way to provide fans a truly differentiated in-person game day experience,” said Michael Kayes, Bengals director of technology. “The partnership with Extreme Networks provides the foundation to do just that, affording fans the connectivity they’ve grown to expect. We’re also excited to be able to offer our fans the opportunity to get field-level action through NFL RedZone.”

Here’s a Twitter pic of the press conference today:

Niners: More than 1,000 fans used in-seat food delivery at 2nd Levi’s Stadium preseason game

Screen grab from Levi's Stadium app showing in-seat food delivery option. Credit all photos: Paul Kapustka, Mobile Sports Report.

Screen grab from Levi’s Stadium app showing in-seat food delivery option. Credit all photos: Paul Kapustka, Mobile Sports Report.

One of the more unique features the San Francisco 49ers are introducing at their new home this year is the ability for all fans to have food and drink delivered to them, no matter which one of Levi’s Stadium’s 68,500 seats they are in. And according to the Niners, the feature is quickly catching on, with more than 1,000 in-seat orders delivered at the second preseason game at Levi’s on Aug. 24.

At a Levi’s media technology tour Wednesday, the Niners’ tech staff provided an in-depth and up-close look at some of the new stadium’s network infrastructure, including a quick glimpse of one of the several data-center rooms. Dan Williams, vice president of technology for the Niners, said that at the first preseason game at Levi’s on Aug. 17, more fans used the “express pickup” service to order food that they could then pick up at concession stand windows than the seat-delivery feature. But at the second game against the San Diego Chargers, more fans went for the in-seat option, perhaps a sign that Niners fans are learning and testing the new stadium services as they go.

“It’s going to be an ongoing education process,” said Martin Manville, business operations analyst for the Niners and one of the key tech leaders on the Levi’s app team. Manville said the Niners had learned a lot about food delivery in test situations at Candlestick Park last season — and some of those lessons are now evident in the Levi’s delivery menu, which is stripped down to ensure the food runners can get grub to fans before it gets cold (or warm, in the case of cold beverages). According to Manville the average delivery time at the Aug. 24 game was between 10 and 12 minutes, but the team expects that “normal” delays during the regular season will be closer to 15 to 20 minutes per order.

Still, that’s not a bad option if you don’t want to leave your seat and you don’t mind the extra $5 delivery charge. One early interesting stat from the food-delivery process is that the Niners found more orders coming from the “cheap seats” in the north and south stands at Levi’s than from the 50-yard-line seats where the high rollers sit. According to Manville, since the fans in the club seats have easy access to numerous uncrowded concession stands they may not see the need for the in-seat options.

Wi-Fi APs: 600 in Levi’s bowl seating

Under-seat Wi-Fi AP at Levi's Stadium.

Under-seat Wi-Fi AP at Levi’s Stadium.

Some other news nuggets from the tech tour day: According to Williams, of the 1,200 Wi-Fi access points at Levi’s, 600 of those are distributed in the seating areas (aka “the bowl”), with the other 600 placed in suites, concourses and other stadium areas. Williams said the Aruba Networks Wi-Fi antennas are basically split into three types — regular enterprise-type APs for suite and office areas, regular outdoor APs for concourse areas, and more specialized versions (including the under-seat APs) for bowl placements.

– For the app itself, the Niners said that there have been 80,000 downloads so far, with almost half of the season ticket holders having put their ticketing information into the app. The “NiNerds,” the geek-dressed help squad that provides fans with personal assistance with the app, is now at about 50 or 60 strong at each event (originally the team had said it wanted to hire 150 such Wi-Fi coaches). The Niners said the NiNerds will be doing more pro-active app education going forward, doing things like approaching fans in concession lines to see if they know about the express line or in-seat ordering options.

– Though Comcast’s 10-year deal with the Niners calls for the cable provider to bring in two 10-gigabit backbone pipes, the Niners are often quoted as saying they have 40 GB of backbone bandwidth. We solved this mystery today: According to Comcast, the other two 10-GB pipes are a redundant channel from another (unnamed) provider. So: the stadium does have four 10 GB bandwidth pipes, by far the most capacity in any stadium we’ve heard of.

– More traffic stats: Though we will break these down in a separate post, the Niners said that for the Aug. 24 game fans used 1.96 Terabytes of Wi-Fi traffic, just a bit lower than the 2.13 TB used at the first preseason game on Aug. 17. The team also provided some DAS stats, claiming fans used another 1.02 TB of cellular data at the two preseason games combined.

Tech tour photos follow… including a sighting of some (shhhh!) Cisco equipment in the data center racks… click on photos for larger images.

Niners VP of technology Dan Williams (left) and COO Al Guido kick off the Wednesday tech tour.

Niners VP of technology Dan Williams (left) and COO Al Guido kick off the Wednesday tech tour.

Ted Girdner, Comcast VP of business services for California, talks stadium networking.

Ted Girdner, Comcast VP of business services for California, talks stadium networking.

Dan Williams talks Wi-Fi while the Levi's Stadium new turf grows silently behind him.

Dan Williams talks Wi-Fi while the Levi’s Stadium new turf grows silently behind him.

Mystery Cisco gear inside Levi's Stadium data center. Alert! Intruder!

Mystery Cisco gear inside Levi’s Stadium data center. Alert! Intruder!

Brocade router at Levi's Stadium data center. One of many. As in, many many.

Brocade router at Levi’s Stadium data center. One of many. As in, many many.

Wi-Fi gear in Levi's Stadium data room.

Wi-Fi gear in Levi’s Stadium data room.

Franks and DAS: DGP DAS antennas above food station.

Franks and DAS: DGP DAS antennas above food station.

Screenshot of food feature on Levi's Stadium app. Note the green light buttons to show expected wait times for express option.

Screenshot of food feature on Levi’s Stadium app. Note the green light buttons to show expected wait times for express option.

Obligatory Levi's Stadium selfie. MSR shirts complete the style.

Obligatory Levi’s Stadium selfie. MSR shirts complete the style.