Report excerpt: AT&T’s Donovan talks stadium DAS

Editor’s note: The following excerpt from our exclusive interview with AT&T senior executive vice president John Donovan comes from our Stadium Tech Report for Q2 2014, which includes a wealth of information, research and analysis about the stadium tech marketplace. With a focus on Major League Baseball technology deployments, the report is available free for download so get your copy today. Enjoy the excerpt that follows.

AT&T senior executive vice president John Donovan

AT&T senior executive vice president John Donovan

John Donovan: The ‘network chief’ talks about AT&T’s successful stadium strategy

In a strategy borne by necessity, AT&T has become far and away the leader in deploying distributed antenna system (DAS) technology in stadiums and other large venues across the country. In a recent phone interview, Mobile Sports Report spoke with the man behind the plan, AT&T senior executive vice president John Donovan, who told us the hows and whys behind AT&T’s DAS strategy, and how AT&T is continuing to innovate to improve DAS performance. Donovan also offered some interesting insights about large-venue wireless consumption patterns, based on AT&T’s thorough and varied operator experience, which is likely second to none.

With 670 DAS systems deployed in large venues, including 150 in the past year, AT&T has no plans to slow down. In fact, Donovan said AT&T will deploy another 250 DAS systems this year, adding to its impressive totals for presence in large public venues. According to Donovan, AT&T DAS systems are currently active in 75 percent of the “big 4” professional sports venues in the U.S. (football, baseball, basketball and hockey), a list thaat includes 90 percent of NFL stadiums. AT&T also has DAS deployments in a large number of top college stadiums, making Donovan confident that his company is far and away the top provider of enhanced cellular services in stadiums.

“We are absolutely crushing the competition on performance in venues,” said Donovan, who 5 years ago spearheaded a move inside AT&T to create a team that specifically targeted large public venues. Now, the fruits of that team’s labors are paying off.

Building the DAS group

The several-hundred strong group, which Donovan said included employees with experience in tower rental operations, building right-of-way negotiations, and “young, aggressive wireless” technicians, had a mandate, he said, to target all the premium venues in the U.S., and get them a world-class wireless architecture.

“The objective was, to wire them all, really,” Donovan said. Armed with a large budget (“in the early days, it was ulimited,” Donovan said), the group started ranking every large public venue, calculating stats like “seat minutes,” a value of how often a seat in a stadium would be filled.

“The Staples Center [in Los Angeles] blew everyone away — one year they had 367 events,” said Donovan.

The main reason why AT&T had to improve cellular connectivity at large venues had to do mainly with the company’s legacy as the initial, exclusive carrier for the Apple iPhone. Though Verizon Wireless and other carriers eventually got access to the iPhone in 2011, AT&T’s early lead meant that many iPhone owners were still AT&T customers — and according to Donovan, the kind of people who bought iPhones were also the kind of people who went to sporting events and concerts.

“In the early days, we were the only ones with the problem [of congestion in arenas],” Donovan said. “If you take the demographic of an event, and map it to the demographic of an iPhone buyer, you get a big overlap.” While AT&T may have only had 30 percent of the overall wireless market share, in some arenas Donovan said AT&T’s “internal” market share could represent as many as 75 percent of the fans in attendance.

In those early days of a few years ago, with many of AT&T’s iPhones still using older 2G and 3G technology, making them work in crowded arenas was a challenge, Donovan said.

“We were really forced to innovate around architectures and manage RF [radio frequency],” Donovan said. “We got really good at design.”

Fast-forward to 2014, and the team is in a much different space, innovating ahead of the curve instead of scrambling to respond to pressures. One example of the new thinking is the debut of some large, spherical antennas that AT&T used at the Coachella music festival in April.

“We had this huge thing that looked like a human-sized bowling ball with 12 [antenna] sectors in it,” Donovan said. “We’ve got a 20-sector version coming out next year.” AT&T in the last couple years has also debuted antennas that allow the carrier to focus signals into smaller geographical space, to better target the packed crowds in arenas and large venues.

“We’ve gotten a lot better at design,” Donovan said.

To read the rest of the interview, download your free copy of our Stadium Tech Report for Q2 2014.

Stadium Tech Report: Read why we think MLB is the sports digital experience leader

STRQ@_thumbIs there any doubt that when it comes to the digital fan experience, Major League Baseball is in first place? That’s the way we see things, and research we did for our second quarterly long-form STADIUM TECH REPORT issue bears that opinion out. Thanks to a far-sighted strategy that kept league control over all Internet content, and some innovative, forward-thinking technology leaders at several MLB teams, baseball is ahead of all other U.S. sports when it comes to delivering a consistent, enriched fan experience through technology. But will the lead last?

You can find some of the answers to that question in the second issue in our STADIUM TECH REPORT series which you can download for free right here. If you’ve already registered with us, all you need is a username and password; if you’re new to MSR, we just need an email address and title and you’re on your way to the best long-form compilation of research and analysis, as well as in-depth interviews with industry experts in the stadium technology marketplace. We’d also like to thank our Stadium Tech Report sponsors, which for this issue inlcude Crown Castle, SOLiD, Corning, ExteNet Systems and TE Connectivity — without their support, we couldn’t make all this excellent content free for readers.

AT&T senior executive vice president John Donovan

AT&T senior executive vice president John Donovan

MLB tech profiles, interview with AT&T’s John Donovan

What’s in our issue #2 of STR? Glad you asked! Inside the report our editorial coverage includes:

— MLB stadium tech research: This editorial research provides a technology update on stadiums used by all 30 MLB teams, gauging the level of deployment of Wi-Fi, DAS and beaconing technologies.

— MLB tech deployment profiles: These mini-case studies will take an in-depth look at technology deployments at MLB facilities including AT&T Park in San Francisco, Target Field in Minneapolis, and Miami’s Marlins Park. This issue also includes an in-depth interview with AT&T senior executive vice president John Donovan, the man behind AT&T’s successful DAS deployment strategy.

— MSR exclusive stadium tech analysis: The report also includes MLB stadium tech analysis from MSR editor in chief Paul Kapustka, as well as a bonus mini-case study of DAS deployment at historic Churchill Downs, home of the Kentucky Derby.

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park.

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park.

MLB stadiums: Wi-Fi and DAS deployment is strong

Since last year, 8 more MLB teams have added fan-facing Wi-Fi to their ballparks, bringing the league total to a respectable 67 percent, with 20 out of 30 stadiums with Wi-Fi. On the distributed antenna system (DAS) front things are even better, with 25 out of 30 parks having enhanced cellular connectivity thanks to a DAS (our report currently erroneously shows that the Washington Nationals don’t have a DAS — we learned late this weekend that they do, so DAS is doing even better than we thought). Though the adoption rate is lower than that found in the NBA (where 26 of 29 stadiums have fan-facing Wi-Fi), baseball as a league does a much better visible job of promoting the service, which is more impressive when you consider that deploying Wi-Fi in an open-air arena is a considerably tougher task than in a building with a ceiling, like an NBA stadium.

Perhaps it’s no surprise that the National League West Division leads the pack in MLB connectivity, with all 5 teams having both Wi-Fi and DAS deployments in their park. The adoption rate in that division may have something to do with other teams following what is perhaps the overall connected-stadium leader in any sport, AT&T Park in San Francisco. The first with Wi-Fi (since 2004), AT&T Park continues to lead in innovation and experimentation, as witnessed by their embrace of new tricks like under-the-seat Wi-Fi APs and the new iBeacon technology, which is being tested in 20 MLB parks this season.

All of this is explained in greater detail in our STADIUM TECH REPORT for Q2 2014 — so download your free copy today!

Stadium for next U.S. World Cup game will have Ruckus-powered Wi-Fi available

Talk about luck of the draw: Fans of the U.S. men’s soccer team who are on hand for the squad’s next game on Tuesday, July 1, should be able to use a Ruckus Wireless-powered Wi-Fi network at the 51,900-seat Arena Fonte Nova in Salvador, Brazil.

As MSR reported last year, Ruckus Wireless won contracts to supply the Wi-Fi gear for at least two of the stadiums being used in the World Cup tournament. One of those was Arena Fonte Nova, which will now host the USA-Belgium tilt in the elimination round. And with U.S. fans leading the way in on-scene participation in the World Cup, it’s a good bet that the Arena Fonte Nova network will get a workout on Wi-Fi since it’s probably a good guess that cellular roaming fees might be prohibitive. (We say this having no idea what the roaming charges might be for U.S. carriers in Brazil; but if the Wi-Fi is free, we can say without a doubt that’s the cheaper option.)

According to Ruckus the Wi-Fi network at the arena (which is also called the Estadio Octavio Mangabeira) will have about 150 access points, which to us sounds a bit low for full stadium coverage for a 51,000-seat arena, but again, we’ve not seen any stats for a Ruckus solution in real-world situations so maybe it’s enough.

With more news about players biting players and Brazilians protesting the World Cup, it’s not surprising that we haven’t heard much in the way of news about performance or non-performance of wireless networks at the numerous stadiums. Unlike an Olympics or a Super Bowl, where you are in one place for a long time, the World Cup seems to be more about sporadic visits to different stadiums, and a lot of travel, and probably a lot more time spent just enjoying the scene rather than worrying about tweeting it or posting to Facebook. And since there is no reliable news source for overall network operations for the World Cup, we don’t have any statistics or performance metrics to report. But so far so good, eh? If anyone knows of any specific problems or reports of poor performance send them our way. And if anyone is in Salvador next week, take a Speedtest of the Wi-Fi and send it along.

Stadium Tech Report: Miami Marlins rely on ExteNet DAS to keep wireless traffic flowing

Marlins Park. Credit all photos: Miami Marlins.

Marlins Park. Credit all photos: Miami Marlins.

If you know anything about Marlins Park, maybe it’s the stadium’s unique retractable roof or the spectacular art that catches your eye. But there’s also something you can’t see that is equally exciting, at least when it comes to the in-stadium connectivity experience: A neutral-host distributed antenna system (DAS) that has more than kept pace with the rapid, continual increase in fan cellular activity.

“When it came to DAS, we were ahead of the game,” said David Enriquez, senior director of information technology for the Miami Marlins, in a recent phone interview. Well before the 37,000-seat stadium opened in 2012, Enriquez said the Marlins’ IT team was researching and planning for enhanced cellular connectivity – even before “DAS” became a hot industry acronym.

“We planned for a DAS even before they were in vogue,” said Enriquez. “We saw it as a necessary evil.”

With the iPhone and all its cataclysmic changes already in motion, Enriquez said the Marlins wanted to avoid what had happened recently at another arena that opened in the Sunshine state without good connectivity.

“What we didn’t want to see was something like what happened in Orlando, when they opened the arena [in 2010], it had bad coverage, and they were crucified in the press for bad [cellular] service,” Enriquez said. “We said, what we’d love to have is the complete opposite of that.”

David Enriquez

David Enriquez

At the opening of Marlins Park, the connectivity inside the walls was better than most, with a full-park Wi-Fi network using gear from Meru Networks and a neutral-host DAS deployed by integrator ExteNet Systems. And though Wi-Fi often gets the headlines when there is talk about stadium networks, in many facilities like Marlins Park, the DAS is an equal workhorse, since many fans still either don’t know how or don’t take the time to switch their devices over to Wi-Fi.

DAS is the workhorse

According to Enriquez, on an average night at the ballpark the Wi-Fi network will handle 40 percent of the wireless traffic, with the DAS taking care of the other 60 percent. That may be because of lack of knowledge, or perhaps satisfaction with the signal the DAS is giving them, Enriquez said.

“Early on, most people, honestly, did not know how to change [their phone] to Wi-Fi,” Enriquez said. Most fans, he added, weren’t typically streaming lots of video — they may, he said, have used the MLB At Bat app to look at a replay or two, but that could all be handled by DAS. “That trend is changing though and we are seeing much more video traffic, especially with the younger generation of guests,” Enriquez said.

Marlins Park outside

Marlins Park outside

“The truth is, many users may not take the time to switch [to Wi-Fi],” Enriquez said. “If they’re getting 4 to 5 bars on their cellular signal, they’re happy.”

Though the Marlins and ExteNet now have five major carriers on their DAS – AT&T, Verizon Wireless, Sprint, T-Mobile and MetroPCS (now part of T-Mobile), Enriquez said there was a bit of the chicken and egg problem at the start.

“Early on, nobody wanted to be the first on (the neutral DAS),” Enriquez said. “ We [the stadium] were just another node. Now, 3 years later, we are a central node in the Miami area and all the carriers are here. We’re a very central location.”

Staying in neutral

Enriquez, who has considerable experience in the large-venue IT world, said that having a neutral host for the DAS eliminates any potential concerns about favoritism between service providers. Even though costs to the team or stadium may be lower if they allow a carrier to take over DAS deployment, Enriquez said that for the Marlins a neutral host was worth the extra price.

“We didn’t want an advantage to be held by one carrier,” Enriquez said. Even if a carrier says it will act as a neutral host, when one carrier owns the deployment, others can “find it hard to believe there will be an equal time slice” when it comes to antenna access.

“We just wanted to avoid that, and make it irrelevant [as a concern],” Enriquez said.

The choice of bringing in an integrator like ExteNet, he said, provides an additional streamlining of operations, as there is now a single point for vendors to interact with to work out technology and deployment issues.

“We wanted to deal with one vendor – I didn’t want to be the middleman between the carriers and the Marlins,” Enriquez said. In that regard, he said, ExteNet has been “wonderful” as a neutral host. “They deal with all the carrier issues that I have no desire to deal with,” Enriquez said.

Less space needed for DAS upgrades

And even as fan cellular bandwidth use continues to grow – requiring carriers to constantly upgrade their systems – Enriquez said that DAS infrastructure is benefiting from improved technology to the point where even as carriers upgrade, their head end footprint is shrinking.

AT&T, for instance, has upgraded its DAS presence in Marlins Park four times over the past 2 years, Enriquez said, to the point where the carrier now has coverage for all four frequency bands. “They [AT&T] have done quite a bit to expand their coverage,” Enriquez said.

Still, the Marlins Park DAS head end hasn’t had to find new space beyond its original 1,500-square foot enclosure.

“Every time someone comes in to replace gear, we have a smaller [DAS] footprint,” Enriquez said. “It’s not going to eat you out of house and home anymore.”

Like other stadium IT directors, Enriquez is still surprised by the amount of wireless traffic generated by the fans who come to the games. “It’s incredible to see the need [for bandwidth” grow,” he said. “But people continue to give our network a thumbs up, we see that in our guest comments all the time. I just don’t know what we would do without the DAS.”

Stadium Tech Report: Atlanta Hawks’ Philips Arena signs Boingo for DAS, Wi-Fi

Philips Arena. Credit: Atlanta Hawks.

Philips Arena. Credit: Atlanta Hawks.

As the central event stadium of its kind in the region, Atlanta’s Philips Arena is well known for the entertainment options it offers, from professional sports to concerts, ice-show extravaganzas and even the circus. What it didn’t want to continue to be known for was its inability to provide service when fans at the stadium tried to connect to the Internet.

“We wanted to avoid the spinning [timeout] wheel of death,” said Andrew Steinberg, senior vice president and chief revenue officer for the Atlanta Hawks and for Philips Arena, in a recent phone interview. Home to the NBA’s Hawks, Philips stood out as one of just six NBA arenas without fan-facing Wi-Fi services, and with no cellular DAS.

That will change soon, thanks to a deal Philips signed with Boingo Wireless to provide free fan Wi-Fi and a neutral host DAS for the facility by the start of the next NBA season.

Andrew Steinberg

Andrew Steinberg

According to Steinberg, the area’s well-connected citizens, who probably trail only California’s Silicon Valley in devices per person, weren’t happy when they tried to access their social networks at Philips.

“We know people are coming to arena have been very challenged, just to tweet or post a picture to Instagram,” Steinberg said. “We needed to correct that.”

The process to do so actually started a couple years ago, when the Philips IT team put out a request for proposals that was very detailed in nature. “We wanted a robust backbone and infrastructure, and we were very thorough,” said Steinberg about the review process. “We wanted to find the best solution. And Boingo [Wireless] had the best fit for our plans.”

Neutral DAS a necessity

Philips Arena

Philips Arena

One of the requirements for Philips was that its DAS host needed to be a neutral third party, and not a single carrier. “I much prefer a neutral host DAS,” said Steinberg. “I’ve seen it done different ways. What I prefer is a best in class solution from a partner who can further engage the carriers.”

Boingo, Steinberg said, “delivered on that promise” and will have all the major carriers online when the DAS deployment is ready to go for the next NBA season.

Doug Lodder, vice president of business development at Boingo Wireless, said Steinberg and the Hawks were “very thorough” with their request for technology. “They took the time and did due diligence – they were way out in front of most venues [in their proposals],” Lodder said.

According to Steinberg, the infrastructure at Philips – which opened in 1999 – presented no special challenges to deployment. There was even plenty of space on site for the DAS head end, which is sometimes a challenge for stadium retrofits.

If there is a challenge, it could come from Boingo’s crews having to work around the extensive activity schedule at Philips, which includes concerts, games, and even the occasional “mudder” race with mud pits and obstacles. According to Steinberg, the arena averages about 130 events a year. But Lodder said Boingo is used to such deployment scheduling. “We can do three shifts in a row if needs be,” Lodder said.

Build it, then you can use it

Even though Philips hasn’t had high-end connectivity, that doesn’t mean Steinberg and his team haven’t been looking at what a good network will allow them to do.

“One of the foremost challenges is to increase revenue while improving the fan experience,” said Steinberg, “while also keeping the fans engaged with the game or event.”

From activities like seat upgrades to future apps, Steinberg said the Philips team knew they needed a good network, as soon as possible.

“The fan engagement potential and opportunities are large, but they require the necessity of connectivity,” said Steinberg. “We have a robust tech roadmap, and having connectivity like this was something we felt was paramount.”

Especially so in an area like Atlanta, which has more than its share of people whose lives already revolve around connectivity. Steinberg said that expectation shouldn’t end just because those people wanted to attend an event at Philips.

“We want people to experience a seamless transition from their office to our venue, and not have a disconnect,” Steinberg said.

Boingo bags DAS for Zags

McCarthey Athletic Center, Gonzaga University. Credit: Gonzaga Athletics

McCarthey Athletic Center, Gonzaga University. Credit: Gonzaga Athletics

Boingo Wireless bagged two more deals to deploy neutral-host DAS installations, including one at the home of Gonzaga University’s McCarthey Athletic Center, according to a release from the company.

Boingo, which is on a bit of a roll with contract wins for DAS and Wi-Fi deployments in stadiums, said it will be the exclusive DAS host partner for both the 6,000-seat McCarthey Center — home of the well-known men’s and women’s basketball teams from Gonzaga — as well as the nearby Spokane Veterans Memorial Arena in Spokane, Wash. According to the Boingo release, the Veterans Memorial Arena seats more than 12,000 fans, and is home to the Spokane Chiefs Hockey Club of the Western Hockey League and the Spokane Shock of the Arena Football League. The Spokane Arena is the also the largest entertainment venue in the region, and according to the release hosts more than 120 major events a year.