Stadium Tech Report: Churchill Downs picks a winner with Mobilitie DAS

The iconic twin spires of Churchill Downs, home of the Kentucky Derby. Credit all photos: Churchill Downs

The iconic twin spires of Churchill Downs, home of the Kentucky Derby. Credit all photos: Churchill Downs

Move over, Super Bowls — there’s a new wireless-traffic event king on the U.S. sports landscape, and it’s one that’s been around longer than anything else. Thanks to a new distributed antenna system installed this year at historic Churchill Downs, fans at the recent Kentucky Derby were able to set single-event wireless traffic records, at a place where not too long ago getting a decent signal on your phone was a longshot at best.

According to Churchill Downs Racetrack general manager Ryan Jordan, up until last year cellular connectivity was a pain point at the historic facility, which opened in Louisville, Ky., in 1875. Though recently cellular providers tried to address the smartphone crunch by bringing in technology like COWs (cellular towers on wheels), the spread-out nature of the track and its architectural challenges made the COWs an unsatisfactory solution.

“The temporary COWs really couldn’t handle the traffic,” said Jordan in a recent phone interview. In post-race surveys, he said, the track heard complaints about connectivity, as did the cellular carriers.

“We heard it [the complaints] and the carriers heard it,” Jordan said. “But we never had a solution to handle the problem.”

In 2013, however, Churchill Downs enlisted infrastructure provider Mobilitie to deploy a neutral-host DAS to bring better cellular connectivity to the entire facility. While the deal was struck too close to last year’s race to allow for a full buildout, by this year’s 140th running of the “Run for the Roses,” a full-scale, 253-antenna DAS was in place to produce some winning results. According to AT&T, the Churchill Downs DAS carried event-record traffic over the Kentucky Derby and Kentucky Oaks weekend, with a total of 2 Terabytes of overall traffic and a peak hour of 180 Gigabytes of traffic, in the 60 minutes just before the Derby race on May 3.

Ryan Jordan, general manager, Churchill Downs

Ryan Jordan, general manager, Churchill Downs

By comparison, during the most recent Super Bowl at MetLife Stadium in New Jersey, AT&T saw a peak hour of 119 GB of traffic before kickoff, and 624 GB of total traffic on game day. Of course, some of the specifics of the Kentucky Derby make the big numbers easier to understand, including the 160,000 fans on race day as well as the full day of attendance, which featured less than a half-hour total of actual horses running fast. It’s important to note that the Derby figures are for AT&T traffic only, and do not include any potential traffic from Verizon Wireless customers, whose carrier also used the Mobilitie DAS to improve reception at the track.

“Derby Day is 13 races, each about 2 minutes long,” Jordan said. “When you figure out that the gates open at 8 a.m. and the Derby doesn’t start until 6:30 p.m., that’s almost 12 hours [of fans in attendance] and only 26 minutes of action.”

In other words, there’s a lot of time for selfies and other Internet activity.

Bucket list pictures, and race handicapping

Like the Super Bowl and other big sports events, the Kentucky Derby is a “bucket list” type of happening, one made even more receptive thanks to the see-and-be-seen nature of the event.

“People spend a lot of time taking in the things that make [the Derby] special, like hats, fashions, and the singing of ‘My Old Kentucky Home,’ ” Jordan said. “It’s the kind of event where people take pictures and send them to their friends to show them what they’re missing.”

Jordan said that in addition to social media activities, many fans at the track used the new DAS connectivity to try to improve their betting odds, using the Internet to help them handicap the races. (Churchill Downs also has a small amount of public Wi-Fi available only in the main building, used mainly to support an on-site mobile betting application.)

In previous years, fans at the event might not have been able to perform all those wireless functions. If the decision to upgrade the communications was an easy one, the decision to go with a “neutral” third party partner like Mobilitie came from a desire to serve all wireless customers equally. Though wireless carriers like AT&T and Verizon can and do act as lead contractors for many DAS deployments, the negotiations to get each carrier what it wants in terms of antenna deployments and other technology needs can get complex in a hurry.

“We know how to put on a great event,” said Jordan of the Derby’s owners and operators. For communications, Jordan said, the track sought out a firm with an equal amount of smarts and reputation in their field.

“We wanted to find someone who was an expert in the conversations [with carriers],” Jordan said, “and to make sure our system could cover as many of our fans as possible.”

Old buildings a challenge for new technology

Among the challenges encountered by Mobilitie at Churchill Downs were some structures dating back to the 1890s, including the iconic twin spires structure, which Jordan said dates back to 1895.

“Some areas have a lot of concrete and steel, and then there are seating areas open to the outside with enclosed dining rooms behind them,” Jordan said. “There were a lot of things for Mobilitie to design around, and it wasn’t easy for them. But walking around the facility, you don’t see a lot of large antennas. Everything is pretty discreet. We’re very pleased with Mobilitie. It was a great win for us.”

AT&T sees massive traffic on new Kentucky Derby DAS deployment from Mobilitie

Churchill DownsWe’ve got a more in-depth review of the wireless experience at the Kentucky Derby on the way, but we did want to share with our readers the somewhat amazing stats from the AT&T wireless network over the weekend of racing at Churchill Downs. With a new DAS from Mobilitie in place, AT&T said it saw a total of 2 Terabytes of traffic over the weekend and a stunning total of 180 Gigabytes of traffic during the peak hour of 5-6 p.m. EDT, just before the 140th Kentucky Derby race went off on May 3.

According to a press release, that peak hour of wireless traffic was the most ever for AT&T at any in-venue event, including the various Super Bowls. What’s even more impressive is that the new DAS also hosted traffic from Verizon Wireless, which did not provide any statistics from the event; however, it’s a good guess that Verizon customers among the 260,000 attendees during race weekend were doing the same things AT&T customers were, taking pictures and sending messages from one of the pure “bucket list” events in sports.

Like we said, more details on the network deployment and challenges at Churchill Downs, in our next Stadium Tech Report. Stay tuned!

Stadium Tech Report: Utah Jazz, Boingo Wireless bring DAS to Energy Solutions Arena

Energy Solutions Arena. Credit, all photos: Utah Jazz

Energy Solutions Arena. Credit, all photos: Utah Jazz

With a big new video board added this year, the Energy Solutions Arena had taken a signficant step toward improving the experience for fans at Utah Jazz basketball games. But when those fans went to look at the small screens in their hands, they often experienced frustration.

“It’s really important to our fans to be able to text or post something to Facebook,” said Jamie Galileo, vice president for facilities at the Energy Solutions Arena in Salt Lake City, Utah. But without Wi-Fi or a modern distributed antenna system (DAS) in the building, sometimes fans were left disconnected.

“Even our employees were able to watch the [cellular] service degrade quite sharply during games,” Galileo said. Next year, that shouldn’t be a problem thanks to the new neutral-host DAS currently being installed at Energy Solutions Arena by Boingo Wireless. With the ability to support all major carriers and their new 4G LTE technology, the new DAS should significantly improve connectivity at Jazz games and other events inside the 19,911-seat arena.

DAS first, Wi-Fi maybe later

Opened in 1991 (known then as the Delta Center), the Jazz’s home is one of the few NBA arenas without advanced wireless connectivity. One of only six NBA arenas without fan-facing Wi-Fi, Energy Solutions Arena is still studying Wi-Fi deployment, Galileo said.

Jamie Galileo

Jamie Galileo

Wi-Fi is still “something to look at,” Galileo said. “We want to first get past [installing] the DAS, and see what that does for the fan base.”

Through this past NBA season, fans at the arena had limited enhanced connectivity. According to Galileo, the building only had small DAS-style infrastructures for Verizon Wireless and T-Mobile customers, and an older system for AT&T customers that was installed prior to the 2002 Winter Olympics, when the arena was used for figure skating and short-track speed skating competition.

“There was some legacy DAS equipment [in the building],” Galileo said. “But you wouldn’t consider it a DAS by today’s technology.”

Neutral host was key

After what Galileo called an extensive RFP and review, Boingo Wireless was chosen for the DAS deployment, which is scheduled to be ready for the start of the next NBA season, beginning in the fall. According to Galileo, Boingo’s ability to be a neutral host was extremely attractive to Energy Solutions Arena.

“Neutral was the key word,” Galileo said. “We did not want to have every carrier have its own antennas. One set of antennas is much easier to blend in.”

Inside shot of Energy Solutions Arena, with big new video board

Inside shot of Energy Solutions Arena, with big new video board

Doug Lodder, vice president of business development at Boingo Wireless, said Energy Solutions Arena has good cabling runs, which should make deployment of the DAS a fairly straightforward task.

“The bowl is always challenging,” said Lodder, who added that Energy Solutions Arena’s somewhat vertical shape of its main seating area might mean a little extra deployment work for Boingo engineers. And while no major carriers have yet signed deals to be on the DAS, Lodder is confident that most will want to have a presence in the building and will sign on before the Jazz start playing again.

“The demand is very high to be in a tier 1 NBA building,” Lodder said.

Galileo said that the arena has plenty of room for the DAS head end equipment, a space requirement that is sometimes challenging for older structures.

“We are on 3.5 acres of a 10-acre site,” Galileo said, “so we have some auxiliary areas. We won’t have to build out additional [head end] space.”

Between Jazz games and other events, Galileo said that Energy Solutions Arena typically has guests in the building 100 times a year. It is also a regular host to NCAA basketball tournaments, and Galileo said the new connectivity should help with future bids.

“It doesn’t hurt to have a DAS,” he said.

AT&T running ads about DAS in stadiums… MSR approves

Sure this is an AT&T ad. But an ad about putting in DAS at a stadium? Who would run a copy of this except us? Enjoy all you antenna geeks out there. And… there’s gotta be work for the fan, maybe conduit work? C’mon DAS dudes, let me in on the fun!

You had me at “Do you know how to optimize a 9-beam multi-beam antenna system?”

Stadium Tech Report: Verizon, AT&T DAS upgrades at MetLife Stadium await Super Bowl Sunday

Verizon branded gate at MetLife Stadium. Credit: Verizon Wireless

Verizon branded gate at MetLife Stadium. Credit: Verizon Wireless

Super Bowl foes the Seattle Seahawks and the Denver Broncos have only had two weeks to prepare for each other. But Verizon Wireless and AT&T have been preparing for the roman-numeral date at MetLife Stadium for more than a year. Will the Verizon Wi-Fi and DAS, and AT&T’s separate DAS be able to handle the wireless needs of the fans at the NFL’s biggest game? Tune in Sunday to see!

We might be one of the only news outlets who care more about the wireless networking at Super Bowls than the game itself, but for many in the stadium tech industry the biggest single game in America’s most popular sport is always somewhat of a wireless watershed. Perhaps at no other event do attendees spend so much time shooting selfies and posting them as they do from the site of Super Bowl Sunday. Even in the expected cold, it should be no different this week at Super Bowl XLVIII at MetLife Stadium in New Jersey, where game time is scheduled for around 6:30 p.m. Eastern.

But well before that, fans will be testing the Verizon-built Wi-Fi network and both of the big carriers’ DAS deployments inside MetLife. According to reps from both companies that we spoke with last week, the carriers are ready.

Verizon spokesman David Samberg told us via email last week that an upgrade of the Verizon DAS in MetLife this past season means there are now more than 500 DAS antennas inside the facility. See some of the photos provided by Verizon that show the clever hiding spots Verizon engineers have found over the last 18 months as they’ve added capacity to a network built just a few years ago.

AT&T has also put in a brand-new DAS over the past year, with another 500-plus antennas of its own. “For the last year or so we’ve been working on our pre-game and game day network playbook in an effort to provide the best possible wireless experience for our customers,” said Michael Maus, assistant vice president of network services at AT&T, via email. “In anticipation of the huge volume of data and voice usage expected [for the Super Bowl], we’ve built a new state of the art antenna system inside the stadium, we’re rolling in portable cell sites both at the stadium, and to support the tailgate areas, and we’ve augmented coverage in New York City to support the activities there.”

Rooftop DAS equipment. Credit: Verizon Wireless

Rooftop DAS equipment. Credit: Verizon Wireless

For Verizon’s own customers, 4G LTE capacity at MetLife has been quadrupled since last year, according to Samberg, who said that all stadium upgrades were completed by October, giving Verizon multiple chances to test its system against crowds at New York Giants and New York Jets home games. So far, Samberg said, so good.

More traffic than last year’s Super Bowl already

While yours truly opined earlier this year that this year’s Super Bowl might not set a wireless traffic record, we didn’t take into account some simple numbers — mainly, that MetLife Stadium’s official capacity of 82,566 is bigger than the Superdome’s 72,003. So, even if it’s cold, having 10,000 more people on hand probably means more bandwidth consumed, even if this year’s game doesn’t have a power blackout in the second half. (And even if it does, Samberg said the network shouldn’t go down since Verizon has backup power supplies on hand.)

Find the DAS antenna! Credit: Verizon Wireless

Find the DAS antenna! Credit: Verizon Wireless

Our only problem with record wireless numbers from Sunday’s game is that we probably won’t ever see an actual number, since Verizon historically shies away from providing a score. Instead it just issues press releases saying things like “way more traffic this year than last!” and then expects us all to believe that without numbers. The good news for fans at the game is that the in-stadium Wi-Fi network, also built by Verizon, is free and open to customers of all carriers, or basically anyone with a device that has a Wi-Fi chip. But Verizon, like big competitor AT&T, has been beefing up its DAS installations significantly because most people try cellular first, even at stadiums, before instructing their phones to find a Wi-Fi network. AT&T, to its credit, usually does deliver a wireless scorecard quickly after big events. So at least from AT&T’s perspective we should find out if this year’s game sets another record.

Aside from the stadium improvements, Verizon will be showing a demonstration of a technology this week that could make DAS more of a competitor to Wi-Fi on the high bandwidth side of things. Called LTE multicast, the technology basically establishes set channels for LTE devices that will “broadcast” video, like a TV channel. (This idea is similar to the StadiumVision Mobile technology Cisco uses at stadiums like Barclays Center.) Theoretically, LTE multicast could let fans use a cellular connection to view multiple video streams, something you would need to use Wi-Fi for it to have any chance of working. But the multicast demo won’t take place at MetLife, but instead at Bryant Park in Manhattan this week. If you are in the city, check out the demo and let us know what you think.

No NFL Mobile at MetLife

And here’s something else you won’t be able to use at MetLife during the Super Bowl: Verizon’s own NFL Mobile app, which outside the stadium will be the only smartphone platform you’ll be able to watch the game on. (The Fox streaming site and app will only work with tablets and desktops or laptops, per the league’s rights agreement with Verizon.) Next year, the rights for NFL Mobile will change and if the local game (like, say, the Super Bowl) is on TV, you’ll be able to use NFL Mobile to watch it even if you’re at the stadium. But not this year! (To give one answer as to why, if you are at the game, you might want to watch it on your cell phone, we say: Bathroom or beer lines!)

More stadium infrastructure photos below:

AT&T DAS antennas at MetLife. Credit: AT&T

AT&T DAS antennas at MetLife. Credit: AT&T

AT&T's new head-end building at MetLife, where its DAS gear is housed. Credit: AT&T

AT&T’s new head-end building at MetLife, where its DAS gear is housed. Credit: AT&T

Inside the AT&T head-end building at MetLife. Cables! Credit: AT&T

Inside the AT&T head-end building at MetLife. Cables! Credit: AT&T

Stadium Tech Report: Boingo, AT&T answer call for more DAS bandwidth at Chicago’s Soldier Field

Soldier FieldWhether or not the Chicago Bears will beat the Green Bay Packers this Sunday is a question still answerable only by higher deities. But if those powers that be decide to unleash weather conditions of biblical proportions again, at the very least the Soldier Field folks can be confident that their in-stadium network will stand up to the challenge.

Earlier this year, tornado weather hit Soldier Field and caused an unusual 2-hour delay during the Nov. 17 Bears-Ravens game. While the deluge was too dangerous for football, for Doug Lodder and Luca Serra, it was a big test to see if the stadium wireless network upgrades they’d recently helped install could stand up to a huge, unexpected traffic surge.

“We thought we’d probably see some good [network] usage with the weather delay and tornados,” said Lodder, vice president of business development at Boingo Wireless, which recently concluded a big network upgrade at the home of the NFL’s Chicago Bears.

“We saw double the usual traffic from both AT&T and Verizon customers during the 2-hour delay,” said Serra, director of sponsorship and media for SMG, the outside entity that manages Soldier Field for its owners, the Chicago Park District. “During that break, we didn’t hear a single complaint [about the wireless network] from the media or the fans. It was a great test — not a desired one in any way, shape or form, but a great test.”

Upgrades needed at ‘oldest’ big stadium Wi-Fi network

Luca Serra, SMG

Luca Serra, SMG

Even though you can’t really plan or design for surges that might happen during extreme situations, Serra and Lodder and others running the Soldier Field network knew last year that an overall upgrade was needed. Home of one of the earliest (if not the first) big-stadium DAS and Wi-Fi networks, Soldier Field was showing 80 percent year-over-year traffic growth on the Wi-Fi network, Lodder said in a recent joint phone interview with Serra.

Along with DAS partner AT&T, SMG and Boingo spent last year adding more DAS capacity so that Solider Field now has 250 DAS antennas to go along an undisclosed number of new Wi-Fi APs. Though AT&T did some specific upgrades to its DAS presence at Soldier Field (including adding support for its new 4G LTE network), all the major wireless carriers are represented in the DAS via the neutral hosting of Boingo.

Why did the firms decide Soldier Field needed an upgrade? While Lodder said its DAS, which first started operating in 2011, was “pretty cutting edge at the time,” during the past couple years, data usage at Soldier Field has skyrocketed.

“Everyone has a device, and there’s a lot more social engagement and sharing,” Lodder said. “We’ve seen a huge increase in data use.”

“With this large increase in use, we said ‘let’s address the issue’ and not let it get to a point where it’s a problem,” Serra said.

Moving fast and finding space

DAS gear at Soldier Field. Credit: Boingo Wireless

DAS gear at Soldier Field. Credit: SMG/Soldier Field

Both Serra’s and Lodder’s teams were familiar with the need to balance performance and aesthetics at the venerable stadium perched on the edge of Lake Michigan. Since the network was put in after Soldier Field’s extensive renovation was finished in 2003, special attention needed to be paid to placements of Wi-Fi and DAS antennas.

“It wasn’t just bolting APs on,” Serra said. “We had to be very protective of the aesthetics of the building.”

For Lodder, that sometimes means that the best placement isn’t always available.

“There’s always challenges with architecture at stadiums,” Lodder said. “In a perfect world, there might be a certain place you want to put an antenna. In the real world, you can almost always count on changing that 20 percent of the time.”

Doug Lodder, Boingo

Doug Lodder, Boingo

A bigger construction concern of late is where to put the extensive amount of back-end telco gear needed to support the DAS infrastructure and upgrades. “Carrier base stations need several thousand square feet of space, and those need to be rooms with connectivity and power,” Lodder said. At Solider Field, Serra said some unfinished space north of the stadium proper was converted into a DAS equipment room, since there wasn’t enough available space in the main building. Lodder said for the recent upgrade, additional power upgrades were installed as well.

Still, even with all the challenges the Boingo, AT&T and SMG teams were able to go from design to finished construction of the recent upgrade in 45 days this past summer, in part thanks to some interesting work schedules.

“After one preseason [football] game we had scaffolding going up at night, with three 24-hour shifts,” Lodder said. “It was a pretty accelerated build.”

There is no such thing as free Wi-Fi

One other interesting twist to the Boingo Wi-Fi network at Soldier Field is its cost — the company charges a $1.99 per day fee for Wi-Fi use, or you can purchase a regular Boingo monthly plan for $7.95 which will also provide access at Soldier Field.

Wi-Fi antennas hidden under stands at Soldier Field. Credit: SMG/Soldier Field

Wi-Fi antennas hidden under stands at Soldier Field. Credit: SMG/Soldier Field

While many if not most professional teams are making Wi-Fi services free to paying ticketholders, Serra and Lodder said that the Bears aren’t.

“The Bears are a tenant of the Park District, and no one [from the team] has ever said ‘we’re going to pay to turn it [Wi-Fi] on for free,’ ” Serra said. Lodder noted that in the end, someone somehow has to pay the cost of connectivity.

“Someone’s always got to pay for it — Wi-Fi isn’t free,” Lodder said. “Some teams may feel they’ve made enough on the tickets to offer [Wi-Fi] free, while in other cases users pay either with money or with eyeballs, by viewing some ads.”

While the NFL itself is still somewhat vague on its league-wide plans for in-stadium Wi-Fi — other than saying that teams should provide it — Serra and Lodder are confident that Soldier Field will be at the top of the chart when it comes to network performance.

“If and when they [the NFL] roll out standards for network performance, we’re going to exceed them,” Lodder said.