Super DAS: Crown Castle’s neutral host infrastructure aims to keep Super Bowl XLIX fans connected, inside and outside the stadium

Editor’s note: This story is part 1 of a series of profiles of the providers of the extensive Distributed Antenna System (DAS) deployment for Super Bowl XLIX at and around the University of Phoenix Stadium in Glendale, Ariz. and other parts of the Phoenix city area as well. Stay tuned all week as we highlight how DAS will keep Super Bowl fans connected, no matter where they roam in and around Phoenix and Glendale this week and weekend.

University of Phoenix Stadium getting its Super Bowl on. (Click any photo for a larger image) Credit all photos: Paul Kapustka, MSR

University of Phoenix Stadium getting its Super Bowl on. (Click any photo for a larger image) Credit all photos: Paul Kapustka, MSR

When is a big air-fan vent not an air-fan vent? When it’s a fake vent covering a hidden cellular antenna, put there to keep people from noticing the technology that’s keeping their cell phones connected. Before kickoff at Super Bowl XLIX Feb. 1 in Glendale, Ariz., many fans outside the University of Phoenix Stadium will walk right by a faux vent and its sheltered equipment, never knowing the attention to detail that goes into a major-venue Distributed Antenna System (DAS) deployment.

But to stadium technology connectivity professionals, such leaps of aesthetic deception are just part of a day’s, or perhaps month’s, DAS deployment work. For neutral host DAS provider Crown Castle, the fake vents on the shell of the University of Phoenix Stadium — and the powerful antennas behind — are just one part of a massive project to ensure there is excellent mobile-device connectivity both inside and outside the Super Bowl stadium, so that fans never get a dropped signal anywhere between the parking lot and their prized seat.

During a recent press tour, a small team of Crown Castle employees showed off some of the upgraded DAS network deployed at the University of Phoenix Stadium as well as in the surrounding Westgate Sports and Entertainment District, a sort of open-air mall that stretches from the UoP Stadium past numerous attached restaurants and stores, also encompassing the Gila River Arena, home of the Phoenix Coyotes of the NHL. Over the past year or so, Crown Castle has been upgrading the DAS inside and outside the arena, throughout the mall areas as well as into the huge parking lots that surround it and the football stadium, bringing connectivity to phones being used by customers from all the four major U.S. wireless carriers.

Since the mall and all its food outlets are conveniently located a short stroll from the stadium, it’s a good bet that a large portion of the Super Bowl crowd will spend time wandering around the Westgate area before and after the big game. Thanks to Crown Castle’s efforts, there shouldn’t be many connectivity problems, as antenna deployments on light poles, building rooftops and — yes, even behind fake vents — should be able to keep devices on the cellular networks without a glitch.

Game day connectivity starts in the parking lot

Since we couldn’t actually spend much time wandering around the stadium itself — even three weeks before the big game, the facility was already on NFL security lockdown — most of the Crown Castle tour consisted of walking around the Westgate mall/neighborhood, hearing about the various methods Crown Castle used to locate the necessary DAS antennas. In all, there are five separate DAS networks Crown Castle is responsible for in the area around the stadium: The football stadium itself; the Gila River Arena (which we will profile in an upcoming feature on hockey stadiums); the Westgate shops and restaurants; the nearby Renaissance Hotel; and the surrounding parking lots.

Parking lot light poles, Westgate entertainment district. Can you spot the DAS?

Parking lot light poles, Westgate entertainment district. Can you spot the DAS?

The curious start of the tour in a far-flung parking lot made sense when we found ourselves next to a small DAS equipment box and a light pole with multiple antennas (which had not yet been covered with their final aesthetic sheaths). Aaron Lamoureux, program manager for Crown Castle’s small cell solutions, served as tour guide, and said that for the Westgate area alone there were 18 individual node locations, with about 52 antennas total. Some were located on light poles, some on rooftops, and some along walkways between buildings, to conquer the unique RF characteristics of the open-air/large building outdoor mallish area that is Westgate. (See photos for DAS geek views)

For the University of Phoenix Stadium itself, Crown Castle deployed 228 DAS antennas inside (more on this in an upcoming profile) and at 21 different locations outside the stadium, 13 of those on parking lot poles and 8 mounted on the building itself. Why building-mounted antennas? If you’ve never been there, the University of Phoenix Stadium has a large plaza area on one side, which is used for pre-game activities like rallies, bands and other walk-up amenities where fans gather before entering. The challenge for Crown Castle was finding places to deploy antennas at a low enough height to cover crowds of people standing in one location. While some parts of the building allowed for regular antenna placements, a big part of the plaza faces part of the stadium wall that is a sheer sheet, with no aesthetic place to mount a DAS antenna — unless you add a fake vent or two to the existing design, that is.

Keeping everyone happy is part of the neutral host job

See the big air vents? Nobody would tell us which ones were 'faux vents,' there to hide DAS antennas

See the big air vents? Nobody would tell us which ones were ‘faux vents,’ there to hide DAS antennas

To people outside the industry it might seem silly to go to such lengths just to keep folks from noticing antennas, but anyone who’s deployed a network for a detail-oriented building owner knows why aesthetics are important. That’s why you paint antenna enclosures to match the surrounding walls, or build sheaths to keep wires and other obvious gear out of main sight. It’s part of the art of wireless network deployment, and not as simple as it sounds. Experience counts.

The complex owner and operator relationships involved in the stadium and surrounding-area DAS also seem tailor-made for a big, experienced provider like Crown Castle, which has a long history of deploying and operating multiple-tenant networks. With five different landlords and four different carriers, being the neutral DAS host for this year’s Super Bowl is a task with many moving parts; but, as Mike Kavanagh, president of sales for Crown Castle’s small cell solutions, said, “We understand how to run networks, how to manage them and deal with carriers. It’s high touch and very fluid. But we know that business.”

COMING UP NEXT: What’s inside the network inside the stadium.

MORE PICTURES BELOW! (Click on any picture for a larger image.)

Sky Harbor Airport: Ready for Super XLIX

Sky Harbor Airport: Ready for Super XLIX

Verizon's NFL Mobile ads were in airport walkways well before the Big Game

Verizon’s NFL Mobile ads were in airport walkways well before the Big Game

If you stumble off the escalator, Bud Light is there to catch you

If you stumble off the escalator, Bud Light is there to catch you

The Westgate uber-mall should see a lot of fan activity (and connectivity) on game day

The Westgate uber-mall should see a lot of fan activity (and connectivity) on game day

Here's the official Super Bowl replay HQ (actually a place with DAS antennas on the roof that you can't see)

Here’s the official Super Bowl replay HQ (actually a place with DAS antennas on the roof that you can’t see)

Mama Gina's will offer you pizza and DAS on the roof

Mama Gina’s will offer you pizza and DAS on the roof

More DAS antennas, on a Westgate walkway

More DAS antennas, on a Westgate walkway

Outside UoP Stadium, where the architecture allows for DAS antenna placement

Outside UoP Stadium, where the architecture allows for DAS antenna placement

Close-up of that placement. Still pretty well hidden.

Close-up of that placement. Still pretty well hidden.

Parking lot light mounts. These will have sheaths by Super Sunday.

Parking lot light mounts. These will have sheaths by Super Sunday.

Here's the remote equipment box that powers the light pole antennas. Also scheduled for more concealment.

Here’s the remote equipment box that powers the light pole antennas. Also scheduled for more concealment.

Every artist leaves a signature...

Every artist leaves a signature…

YinzCam’s Super Bowl stadium app will have instant replays, Super Bowl commericals, stadium maps and more

Screen shot of Super Bowl app for this year's game.

Screen shot of Super Bowl app for this year’s game.

We’ve been waiting for official word on what the YinzCam-developed app for the Super Bowl will look like, and though there’s no press release the page where we are guessing it will eventually be available is offering some details, like the availability of instant replays from different camera angles, video of Super Bowl commercials, and stadium maps.

On the Seahawks.com site we found a good how-to story for fans going to the game, which included a link to this page, where we are guessing the Super Bowl stadium app will be available for download. Here is the boilerplate:

New for Super Bowl XLIX, the Super Bowl Stadium App Presented by Verizon aims to take the fan experience inside University of Phoenix Stadium to the next level. Features that will enhance Super Bowl ticketholders’ experiences include exclusive in-stadium video content such as Super Bowl commercials and replays from four different camera angles, stadium seating and concession maps, once-in-a-lifetime gameday opportunities visible only to fans inside the stadium and the option to receive up-to-the-minute gameday notifications. Available on iOS, Android and Windows. Goes Live 23rd January 2015

(Looks like the app is already available in the App Store and in Google Play, but nothing is live; we downloaded the app and the only three buttons available, for highlights, commercials and memories, all say they will be available on Feb. 1 at the stadium, so no idea what the “goes live” on the splash page above means yet.)

Screen Shot 2015-01-23 at 11.40.41 PMYinzCam founder and CEO Priya Narasimhan had told us earlier this year that a Super Bowl app was in the making, and apparently it will contain features found in some of the latest YinzCam app deployments, such as the Seattle Seahawks’ new stadium app, which has multiple camera angle replays. The Super Bowl app is different from the Arizona Cardinals’ regular stadium app, which was also built by YinzCam, which also features instant replays.

We were able to download the app for iPhone (it’s free) and apparently you will need to be connected to the stadium Wi-Fi (which has the clever SSID of “Stadium WiFi”) in order to view highlights and other video options.

The good thing for fans at the big game, there will be plenty of networking horsepower to keep the app running, no matter where you are. If you’re inside the stadium there is a new Wi-Fi network and a refurbished DAS deployment to keep fans connected; stay tuned next week for our big breakdown of DAS deployments and carrier plans to keep the Super Bowl crowds super-connected.

NBC will live-stream Super Bowl online for tablets, computers; Verizon NFL Mobile will carry for smartphones

Screen shot 2015-01-20 at 10.37.03 AMIf for some reason you are banned from the living room couch for the Super Bowl, NBC has you covered — the network will be streaming the game live online, along with hours of pregame, postgame and halftime festivities — for anyone with an Internet connection and a laptop, PC or tablet.

According to a press release sent out Monday NBC said it will also not require viewers to have a qualifying cable or satellite contract to view the game, thereby eliminating the often annoying login process that accompanies many other online live sports streaming activities. You will, of course, be subject to multiple NBC advertisements but hey — a small price to pay for the convenience of being able to watch the game online.

(Mobile Sports Report is old enough to remember Super Bowl parties where we rented extra TV sets for the bathrooms and the kitchen; now you can just use in-house Wi-Fi and a tablet or laptop, perhaps with a splash guard.)

On the cellular side, if you are stuck somewhere and want to watch on your phone, the only option is having a Verizon contract and using the NFL Mobile app. If you are a More Everything customer the live viewing of the game is free, if not you must pay a $5 monthly charge for the one day in February that you will need premium access. (Pro tip for Verizon customers — don’t forget to cancel that premium access charge the day after the game, since Verizon will happily charge you $5 a month all summer long even though there are no NFL games during that time.)

It’s an easy guess so we will predict right now that this year’s Super Bowl will set new online records for most Internet viewers — without fail this has happened every year since the networks and the league started making the game available online. According to the NBC folks the online stream will have some handy extras, like the DVR feature that lets you scroll back to important plays, as well as additional camera angles and in-game stats.

Now our next dream is for the Shield and its broadcasters to follow ESPN’s lead on the college championships and provide online “Megacast” options for alternate announcers. A man can hope.

College championship game at AT&T Stadium breaks 6 Terabyte wireless data mark, with almost 5 TB of Wi-Fi traffic

AT&T Stadium before the college football playoff championship game. (Click on any photo for larger image) Credit all photos: Paul Kapustka, MSR

AT&T Stadium before the college football playoff championship game. (Click on any photo for larger image) Credit all photos: Paul Kapustka, MSR

Not only did Monday night’s College Football Playoff championship game crown a new new national title team — it also broke the unofficial record for most wireless traffic at a single sporting event, with more than 6 terabytes of data used by the 85,689 fans in attendance at AT&T Stadium in Arlington, Texas.

John Winborn, chief information officer for the Dallas Cowboys, said the AT&T-hosted Wi-Fi network at AT&T Stadium carried 4.93 TB of traffic during Monday’s game between Ohio State and Oregon, a far higher total than we’ve ever heard of before for a single-game, single-venue event. AT&T cellular customers, Winborn said, used an additional 1.41 TB of wireless data on the stadium DAS network, resulting in a measured total of 6.34 TB of traffic. The real total is likely another terabyte or two higher, since these figures don’t include any traffic from other carriers (Verizon, Sprint, T-Mobile) carried on the AT&T-neutral host DAS. (Other carrier reps, please feel free to send us your data totals as well!)

The national championship numbers blew away the data traffic totals from last year’s Super Bowl, and also eclipsed the previous high-water Wi-Fi mark we knew of, the 3.3 TB number set by the San Francisco 49ers during the opening game of the season at their new Levi’s Stadium facility. Since we’ve not heard of any other event even coming close, we’ll crown AT&T Stadium and the college playoff championship as the new top dog in the wireless-data consumption arena, at least for now.

University of Phoenix Stadium, already with Super Bowl prep under way

University of Phoenix Stadium, already with Super Bowl prep under way

Coincidentally, MSR on Tuesday was touring the University of Phoenix Stadium and the surrounding Westgate entertainment district, which is in the process of getting the final touches on a new complex-wide DAS installed by Crown Castle. The new DAS includes antennas on buildings and railings around the restaurants and shops of the mall-like Westgate complex, as well as inside and outside the UoP Stadium. (We’ll have a full report soon on the new DAS installs, including antennas behind fake air-vent fans on the outside of the football stadium to help handle pre-game crowds).

The University of Phoenix Stadium also had its entire Wi-Fi network ripped and replaced this season, in order to better serve the wireless appetites coming for the big game on Feb. 1. At AT&T Stadium on Monday we learned that the network there had almost 300 new Wi-Fi access points and a number of new DAS antennas installed since Thanksgiving, in anticipation of a big traffic event Monday night. Our exclusive on-the-scene tests of the Wi-Fi and DAS network found no glitches or holes in coverage, which is probably part of the reason why so many people used so much data.

UPDATE: Here is the official press release from AT&T, which basically says the same thing our post does.

All NFL playoff games available online; Verizon only for smartphone watching

vzn_playoffWe’ve come a long way from the days when it was a struggle to even find NFL live action online. This season, all NFL playoff games, including the Super Bowl, will be available for online viewing, via a desktop computer, laptop or tablet, no matter which network is carrying the games. The league has even created a handy single web page to find instant access to the live streams, no small matter since network pages usually make you jump through several clicks to find the actual live stream.

Depending upon the broadcast network, you may need to have a qualifying cable or satellite contract to view the games. For this morning’s CBS game there was no confirmation process but for the afternoon Fox game I needed to submit cable provider info. Let me know what you see.

However, if you want to watch playoff games on your smartphone, your only option is to be a Verizon Wireless customer since that provider is the exclusive NFL live action host for smartphone devices. To view the live action, Verizon customers need to have either a More Everything plan or pay the $5 monthly premium fee for the NFL Mobile application. To be clear, customers of any wireless provider can download the NFL Mobile app to get all the stats and archived video it provides, but only Verizon customers can purchase access to live action on phone platforms.

UPDATE: Congrats to FOX for having the first playoff online broadcast fail:

Screen Shot 2015-01-04 at 3.01.07 PM

No word from Fox yet on when its stream will come back, or why it was down.

Stadium Tech Report: Arizona Cardinals get stadium ready for Super Bowl with Wi-Fi upgrade

University of Phoenix Stadium. Credit all photos: Arizona Cardinals.

University of Phoenix Stadium. Credit all photos: Arizona Cardinals. (click on photos for larger image)

As they get set to host their second Super Bowl this February, the IT team at the University of Phoenix Stadium in Glendale, Ariz., knows now what they didn’t know then: The big game requires a big wireless network. Bigger than you think.

“It’s funny to look back now on that first Super Bowl,” said Mark Feller, vice president of information technology for the Arizona Cardinals, speaking of Roman numeral game XLII, held in the barely 2-year-old facility on Feb. 3, 2008. With a couple Fiesta Bowls and one BCS championship game (2007) under his belt in a facility that opened with Wi-Fi and DAS, Feller said he and his team “thought we had a good handle” on what kind of network was required for a Super Bowl crowd.

The NFL, he said, begged to differ. Those college games might have been big, but the Super Bowl was bigger.

“We had the Fiesta Bowl that year at night, and when the game was over there were people from the NFL there wanting to know when they could set up,” said Feller in a recent phone interview. “This year, we’re much better prepared. We know what the water temperature is this time.”

Rip and replace, with more and better gear

Wi-Fi railing antennas

Wi-Fi railing antennas

For Super Bowl XLIX, scheduled to take place in Glendale on Feb. 1, 2015, Feller and his team have not just tuned up their network — they have done a full rip and replace of the Wi-Fi system, installing new Cisco gear from back end to front, in order to support a wireless game-day demand that is historically second to none. Integrator CDW has led the Wi-Fi effort and Daktronics and ProSound did the installation of new video screens, and neutral host Crown Castle has overseen a revamp of the DAS system, again with more antennas added to bolster coverage. In all, there has been more than $8 million in wireless improvements before this year, Feller said, as well as another $10 million for two new video boards that are each three times larger than what was there before.

“The last three or four years there have been things we knew we needed to improve [before the Super Bowl],” Feller said. After extensive work with the NFL’s technical team — this time well before the Fiesta Bowl — Feller oversaw a “top to bottom” refurbishment that included replacing core Cisco networking gear with newer gear, and new and more Wi-Fi access points that now total somewhere north of the 750 mark, with some more to be added before the big game. The new network, which was in place for the start of the current NFL season, has undergone testing by CDW at each home game, Feller said. CDW also plans to expand the network outside the stadium before the Super Bowl, in part to handle the extra events that take place not just on game day but in the days leading up to the game.

“The plan is to install more [coverage] outside, in the plaza areas,” Feller said.

When it opened in 2006, the $455 million University of Phoenix Stadium was one of the first with full-bowl Wi-Fi, using Cisco gear from the inside out. “Cisco was in here before they called it [their solution] ‘connected stadium’,” Feller said. From core network switches to firewalls to edge switches, this year there is all new Cisco gear in the venue, as well as new 3700 series APs, with panel antennas and antennas in handrails.

“Handrail [antennas] are sometimes a bit of a challenge, because you need to drill through concrete that’s 40 feet up in the air, behind another ceiling,” said Feller, describing one particular design challenge. Another one was mounting antennas on drop rods from the catwalks below the stadium’s retractable roof, to serve the upper-area seating. There are also some new Wi-Fi APs on the front row of the seating bowl, pointing up into the crowd.

“It was a fun project,” Feller said.

Stadium with roof open

Stadium with roof open

All on board for the DAS

The upgrade for the stadium’s DAS, led by Crown Castle, was just finished a few weeks ago, Feller said, and included more coverage outside the stadium as well, with antennas placed on light poles and on the stadium’s shell.

“Crown Castle did a great job of managing the carriers” on what is a 48-sector DAS, Feller said. “It [the upgrade] really required a lot of creative thinking from their engineers.”

Since the stadium was originally designed with wireless in mind, Feller and his team didn’t need to build new head end room for the DAS upgrades. “But I wouldn’t say we have plenty of space left,” he added. “We’ve got a lot of new equipment.”

Though all the major carriers are expected to be on the DAS by the big game, league partner Verizon Wireless should have some special projects up its sleeve for the big game, including another demonstration of its LTE Broadcast technology, which optimizes things like live video over LTE cellular links.

New Cardinals app a preview of Super Bowl version?

The Cardinals also had a new version of the game-day team app for this season, built by stadium-app leader YinzCam. According to Feller the new app supports three different live video feeds, as well as instant replays.

Wi-Fi antenna on railing

Wi-Fi antenna on railing

“It’s really cool to have that ability to watch things like a touchdown pass at the end of the game,” Feller said. And while no details have yet been revealed, in an interview with NFL CIO Michelle McKenna-Doyle earlier this year MSR learned that the league and YinzCam are working on a Super Bowl app with its own new bells and whistles. (Stay tuned for more info on the Super Bowl app.)

In addition to two more regular-season home games in December, the University of Phoenix Stadium will have at least a couple more dry runs to help test the network, during the Dec. 31 Fiesta Bowl and during the NFL’s Pro Bowl, scheduled for Jan. 25. And though the Cardnials lost to the Atlanta Falcons Sunday, at 9-3 they are still tied with the Green Bay Packers for the best record in the NFC, something that has the Phoenix faithful optimistic about the postseason.

“We’re going to get some more test runs, on New Year’s Eve and during the Pro Bowl,” Feller said. “And maybe some home playoff games as well!”

(more photos below)

Wi-Fi antenna in roof rafters

Wi-Fi antenna in roof rafters

More antennas in rafters

More antennas in rafters

Wi-Fin antenna under overhang

Wi-Fi antenna under overhang