Stadium Tech Report: Levi’s Stadium network lives up to hype, but team app still needs work

Levi's Stadium from Section 244. All photos: Paul Kapustka, Mobile Sports Report

Levi’s Stadium from Section 244. All photos: Paul Kapustka, Mobile Sports Report

At the very least San Francisco 49ers fans Sunday could take heart in the fact that the wireless network in Levi’s Stadium largely lived up to its advance billing, performing quite well even as the team on the field sputtered and failed to connect. In its first “real” test with an almost-full house on Sunday the Levi’s Wi-Fi and cellular networks seemed to work well throughout the game, delivering solid speed test results from almost every part of the new 68,500-seat facility, even as Colin Kaepernick and the rest of the 49ers were dealt a 34-0 preseason drubbing by Peyton Manning and the visiting Denver Broncos.

And just like the team, the Niners’ stadium technology lineup still has some weak spots that will hopefully be fixed before the regular season home opener on Sept. 14. Among the disappointments Sunday was a no-show by the highly heralded instant replay feature, the crown jewel of the new Levi’s Stadium app. We also experienced some location-connection problems with one of our devices, exposing what we consider a flaw in the Levi’s app, namely an over-reliance on location technologies to enable key parts of the app, like wayfinding and on-site video streaming.

Ticket scanner with Niners visor to block sun

Ticket scanner with Niners visor to block sun

Some other not-so-advanced technology flaws that could use fine-tuning include the volume level on the main stadium public-address and announcing system, which was so loud that it made it a struggle just to talk to the person next to you for long stretches of time. The ticket scanning machines also seemed to have issues working in the bright sunlight, a problem that found a low-tech fix when ticket personnel placed Niners’ visors around the tops of the machines to shade the scanning area. And many concession stands around the stadium were unable to serve guests or could only take cash because the staff operating the stands said they weren’t given access codes to the point-of-sale systems.

Overall, however, the first football game at Levi’s was a success on many levels, including the fantastic sight lines available from most seats and largely incident-free travel and parking operations, with noticed improvements especially on the VTA light rail front that struggled mightily during the stadium’s opening-event soccer game two weekends ago. Most fans also probably got a little weight loss from the no extra-charge sauna situation, thanks to the cloudless day and bright sun that bathed most of the seats in searing heat for long times after the 1 p.m. start.

Smooth start for early VTA riders

What follows here is a somewhat minute-by-minute account of my trip to the game, and my experience with the network and stadium operations on site.

Mtn View lot sign, not in operation at 9:30 a.m.

Mtn View lot sign, not in operation at 9:30 a.m.

Since I wasn’t given press access to the game, Mobile Sports Report attended like a regular fan, purchasing a single ticket through the NFL Ticket Exchange service on the 49ers’ web site. My plan to get to Levi’s from San Mateo was to drive to downtown Mountain View, park there and take VTA the rest of the way. (I didn’t take CalTrain mainly because I didn’t want to have to sync my return schedule with the CalTrain options going northbound on Sunday.)

Though I was somewhat incredulous about having to buy tickets online — VTA said that the ticket machines in Mountain View would be shut down Sunday to keep big lines from forming — upon further review the VTA app was slick and easy to operate and understand. After purchasing a ticket for $6.50 Saturday night I activated it Sunday, and showed it at the gate where they checked boarding passes. For people who didn’t have tickets there was a tent set up where they could buy a pre-loaded Clipper card for $10 good for a day’s worth of VTA riding. There was an abundance of VTA workers on hand, as well as a large and very obvious police presence. As a nice touch there was also a large bank of porta-potties, and behind the trains there were express buses waiting, according to one VTA employee, in case of crowd overloads.

“We learned some lessons from two weeks ago,” he said.

If there was a glitch in the VTA operations it was with the city of Mountain View — though a couple city lots were designated as places where fans could buy all-day parking passes, and there were clear signs to those lots, at 9:30 a.m. those lots were not yet staffed with anyone to pay; MSR found one sign leaning up against a post, waiting to be deployed. Fans could also park in the CalTrain lot for $5, payable via the CalTrain track podium ticket machines.

Fans transferring from CalTrain to VTA at Mtn View station

Fans transferring from CalTrain to VTA at Mtn View station

I boarded the first VTA train to leave for the stadium, along with many fans who had just gotten off CalTrain. The pleasant, air-conditioned trip took just 27 minutes, passing many Silicon Valley company headquarters and one neighborhood with “no parking here” patrols before stopping pretty much right at the Levi’s Stadium entrance. A few steps later I was in the parking lot, and took the first of many Wi-Fi speed tests and got a signal of 29 Mbps download and 23 Mbps upload, a good sign for network operations.

Looking for Wi-Fi, finding lots of it

DAS antenna in "Faithful Mile" area

DAS antenna in “Faithful Mile” area

Once inside the gates — and past the shaded scanners — I started speed testing in earnest, with the two devices I brought with me: A Motorola Droid 4 on Verizon, and a Samsung Galaxy Note 3 on AT&T. While waiting for the main stadium gates to open at 11 a.m. I got the weakest Wi-Fi signals of the day along the “Faithful Mile” area where promotional booths from sponsors kept early arrivers entertained. Wi-Fi on both devices out there only hit 2 to 3 Mbps on the download, while cell signals on both devices were in the 6-7 Mbps download range. Since I could see multiple DAS radios in the area but not any Wi-Fi access points I wasn’t too surprised; but it was an unusual area not to be blanketed with Wi-Fi, especially since there was good access a couple hundred yards away in the parking lots.

After finally entering the stadium proper, I ran into Niners president Paraag Marathe — who said he was “nervously excited,” and looked ready to start sweating in his suit and tie. “We’ve just got to make sure everything works today,” Marathe said, shaking my hand. Then I went up the escalator and saw the “Kezar pub,” an open-air bar filling the top area above the Intel gate. There, draft beers like Shock Top and Goose Island IPA were available for $11, and bottled beers available for $10.25.

As I started walking around the outside concourse I took my first speed test in the stadium and it blew the needle off the edge: 57.92 Mbps download, 41.00 Mbps upload.

A few minutes later on the inside concourse (where most of the concession stands are) I hit 27.85 Mbps/21.34 Mbps, still impressive. Then I tried to launch the app, and — problem. Apparently the device wasn’t connecting because it wouldn’t show my location on the wayfinding app. Luckily, right in front of me was Racquel, one of the “NiNerds,” the team’s new staff of technical experts who are there to help fans make the app work. But Racquel couldn’t solve my problem, even after we both tried turning on all location services, including Bluetooth.

Racquel the NiNerd

Racquel the NiNerd

“I can try to find another NiNerd who might know more about this device,” offered Racquel, who was visibly dismayed at her failure to help solve my problem. Instead, I moved on, hoping that the problem would solve itself later. But it didn’t.

Failure to locate… and other app problems

After downloading the Levi’s app to both devices over the weekend, I noticed that the first item on the app list of functions — Tickets — required me to “sign in” with my “Stadium Ticket Account,” something I didn’t have and didn’t know how to get. I did figure out how to enter my purchased seat location (which I could have used to order food to my seat, or for the express pick-up option), but I could never get the location feature to work on the Samsung device, which kept me from being able to see the live streaming TV option (I kept getting a message that said, “You must be at the stadium to play this video”). I was able to watch the live TV option on the Motorola device, after turing on location services. But for both devices — and, as it turns out, for everyone in the stadium — the final feature on the app, Game Center, where we were supposed to be able to see all the instant replays we could handle, remained labeled “coming soon.”

Finally sitting in my most excellent seat — section 244, row 3, seat 17 — I noticed that the Motorola device could no longer connect to Wi-Fi, even as the Samsung device was hitting marks in the 15-16 Mbps range. I started tweeting about the problem, and instead of a NiNerd coming to help me I got a personal visit from the Levi’s version of a Jedi Master, namely Dan Williams, the team’s vice president of technology. (Never underestimate the power of a complaining tweet!)

Niners VP of technology Dan Williams attempts to fix my Droid 4 Wi-Fi issues (while trying not to laugh at the fact that I actually have and use a Droid 4)

Niners VP of technology Dan Williams attempts to fix my Droid 4 Wi-Fi issues (while trying not to laugh at the fact that I actually have and use a Droid 4)

On my own, I had guessed that the Droid’s inability to connect came from its having only a 2.4 GHz Wi-Fi radio. If you’re not familiar with Wi-Fi networks, the 2.4 GHz band of unlicensed airwaves is what most first-generation Wi-Fi networks used; more recent devices are able to also use the 5 GHz band of unlicensed airwaves, which simply offer more channels and more bandwidth. The iPhone 5s, for example, mainly uses 5 GHz for Wi-Fi, as does my Samsung Note.

In scanning the available Wi-Fi networks, I had also noticed something else that I thought could be gumming up the Droid’s connection — a bunch of personal Wi-Fi hotspots in the immediate area, including several labeled as GoPro cameras. After curiously examining my Droid 4 — and its slide-out keyboard — Williams and a technician from Wi-Fi gear provider Aruba Networks concurred that my device was getting bogged down in the 2.4 GHz mess, and also wasn’t refreshing the available networks list, a device-specific problem.

Bottom line? Levi’s is no country for old phones.

Initial verdict: Wi-Fi and cellular is world class… but app needs work

After staying into the third quarter — and visiting another friend in section 109, where I got another hefty Wi-Fi speed test (24.42/25.39 Mbps) — I followed the lead of many fans and took an early leave of Levi’s, which meant no lines at the VTA trains and just a couple short delays due to track congestion that stretched the return train trip to 40 minutes. Overall, my travel to and from the stadium from San Mateo took just over an hour each way, a happy stat to report.

My initial verdict is that the Wi-Fi and DAS (cellular) networks delivered as promised, with solid speeds all around the stadium every time I checked. It’s no small accomplishment just to deliver such world-class service to such a crowded space, especially in the middle of Silicon Valley. True to its roots, the crowd Sunday was device-happy, with many iPads and GoPros being carried around as video cameras, in addition to all the phones that were in constant use. It’s a tribute to Williams and his staff, as well as the technology suppliers like Aruba, Brocade, Comcast (backbone bandwidth supplier) and DAS Group Professionals, who built the distributed antenna system (DAS) which brings advanced cellular connectivity inside the gates, to have built a solid network that worked well on its first big test.

The team app, however, did not even come close to living up to its advanced billing. To equal the network I think the app needs more advance instructions, especially on the ticketing/registration options as well as on the location services needed to make everything work. And until we see the multiple-camera angle live replays in action, to me the app is an incomplete project. The good news is, the Niners and their technology teams have several weeks to make improvements, including another preseason game Aug. 24 against the San Diego Chargers.

It’d also be helpful for the team to reach out a bit more to the VTA and players like the City of Mountain View, since the VTA site maps and Mountain View’s parking maps are far from what you would call “advanced design.” I think it’s up to the Niners to help pay for improvements to the city and transportation entities’ technology offerings, simply because of the burden placed on those operators by the fans going to Niners’ games. At the very least, more links from football to getting-there operations seems in order, instead of trusting that all parts of the operation will work in sync.

VTA lines going home

VTA lines going home

Stadium Tech Report: Read why we think MLB is the sports digital experience leader

STRQ@_thumbIs there any doubt that when it comes to the digital fan experience, Major League Baseball is in first place? That’s the way we see things, and research we did for our second quarterly long-form STADIUM TECH REPORT issue bears that opinion out. Thanks to a far-sighted strategy that kept league control over all Internet content, and some innovative, forward-thinking technology leaders at several MLB teams, baseball is ahead of all other U.S. sports when it comes to delivering a consistent, enriched fan experience through technology. But will the lead last?

You can find some of the answers to that question in the second issue in our STADIUM TECH REPORT series which you can download for free right here. If you’ve already registered with us, all you need is a username and password; if you’re new to MSR, we just need an email address and title and you’re on your way to the best long-form compilation of research and analysis, as well as in-depth interviews with industry experts in the stadium technology marketplace. We’d also like to thank our Stadium Tech Report sponsors, which for this issue inlcude Crown Castle, SOLiD, Corning, ExteNet Systems and TE Connectivity — without their support, we couldn’t make all this excellent content free for readers.

AT&T senior executive vice president John Donovan

AT&T senior executive vice president John Donovan

MLB tech profiles, interview with AT&T’s John Donovan

What’s in our issue #2 of STR? Glad you asked! Inside the report our editorial coverage includes:

— MLB stadium tech research: This editorial research provides a technology update on stadiums used by all 30 MLB teams, gauging the level of deployment of Wi-Fi, DAS and beaconing technologies.

— MLB tech deployment profiles: These mini-case studies will take an in-depth look at technology deployments at MLB facilities including AT&T Park in San Francisco, Target Field in Minneapolis, and Miami’s Marlins Park. This issue also includes an in-depth interview with AT&T senior executive vice president John Donovan, the man behind AT&T’s successful DAS deployment strategy.

— MSR exclusive stadium tech analysis: The report also includes MLB stadium tech analysis from MSR editor in chief Paul Kapustka, as well as a bonus mini-case study of DAS deployment at historic Churchill Downs, home of the Kentucky Derby.

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park.

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park.

MLB stadiums: Wi-Fi and DAS deployment is strong

Since last year, 8 more MLB teams have added fan-facing Wi-Fi to their ballparks, bringing the league total to a respectable 67 percent, with 20 out of 30 stadiums with Wi-Fi. On the distributed antenna system (DAS) front things are even better, with 25 out of 30 parks having enhanced cellular connectivity thanks to a DAS (our report currently erroneously shows that the Washington Nationals don’t have a DAS — we learned late this weekend that they do, so DAS is doing even better than we thought). Though the adoption rate is lower than that found in the NBA (where 26 of 29 stadiums have fan-facing Wi-Fi), baseball as a league does a much better visible job of promoting the service, which is more impressive when you consider that deploying Wi-Fi in an open-air arena is a considerably tougher task than in a building with a ceiling, like an NBA stadium.

Perhaps it’s no surprise that the National League West Division leads the pack in MLB connectivity, with all 5 teams having both Wi-Fi and DAS deployments in their park. The adoption rate in that division may have something to do with other teams following what is perhaps the overall connected-stadium leader in any sport, AT&T Park in San Francisco. The first with Wi-Fi (since 2004), AT&T Park continues to lead in innovation and experimentation, as witnessed by their embrace of new tricks like under-the-seat Wi-Fi APs and the new iBeacon technology, which is being tested in 20 MLB parks this season.

All of this is explained in greater detail in our STADIUM TECH REPORT for Q2 2014 — so download your free copy today!

StadiumPark touts ‘EZ Pass for stadiums’ parking app idea

Screen shot of a potential StadiumPark app. Credit: StadiumPark.

Screen shot of a potential StadiumPark app. Credit: StadiumPark.

At first blush, it’s an idea so simple you wonder why it hasn’t been thought of before: Why not build a system that mimics highway EZ Pass functionality to make parking at sports events easier?

That’s the simple but powerful idea behind StadiumPark, a Rochester, N.Y. startup that has developed an app that will let fans pay for parking with their phones, in the hopes of curing one of the main pain points of live game interactions with a faster, easier experience that can benefit teams and stadium owners as well. Though StadiumPark doesn’t yet have any announced customers, it’s a good bet that before long some stadium owners and operators will take a chance on the idea, which is designed to also automatically open parking-lot gates, further reducing human overhead.

In a recent phone conversation with StadiumPark’s 26-year-old founder, Jeremy Crane, we learned the skill sets behind StadiumPark’s insights: According to Crane, part of his work background includes time spent with a large parking-lot company in Rochester that handled concerns like apartment buildings and some stadium lots. An interest in learning more about mobile parking payment systems opened Crane’s eyes to the idea of parking-payment methods other than people in vests taking cash payments through a car window.

A request from Syracuse University, Crane said, to develop an “EZ Pass type app” for parking at the school’s Carrier Dome spurred him into entrepreneurial action, and StadiumPark was born. The combination of an app (which requires users to pre-register with a credit card) and the wireless technology smarts to open parking-lot gates is the main selling point for StadiumPark, which Crane said is in discussions with several potential clients.

If the system works as advertised, it could potentially cut down on the amount of time fans spend in parking-lot lines, one of the banes of live-game attendance. For stadium owners and operators, there is an extra possible incentive of having greater control over parking payments, as well as potentially having more data on fan attendance beyond ticket sales.

“For the venue, the idea is to enable a better experience,” said Crane. “We see a clear advantage to both the stadium and the fans.”

StadiumPark’s business plan is to charge users a small convenience fee, while not charging stadiums or venues. For the system to work well it must clearly have buy-in and promotion from the arena owners and operators, to steer traffic to the StadiumPark-enabled lots. But if the quick rise in mobile parking payments for other places — like airports or shopping areas — is any indicator, a simple app that lets you park quickly and conveniently is one of the uses of technology that could probably gain rapid adoption from fans who just want to get to their seats.

Stadium Tech Report: Miami Marlins rely on ExteNet DAS to keep wireless traffic flowing

Marlins Park. Credit all photos: Miami Marlins.

Marlins Park. Credit all photos: Miami Marlins.

If you know anything about Marlins Park, maybe it’s the stadium’s unique retractable roof or the spectacular art that catches your eye. But there’s also something you can’t see that is equally exciting, at least when it comes to the in-stadium connectivity experience: A neutral-host distributed antenna system (DAS) that has more than kept pace with the rapid, continual increase in fan cellular activity.

“When it came to DAS, we were ahead of the game,” said David Enriquez, senior director of information technology for the Miami Marlins, in a recent phone interview. Well before the 37,000-seat stadium opened in 2012, Enriquez said the Marlins’ IT team was researching and planning for enhanced cellular connectivity – even before “DAS” became a hot industry acronym.

“We planned for a DAS even before they were in vogue,” said Enriquez. “We saw it as a necessary evil.”

With the iPhone and all its cataclysmic changes already in motion, Enriquez said the Marlins wanted to avoid what had happened recently at another arena that opened in the Sunshine state without good connectivity.

“What we didn’t want to see was something like what happened in Orlando, when they opened the arena [in 2010], it had bad coverage, and they were crucified in the press for bad [cellular] service,” Enriquez said. “We said, what we’d love to have is the complete opposite of that.”

David Enriquez

David Enriquez

At the opening of Marlins Park, the connectivity inside the walls was better than most, with a full-park Wi-Fi network using gear from Meru Networks and a neutral-host DAS deployed by integrator ExteNet Systems. And though Wi-Fi often gets the headlines when there is talk about stadium networks, in many facilities like Marlins Park, the DAS is an equal workhorse, since many fans still either don’t know how or don’t take the time to switch their devices over to Wi-Fi.

DAS is the workhorse

According to Enriquez, on an average night at the ballpark the Wi-Fi network will handle 40 percent of the wireless traffic, with the DAS taking care of the other 60 percent. That may be because of lack of knowledge, or perhaps satisfaction with the signal the DAS is giving them, Enriquez said.

“Early on, most people, honestly, did not know how to change [their phone] to Wi-Fi,” Enriquez said. Most fans, he added, weren’t typically streaming lots of video — they may, he said, have used the MLB At Bat app to look at a replay or two, but that could all be handled by DAS. “That trend is changing though and we are seeing much more video traffic, especially with the younger generation of guests,” Enriquez said.

Marlins Park outside

Marlins Park outside

“The truth is, many users may not take the time to switch [to Wi-Fi],” Enriquez said. “If they’re getting 4 to 5 bars on their cellular signal, they’re happy.”

Though the Marlins and ExteNet now have five major carriers on their DAS – AT&T, Verizon Wireless, Sprint, T-Mobile and MetroPCS (now part of T-Mobile), Enriquez said there was a bit of the chicken and egg problem at the start.

“Early on, nobody wanted to be the first on (the neutral DAS),” Enriquez said. “ We [the stadium] were just another node. Now, 3 years later, we are a central node in the Miami area and all the carriers are here. We’re a very central location.”

Staying in neutral

Enriquez, who has considerable experience in the large-venue IT world, said that having a neutral host for the DAS eliminates any potential concerns about favoritism between service providers. Even though costs to the team or stadium may be lower if they allow a carrier to take over DAS deployment, Enriquez said that for the Marlins a neutral host was worth the extra price.

“We didn’t want an advantage to be held by one carrier,” Enriquez said. Even if a carrier says it will act as a neutral host, when one carrier owns the deployment, others can “find it hard to believe there will be an equal time slice” when it comes to antenna access.

“We just wanted to avoid that, and make it irrelevant [as a concern],” Enriquez said.

The choice of bringing in an integrator like ExteNet, he said, provides an additional streamlining of operations, as there is now a single point for vendors to interact with to work out technology and deployment issues.

“We wanted to deal with one vendor – I didn’t want to be the middleman between the carriers and the Marlins,” Enriquez said. In that regard, he said, ExteNet has been “wonderful” as a neutral host. “They deal with all the carrier issues that I have no desire to deal with,” Enriquez said.

Less space needed for DAS upgrades

And even as fan cellular bandwidth use continues to grow – requiring carriers to constantly upgrade their systems – Enriquez said that DAS infrastructure is benefiting from improved technology to the point where even as carriers upgrade, their head end footprint is shrinking.

AT&T, for instance, has upgraded its DAS presence in Marlins Park four times over the past 2 years, Enriquez said, to the point where the carrier now has coverage for all four frequency bands. “They [AT&T] have done quite a bit to expand their coverage,” Enriquez said.

Still, the Marlins Park DAS head end hasn’t had to find new space beyond its original 1,500-square foot enclosure.

“Every time someone comes in to replace gear, we have a smaller [DAS] footprint,” Enriquez said. “It’s not going to eat you out of house and home anymore.”

Like other stadium IT directors, Enriquez is still surprised by the amount of wireless traffic generated by the fans who come to the games. “It’s incredible to see the need [for bandwidth” grow,” he said. “But people continue to give our network a thumbs up, we see that in our guest comments all the time. I just don’t know what we would do without the DAS.”

Stadium Tech Report: Niners President Marathe confident that Levi’s Stadium network, apps will deliver as promised

Niners president Paraag Marathe (center) at Intersport Activation Summit panel.

Niners president Paraag Marathe (center) at Intersport Activation Summit panel.

So, Paraag Marathe — will the network at Levi’s Stadium live up to its considerable pre-launch billing and be ready to go when the stadium opens later this year?

“It better work, since we’ve been talking about it,” said Marathe Friday, during a panel discussion at the Intersport Activation Summit presented by SportsBusiness Journal/Daily in San Francisco. “We better be right.”

Even though the short history of in-stadium networks suggests that any new endeavor be launched with words of caution, Marathe and the San Francisco 49ers are instead confident — very confident — that their new stadium will launch with a network second to none, and have game-changing services like food and beverage delivery to seats and on-demand instant replay that will redefine the game-day experience.

Paraag Marathe, president, San Francisco 49ers

Paraag Marathe, president, San Francisco 49ers

In both his panel discussion at the Ritz-Carlton hotel and in an additional interview afterward, Marathe provided some additional details about plans the Niners have talked about previously for the technology features at the new stadium, which is located in Santa Clara, Calif., smack dab in the middle of Silicon Valley. Though Marathe said the stadium’s location — quite literally next door to several high-tech company campuses — made technology “part of the DNA,” he stressed Friday that the Niners are seeking to use technology to improve the fan experience, and not just to have cool stuff.

“It’s not technology for technology’s sake,” Marathe said. “It’s to enhance being at the game.”

But he did add that the stadium’s Wi-Fi network will be the base for much of the innovation.

Wi-Fi is ‘the master key’

An under-the-seat access point. Credit: Aruba Networks

An under-the-seat access point. Credit: Aruba Networks

The Wi-Fi network, which Marathe said “will absolutely be working” when the park opens, is “the master key that unlocks everything,” he said. Currently being built with Wi-Fi access point gear from Aruba Networks and back-end network equipment from Brocade Networks, the Levi’s Stadium Wi-Fi network will also have twin 10-Gigabit broadband pipes provided by Comcast to provide what Marathe said will be throughput “30 times more than any other stadium.”

Marathe said the Wi-Fi network is being built with what he calls a “spider web” of access points, though neither the Niners nor Aruba have yet said just how many access points will be used to create the network. There will also be a neutral-host cellular DAS at the stadium, built by DAS Group Professionals (DGP). Already, DGP has signed up the “big four” carriers of AT&T, Verizon Wireless, Sprint and T-Mobile, to use the DAS at the stadium.

What will the networks be used for? Marathe outlined four main points of technology innovation during his talk, including high-definition, on-demand replays via the new Niners stadium app; in-seat delivery of food and beverages to every seat; way-finding features to perform tasks like locating friends, finding parking spots, and to tell which bathroom lines are shortest; and paperless tickets, based on RFID and near-field communication to fans’ devices. Of the four, the replay idea and the food-delivery service stand out as massive technical and industrial challenges.

Promised: Better replays than those on TV

If there is one promise that has many in the stadium technology industry shaking their heads, it’s Marathe’s pledge of Levi’s being able to deliver “better replays than what the coaches are seeing,” since team coaches only get to see replays provided by the network broadcasts. The Niners, Marathe said, will have “a massive [internal] production crew” working on the replay feature, since replays not only need to be picked out of the video stream, they also need to be coded to work over the Internet and to be delivered to handsets. Though Marathe admitted that the video quality may dip a bit below true HD if a lot of fans try to watch replays at once, he told the conference crowd that the Niners’ stadium app is going to deliver “HD, slo-mo [replays] within seconds after a big play.”

While other stadiums, like Barclays Center in the NBA, use technologies like Cisco’s StadiumVision Mobile to deliver separate “channels” of live video and replays, Marathe said the Niners’ app will allow fans to choose their own replays and when they want to watch them. “If you have a [replay] channel, you’re subject to whatever is on that channel,” Marathe said.

The food-delivery feature, Marathe said, is more than putting a menu in an app — “it’s an immense industrial engineering exercise,” he said, to figure out things like how many runners are needed and when and how food needs to be prepared. In addition to food delivery — an option he said will be available to every seat in the 68,500-seat stadium — the Niners will also have “express pickup” lanes for digitally placed orders at concession stands, an idea that Marathe said helps eliminate or significantly reduce two of the three things that make concession interactions a time-consuming act.

“There’s decision time, transaction time, and preparation time,” Marathe said. “If you can eliminate two of three variables, that’s a few more minutes fans have to watch the game.” Waiting until fans show up at a stand to prepare the food will help keep the order fresh, he added.

Wayfinding, paperless tickets and the 9-Nerds

If there’s one idea that’s already gotten a lot of press, it’s the plan to have wayfinding technology assist features like the one that will let fans know how long the bathroom lines are. Marathe said the idea was to make it simple — “red light, yellow light, green light” — to let fans know that if they have to go, it might be faster to try the bathroom one section over.

“We’re really just trying to be smart,” said Marathe. Other wayfinding apps might include a parking-spot locator, or a friend-finder feature.

The fourth area where Marathe wants Levi’s to innovate in is paperless ticketing, which he said wouldn’t be 100 percent this year but it will eventually get there. A future scenario described by Marathe might use RFID or near-field communications to let fans simply walk through a gate without having to show a ticket or even a bar code to be scanned. Some ski areas, like Aspen and Vail in Colorado, already use such technology to let skiers get on lifts without having to show anyone their RFID-equipped lift tickets.

“The idea is to have greeters who can actually greet you” when you walk in, and perhaps extend a personal offer for discount goods purchases or seat upgrades, Marathe said. “It’s a more human interaction,” fueled by technology.

Wi-Fi coach in the stands at Gillette Stadium. Credit: Extreme Networks

Wi-Fi coach in the stands at Gillette Stadium. Credit: Extreme Networks

Finally, to help fans figure out how to use the new network and apps, Marathe confirmed plans previously reported by Mobile Sports Report to hire a crew of “network coaches” to roam the stands. According to Marathe the coaches will be called “9 Nerds” (say it quickly) and will likely be college students, dressed in what Marathe called “Poindexter outfits.” The Niners are looking to hire 150 such network helpers, which would be the largest such crew we’ve heard of in the stadium networking marketplace.

“They’ll stand out,” Marathe promised.

Lots of network use — and a team ready for its launch

With all the hype about the network, Marathe expects that Levi’s Stadium wireless usage will far eclipse that at other stadiums, where often far fewer than half of the fans in attendance actually ever use things like Wi-Fi or stadium apps.

“Forget 10 percent [fan network use], we’re going to see something higher,” Marathe said. Even people who don’t have digital devices, he said, will probably borrow one “just to bring it to Levi’s to test it out.”

When asked why his team was so confident — in an industry where under-promising seems to be a sensible way to go — Marathe said that both the Silicon Valley heritage and the greenfield nature of the building gives the Niners and Levi’s a technological edge.

“Five years ago, we put together a kind of think tank with VCs and design people, and thought about what would be useful [at a new stadium], well before we ever put a shovel in the ground,” Marathe said. And even though the Niners’ CTO left the team earlier this year, Marathe is confident that his crew of 25 engineers (which he said also still gets some consulting help from the departed CTO, Kunal Malik) will deliver the network and apps as promised.

Having advanced technology in the new stadium, Marathe said, “was our mandate — the DNA of the building is all these tech companies that are around us. It’s who we are.”

Stadium Tech Report: Upgrades keep San Francisco Giants and AT&T Park at front of stadium DAS and Wi-Fi league

Outside AT&T Park. All photos, Paul Kapustka, Mobile Sports Report. (Click on any photo for larger image)

Outside AT&T Park. All photos, Paul Kapustka, Mobile Sports Report. (Click on any photo for larger image)

What’s it like when the best-connected park in Major League Baseball loses its cellular mojo for a month? This winter the San Francisco Giants found out how fun it isn’t to revisit the days of “no signal,” when a DAS upgrade meant about 30 days of little to no connectivity inside AT&T Park.

“It was painful,” said Bill Schlough, senior vice president and chief information officer for the Giants, during a recent in-person interview at AT&T Park. Though no big sporting events took place during the Feburary-to-March overhaul of the main AT&T distributed antenna system (DAS) head end, Schlough said during that time many of the roughly 200 to 300 employees who work at AT&T Park every day were forced to find daylight to make a call, just like the bad old days before DAS.

“We never really knew how much we rely on DAS [for internal operations], but having it down really drove it home,” said Schlough. The good news on the DAS front was that once the upgrade was complete, the Giants had a lot more space in their previously cramped head-end headquarters. According to Schlough, the new back-end equipment for AT&T’s DAS operations takes up less than 50 percent of the previous gear footprint, room that is likely to be filled with gear from yet another carrier slated to join the AT&T neutral-host DAS later this season.

Painful, but worth it.

Second major upgrade in 5 years of DAS

Giants CIO Bill Schlough (left) talks with workers in the park's main DAS head end facility.

Giants CIO Bill Schlough (left) talks with workers in the park’s main DAS head end facility.

If you’re not familiar with a neutral DAS like the one at AT&T Park, it’s an implementation where there is one set of antennas and internal wiring, and then a “head end” where each carrier puts its cellular-specific networking gear, equipment that identifies and authorizes callers and then connects those calls or messages to fiber links back out to the Internet and beyond. As the lead provider of DAS and as the namesake sponsor of the park it makes sense that AT&T has the biggest DAS requirement on site. Verizon, which has been on the AT&T Park DAS for two years now, actually houses most of its head end gear in a separate facility nearby, and links to the AT&T Park system via fiber.

Part of this year’s DAS renovations include a new room specifically being built for Sprint’s DAS equipment, a sort of re-arrange-the-house construction project that saw the ballpark wall off half its painting services workshop to make space for Sprint’s gear. During our visit we saw workers putting up the racks that will hold the Sprint head end gear, as thick fiber cables snaked in the doorway.

Additional carrier(s) would likely be placed in the same room as AT&T and Verizon, on floor space that used to hold AT&T racks before those were un-drilled from the concrete floor and new racks were installed during the February-March overhaul. According to Schlough, the DAS upgrade (which required minimal tweaks to the previously installed DAS antennas) was the second major rip-and-replace action in the 5 years the DAS has been live at AT&T Park.

DAS performance improves over time; Wi-Fi is good too

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park.

White box at bottom is one of the under-the-seat Wi-Fi access points at AT&T Park.

Though Wi-Fi services in stadiums gets a lot of technology headlines, in many big arenas the DAS is an equal workhorse, connecting people who either don’t know how to or prefer not to connect to Wi-Fi services. Through the first 18 games of the 2014 season, Schlough said AT&T Park was seeing average AT&T traffic loads on the DAS of 150 Megabytes on the download side (fans requesting data) and 50 MB on the upload side (fans sending data). Figures for the Wi-Fi network (which is free to all customers) for the same span of games was an average of 400 MB download, 200 MB upload per game.

Schlough said performance stats for the AT&T portion of the DAS have improved vastly since the distributed antenna system was first put in.

“Just four or five years ago, 97 percent [connection rate] was actually relatively respectable,” Schlough said. Now, Schlough said network connect rates regularly hover in the “four nines” region, with a recent report showing a success rate of 99.9925 percent of all calls or texts going through.

The Wi-Fi network at AT&T Park, the first in any major sporting arena and still among the world’s most expansive, has more than 1,200 access points, many of which are now located beneath the seats. According to Schlough this coming offseason will likely represent the final phase of a stadium-wide deployment effort for the new, under-seat access points, which are installed symmetrically under the seats that are out in the open air.

Giants senior VP and CIO Bill Schlough, at the office

Giants senior VP and CIO Bill Schlough, at the office

Since AT&T Park doesn’t have many railings alongside the seats “in the bowl” or those in the upper decks, the under-the-seat APs were the only choice to extend Wi-Fi connectivity, he said. Though the box-like antennas do take away some under-seat storage area from approximately every 40th seat, Schlough said there haven’t been many complaints from fans about the gear.

What he has seen, however, are many compliments about the network services, especially from fellow professionals in the sports IT world.

“I get friends in the business who come here and send me texts with Speedtests attached, showing how great the Wi-Fi is,” said Schlough. My own ad hoc testing before our interview (albeit during non-game hours) showed speeds of greater than 40 Mbps for Wi-Fi just outside the park near McCovey Cove, and speeds of 25+ Mbps just outside the main gate. Schlough also showed us some of the new iBeacon antennas, which are being tested at MLB parks this summer to provide near-field communication marketing opportunities, like automatically checking fans in to the official At Bat app when they pass by a beacon. It’s just another way the best-connected park in baseball seeks to continue to improve the fan experience.

According to Schlough, the connectivity at AT&T Park doesn’t hurt when it comes to ticket sales.

“People do come here more frequently, I think, because they know there will be good connectivity,” said Schlough. “There’s no compromise. I do think that’s part of why we’re currently riding the third longest sellout streak in MLB history.”

MORE PHOTOS BELOW — CLICK ON IMAGES TO SEE LARGER VERSION

Can you find the iBeacon in the bowels of AT&T Park? It's the small grey box to the left of the other antenna.

Can you find the iBeacon in the bowels of AT&T Park? It’s the small grey box to the left of the other antenna.

Sprint's new DAS room at AT&T Park.

Sprint’s new DAS room at AT&T Park.

A close-up of the under-seat AP. Each AP requires holes drilled through concrete to provide wiring access. APs are weather-sealed, according to the Giants.

A close-up of the under-seat AP. Each AP requires holes drilled through concrete to provide wiring access. APs are weather-sealed, according to the Giants.

Bill Schlough's "old phones" collection. How many of these can you identify?

Bill Schlough’s “old phones” collection. How many of these can you identify?