Analytics and big data of prime interest to stadium networks

Editor’s note: Please welcome guest poster R. Scott Raynovich, editor of the Rayno Report, which has just launched a free in-depth report on networking, analytics and big data. Read on for Rayno’s take on the value of intelligent analytics, and take advantage of his new free report. — PK

By R. Scott Raynovich, Rayno Report

You can’t throw a stone on the Internet without hitting some big data. But what’s interesting about big data, and its cousin, analytics, is that the technology has hundreds of applications in hundreds of markets, from digital marketing to security. And there’s a huge, untapped market that is unfolding in networking, including in-stadium and on-campus networks.

People are talking about Software-Defined Networking (SDN), and of course, everybody’s favorite, Network Function Virtualization. The discussion around many of these technologies is about commoditization and reduced cost of hardware. But that’s actually not where the value is created: The value is going to be created in opening up of the system and allowing for a lot more exchange of data and information and intelligent analytics.

Extreme Networks has made a smart move, I believe, by focusing on this with its recent partnership with the NFL. Extreme talks about the business use cases for analytics, such as analyzing what your customers are doing, monitoring performance, and overall, providing a better product. If you look at the logic of it, it’s this that networking vendors have as much data and information as anybody, because it passes through their pipes.

Analytics and the data center: a natural marriage

I recently completed an early study in this, which is included in a new report: Analytics and the Software-Defined Data Center. My research indicates that a new crop of technology is emerging that could tie together storage, computing, and networking, by providing cutting-edge measurement and analysis that allows managers to gain more insight into what’s going on in data centers and networks, and eventually automate their management.

See Extreme’s fancy graphic below on the Super Bowl. This just scratches the surface. What it shows, if anything, is how much information and analysis can be extracted from just about any network application. Imagine the data flowing through a ecommerce data center, or a digital media service. Businesses now have instant access to everything everybody is doing. Or how do some of the largest Websites in the world monitoring their networks for security breaches?

The answer is mostly in measurement and analytics technologies.

What’s interesting is that we’ve just started using this concept to make our networks and data-centers systems and more self-reliant. There is a long way to go, and you will start hearing more about this in the coming months.

Top 10 Benefits of Purview Application Awareness at Super Bowl XLVIII

(Source: Extreme Networks)

AT&T Super Bowl Sunday stadium wireless traffic jumps 60 percent to 624 GB; Verizon claims 1.9 TB

Inside the AT&T head-end building at MetLife. Cables! Credit: AT&T

Inside the AT&T head-end building at MetLife. Cables! Credit: AT&T

Well, our prediction that this year’s Super Bowl might not set new wireless data records was about as solid as the Denver Broncos’ performance in the big game. According to AT&T, its wireless network in and around MetLife Stadium on Super Sunday saw 624 GB of traffic, a 60.8 percent increase from last year’s 388 GB total at the Superdome in New Orleans.

Part of the gain no doubt came from the fact that MetLife Stadium holds approximately 10,000 more fans during a football game than the Superdome, 82,566 to 72,003. But AT&T’s numerical evidence — including a new peak-time high of 119 GB during the hour before kickoff — shows that the desire to access mobile devices inside stadiums is still growing, especially at big games like the Super Bowl.

Verizon Wireless, which like AT&T spent more than a year getting MetLife’s network ready for Sunday’s game, also said it had experienced record traffic on its networks, claiming a total of 1.9 TB of cellular traffic inside MetLife Stadium. During the game Verizon Twitter accounts said new traffic records were set before halftime; however we still have no figures from Verizon for previous years’ data for comparison.

Verizon also claimed its peak hour usage was “800 percent” greater than last year’s, but again, we don’t have exact numbers so can’t confirm the accuracy of such claims. Still unkown is how much traffic was carried by the stadium Wi-Fi network which Verizon manages (and was free to all MetLife fans), since Verizon did not provide Wi-Fi traffic numbers. Our guess is it saw similar traffic leaps as AT&T’s and Verizon’s cellular nets.

Another interesting note from the AT&T data is a trend toward more mobile data use, and fewer voice calls. Fans at the game Sunday made 55,000 voice calls on AT&T networks, down from 73,000 calls in 2012. Some of that might have to do with the fact that last year’s game had a lengthy power outage, which no doubt prompted many “I’m OK” calls to loved ones. But still — a 20,000 call dropoff in the face of more than doubling data traffic seems to show that people are using their phones more, even if they are talking less.

NFL Now: How much will you pay for on-demand NFL content?

For some time now, we here at Mobile Sports Report have been predicting that the NFL would eventually bring its digital assets in-house and market them to fans, for an additional fee. The first step in that process appears to have arrived today with the announcement of NFL Now, an Internet-based TV service designed to provide exclusive NFL content that includes just about everything EXCEPT live game action.

We’ll get to the game-action part later. While NFL Now, which will launch this summer, will include a lot of free content accompanied by a lot of ads, for now, what has yet to be announced is how much the NFL will try to charge its fans for access to the “good stuff,” a subscription tier called NFL Now Plus. Part of what “Plus” will provide is all the in-game highlights packages produced by the league’s media arm, as well as access to all the NFL Films archives that are available in digital format.

So how much will NFL fans pay for such archived gems? And how much will this experiment affect the NFL’s relationship with its TV rights partners? Though some fans may express disgust over the feeling of being charged even more, my guess is that many will just pay whatever it costs to have an easier way to view what matters to them — highlights and information about their teams and fantasy players, a level of customization that may be the key to NFL Now’s success. Is that worth $5 a month? $10? What would you pay for a direct pipeline to NFL content? How much TV time could that save you?

When it comes to watching highlights, sports fans everywhere know the frustration of having to sit through numerous SportsCenter segments that aren’t the ones you want to watch. (In fact, most of the modern world outside New York and Boston hates it when the Boston Red Sox and New York Yankees are both competitive, since when that happens roughly 98 percent of all ESPN highlights are Yankees, Red Sox, or blowhards talking about the Yankees and Red Sox.) If we are reading the reports of NFL Now correctly one of the service’s strengths will be allowing fans to customize their feed by team and by player, which should be nirvana for fanasty followers who often have “team” players scattered across the league. You can sign up to receive more information at this NFL page.

But the real digital asset that fans want — live streaming NFL game action — is still a big mystery and a mess when it comes to figuring out which games you can and can’t watch, and on what type of device. Right now a lot of digital-viewing confusion is tied up in NFL rights contracts, like the $1 billion deal with Verizon for NFL Mobile that gives Verizon exclusive access to live action on smartphones. That deal is why nobody could watch the Fox and CBS playoff streams on phones, and why you can’t watch NFL Mobile on tablets.

There’s also a hot mess of digital rights once fans get inside stadiums, an issue exacerbated by the fact that most NFL stadiums simply don’t yet have a wireless infrastructure that is able to handle numerous fans wanting to watch replays or live game streams. My guess on the NFL’s path to live video is that it will wait for technology to get better, which means another few years of fan frustration and confusion over how to best watch NFL content.

Will there ever be a simple way for a fan to pay a single price and have access to all the NFL content possible, no matter what device or access method? Maybe someday, but best guess is that it’s not someday soon. In the meantime, get your wallet ready for another small surcharge for NFL Now Plus.

Norwegian soccer league signs with Cisco for stadium Wi-Fi, StadiumVision Mobile

In a business agreement unique because of its breadth, Cisco has signed a league-wide deal with the Norwegian Professional Football League to put Cisco’s Connected Stadium Wi-Fi infrastructure and its StadiumVision Mobile video system in possibly as many as 16 different stadiums.

In taking a quick look at some video on the league’s site it doesn’t appear that the NPFL has huge stadiums, but signing league-wide deals for its stadium technology is always a win for a provider like Cisco. With many teams, especially smaller operations, not having a lot of technology expertise on staff can mean that banding together as a league makes sense to strike better deals and to help figure out the technology in a way that benefits everyone.

Working with a local integrator called Datametrix, Cisco said it has four teams signed up already to implement the technology, with two scheduled to be finished before the end of the season. Though the overall deal is managed through the league, each team is responsible for funding its own infrastructure so the schedule of deployments is open-ended.

Though Cisco has been the equipment provider for Wi-Fi services in many stadium deployments worldwide, its StadiumVision Mobile technology — which “broadcasts” as many as four separate channels in-venue, which in the sports world is usually live camera feeds or replays — is fairly new. A signature deployment of StadiumVision Mobile is at the Brooklyn Nets’ Barclays Center, where the stadium app shows live video from different camera angles.

Why the NFL is blocking streaming at the Super Bowl: Blame the network, not the fans

In case you are wondering why you won’t be able to watch the Super Bowl live on your phone while you’re at the game, Jon Brodkin at Ars Technica has a good story about why the NFL is blocking streaming video inside MetLife Stadium. To quickly recap, NFL CIO Michelle McKenna-Doyle told Brodkin that streaming video takes up too much bandwidth, and that it could hamper overall wireless communications on game day, so the league is proactively blocking live feeds at the game.

While I agree with her assessment of the situation and the solution (blocking live video), I don’t agree with her claim that the “vast majority of our fans want to watch the game on the field, watch the replays on the jumbo board, and participate in the event more than they want to be checking their phone,” and I’m surprised that Brodkin didn’t put up more of a challenge to this claim. Her follow-on claim that the league is doing the “vast majority” a favor by blocking the few video viewers rings hollow and reminds me of the old “data hogs” arguments the carriers used to use against people who were exercising their rights to their unlimited data contracts. My point: don’t blame fans who want to watch live video as being the people ruining the network for everyone else. Put the blame where it deserves to be, namely on the in-stadium networks that can’t yet handle the demands of a large crowd that wants video at the game.

We’ve talked before about why people want to stay connected while at the game. It’s not for everyone, but the desire to be online in your stadium seat is way more widespread than just a few people. Trolls will comment and say “watch the damn game and shut up” but plenty of real sports fans want the replays and closeups they are now accustomed to on TV. And not every seat has a good view of the big screens inside the stadium, and many times those things are showing ads, not replays. Then there is the time standing in line for a beer or bathroom. Why shouldn’t you be able to watch the game you are paying big bucks to be at, instead of being penalized because the stadium doesn’t have enough beer vendors or urinals? How about watching a replay while the game is in one of its lengthy TV timeouts? Or catching up on a play that you missed during halftime? Isn’t just having to listen to Bruno Mars punishment enough?

It will be interesting to see what the user statistics are like when the San Francisco 49ers’ new Levi’s Stadium opens this year, if its much-touted network delivers as hoped. The cynic in me is also guessing that when the NFL finally gets its digital video strategy figured out — meaning they clear the rights contracts and find a way to start charging fans more to watch more video online — the stadium network problems will suddenly be solved, and you’ll be able to watch all the live video you can afford.

We’ll be the first to acknowledge that putting networks in stadiums isn’t easy. Our recent Stadium Tech Reports series is designed to profile those in the industry who are trying to bring a quality wireless experience to their fans, so that others may follow. Already, we see places like Barclays Center and Gillette Stadium pushing the envelope when it comes to features like streaming video. In many ways, getting there is a long road that we’re just at the start of. If there is one bit of analysis I can provide after covering this field for the past 3 years, it’s that I don’t think anyone has gotten the stadium-network thing completely figured out just yet — and that any network put in over the past couple years is probably already in need of an upgrade, due to user demands already exceeding capacity. And that’s before most places are even thinking of providing live video feeds.

So sure, go ahead and block live video if it’s going to crash the network. But stop saying it’s something that just a few fans want, because there’s no proof behind that idea. Until the league and carriers like Verizon offer up real data on stadium network usage, there’s no way of telling exactly how many people at a game want to watch video, and whether it’s just for a replay or two or if they want a constant stream going at their seat. I’d be willing to bet more than a pint with McKenna-Doyle that if she polled an average NFL crowd and asked them if they’d like replays at their seats, a “vast majority” would vote for replays on their phones, and not in favor of settling for jumbotrons and PA announcers as she claims. So again, if you need to block the video, fine, but put the blame for the action on the network’s lack of capacity, and not on the fans who are just trying to enhance their own experience.

Stadium Tech Report: Verizon, AT&T DAS upgrades at MetLife Stadium await Super Bowl Sunday

Verizon branded gate at MetLife Stadium. Credit: Verizon Wireless

Verizon branded gate at MetLife Stadium. Credit: Verizon Wireless

Super Bowl foes the Seattle Seahawks and the Denver Broncos have only had two weeks to prepare for each other. But Verizon Wireless and AT&T have been preparing for the roman-numeral date at MetLife Stadium for more than a year. Will the Verizon Wi-Fi and DAS, and AT&T’s separate DAS be able to handle the wireless needs of the fans at the NFL’s biggest game? Tune in Sunday to see!

We might be one of the only news outlets who care more about the wireless networking at Super Bowls than the game itself, but for many in the stadium tech industry the biggest single game in America’s most popular sport is always somewhat of a wireless watershed. Perhaps at no other event do attendees spend so much time shooting selfies and posting them as they do from the site of Super Bowl Sunday. Even in the expected cold, it should be no different this week at Super Bowl XLVIII at MetLife Stadium in New Jersey, where game time is scheduled for around 6:30 p.m. Eastern.

But well before that, fans will be testing the Verizon-built Wi-Fi network and both of the big carriers’ DAS deployments inside MetLife. According to reps from both companies that we spoke with last week, the carriers are ready.

Verizon spokesman David Samberg told us via email last week that an upgrade of the Verizon DAS in MetLife this past season means there are now more than 500 DAS antennas inside the facility. See some of the photos provided by Verizon that show the clever hiding spots Verizon engineers have found over the last 18 months as they’ve added capacity to a network built just a few years ago.

AT&T has also put in a brand-new DAS over the past year, with another 500-plus antennas of its own. “For the last year or so we’ve been working on our pre-game and game day network playbook in an effort to provide the best possible wireless experience for our customers,” said Michael Maus, assistant vice president of network services at AT&T, via email. “In anticipation of the huge volume of data and voice usage expected [for the Super Bowl], we’ve built a new state of the art antenna system inside the stadium, we’re rolling in portable cell sites both at the stadium, and to support the tailgate areas, and we’ve augmented coverage in New York City to support the activities there.”

Rooftop DAS equipment. Credit: Verizon Wireless

Rooftop DAS equipment. Credit: Verizon Wireless

For Verizon’s own customers, 4G LTE capacity at MetLife has been quadrupled since last year, according to Samberg, who said that all stadium upgrades were completed by October, giving Verizon multiple chances to test its system against crowds at New York Giants and New York Jets home games. So far, Samberg said, so good.

More traffic than last year’s Super Bowl already

While yours truly opined earlier this year that this year’s Super Bowl might not set a wireless traffic record, we didn’t take into account some simple numbers — mainly, that MetLife Stadium’s official capacity of 82,566 is bigger than the Superdome’s 72,003. So, even if it’s cold, having 10,000 more people on hand probably means more bandwidth consumed, even if this year’s game doesn’t have a power blackout in the second half. (And even if it does, Samberg said the network shouldn’t go down since Verizon has backup power supplies on hand.)

Find the DAS antenna! Credit: Verizon Wireless

Find the DAS antenna! Credit: Verizon Wireless

Our only problem with record wireless numbers from Sunday’s game is that we probably won’t ever see an actual number, since Verizon historically shies away from providing a score. Instead it just issues press releases saying things like “way more traffic this year than last!” and then expects us all to believe that without numbers. The good news for fans at the game is that the in-stadium Wi-Fi network, also built by Verizon, is free and open to customers of all carriers, or basically anyone with a device that has a Wi-Fi chip. But Verizon, like big competitor AT&T, has been beefing up its DAS installations significantly because most people try cellular first, even at stadiums, before instructing their phones to find a Wi-Fi network. AT&T, to its credit, usually does deliver a wireless scorecard quickly after big events. So at least from AT&T’s perspective we should find out if this year’s game sets another record.

Aside from the stadium improvements, Verizon will be showing a demonstration of a technology this week that could make DAS more of a competitor to Wi-Fi on the high bandwidth side of things. Called LTE multicast, the technology basically establishes set channels for LTE devices that will “broadcast” video, like a TV channel. (This idea is similar to the StadiumVision Mobile technology Cisco uses at stadiums like Barclays Center.) Theoretically, LTE multicast could let fans use a cellular connection to view multiple video streams, something you would need to use Wi-Fi for it to have any chance of working. But the multicast demo won’t take place at MetLife, but instead at Bryant Park in Manhattan this week. If you are in the city, check out the demo and let us know what you think.

No NFL Mobile at MetLife

And here’s something else you won’t be able to use at MetLife during the Super Bowl: Verizon’s own NFL Mobile app, which outside the stadium will be the only smartphone platform you’ll be able to watch the game on. (The Fox streaming site and app will only work with tablets and desktops or laptops, per the league’s rights agreement with Verizon.) Next year, the rights for NFL Mobile will change and if the local game (like, say, the Super Bowl) is on TV, you’ll be able to use NFL Mobile to watch it even if you’re at the stadium. But not this year! (To give one answer as to why, if you are at the game, you might want to watch it on your cell phone, we say: Bathroom or beer lines!)

More stadium infrastructure photos below:

AT&T DAS antennas at MetLife. Credit: AT&T

AT&T DAS antennas at MetLife. Credit: AT&T

AT&T's new head-end building at MetLife, where its DAS gear is housed. Credit: AT&T

AT&T’s new head-end building at MetLife, where its DAS gear is housed. Credit: AT&T

Inside the AT&T head-end building at MetLife. Cables! Credit: AT&T

Inside the AT&T head-end building at MetLife. Cables! Credit: AT&T