Verizon Suffers NFL Mobile Failures on Opening Day

Did you have problems connecting to the Verizon NFL Mobile app Sunday? If so, it’s not your phone — it’s Verizon, which once again proved that it wasn’t ready for the opening day of the NFL season.

As a Verizon customer and a longtime NFL Mobile customer as well, I’ve experienced much frustration over the past couple years due to glitches with the app and programming for it. When it works I am amazed at the ability to watch live football on my phone. But how can two companies, the NFL and Verizon, which make billions in profits each year, have such consistent failures? Does anyone there care if the app actually works or not?

On Sunday I downloaded the new version of the NFL Mobile app, then tried to connect to watch RedZone for the afternoon game endings. I got an authentication failure, which surprised me since I had done everything possible beforehand (turned on all location services, turned off Wi-Fi) to make sure Verizon knew where my phone was.

After going through a half-hour of support hell waiting (including, ironically, a Drew Brees commercial telling me how great NFL Mobile is) I finally got a technician to tell me that because “so many people” were using the NFL Mobile app, Verizon’s network basically went kablooey, and that massive amounts of NFL Mobile users weren’t able to connect. Imagine that! People wanted to watch the NFL today! That’s like not stocking Elmo toys the week before Thanksgiving, or any similarly stupid move. The Verizon rep also told me that “because of high call volume” Sunday there weren’t enough technical reps at work to handle the NFL Mobile outage.

I will give the Twitter reps at NFL Mobile customer support a small bit of respect for finding my tweets and trying to respond, but really — this just shouldn’t happen. Not when the NFL itself says that more people are going to its websites via mobile than via desktops. The biggest app for the biggest sport simply shouldn’t have network failures. And reps shouldn’t be whining that too many people are trying to use it as the reason why it failed. Unless we all get a month’s credit on our Verizon bills.

Is “hiccups” on a server a new technical term? Is there a “hiccup reset” button?

As of 5 p.m. Pacific Time, it still wasn’t working for me. But the support folks found time for humor.

So — Verizon can pay the NFL a billion bucks for NFL Mobile rights, but can’t keep enough engineers on staff on Sundays to make the thing work? That’s fail with a capital F. Which is the grade we give Verizon for its performance on what is probably the sports world’s most-used app. And they’re locked in for four more years. Now I know what a Cleveland Brown fan must feel like.

UPDATE: At 5:42 p.m. Pacific Time NFL Mobile finally authenticated my device. Just in time to watch Tony Romo be Tony Romo.

Wi-Fi in Stadiums — It’s Not About Missing the Game, it’s About Staying Connected

Matthew Casey does an excellent job of summarizing the current state of Wi-Fi in the NFL in his post over on CNN.com, with some comments from yours truly as part of the package. While I think Matthew did a great job on the article, I was struck by how many of the comments are still of the “I go to the games to watch football, not to watch my phone” tenor.

People: Nobody goes to the stadium just to use Wi-Fi. But nobody leaves the house without their cell phone either. Being connected is simply a part of our everyday lives now, including leisure time. Going somewhere for several hours where there is no connectivity at all — and paying for the privilege — is going to seem more stupid with each passing year. Since football games have something like an average of 20 minutes of action for the entire time a live game is played, there’s lots of dead time in between. So what do you do during that time? For many people today, no matter where they are — spare time means time to connect, digitally and wirelessly. And it’s not a bad thing. It’s just who we are, no matter where we are.

Yes, if you’re at a game you want to spend time being there with the friends and fans around you. But our worlds are bigger than that in 2013. We don’t just share with those around us, we share with connected friends and with wider audiences in our social networks. Raise your hand if you spend a lot of time during sporting events these days texting friends or friendly rivals as a game progresses. (Guilty.) Most people, I think, who go to a game might spend at most a few minutes each hour checking messages, or posting a photo to show those who aren’t there what they are missing. I doubt most fans would go to a game and spend the whole time there immersed in the Internet. But I also think that almost every fan, at some point during a game, is going to want to connect to others in some fashion, even if it’s just to let someone know when you’ll be home. And going forward, Wi-Fi is going to be the answer as to how that happens. Why Wi-Fi?

The first difference at a stadium is simply the crush of people, which overwhelms the standard cellular-phone infrastructure that works very well in other parts of the planet. For high-capacity crowd situations, Wi-Fi is going to be the ongoing answer when the question is how to stay connected. Cellular carriers are moving quickly to put in DAS (Distributed Antenna System) deployments, which are basically a number of small cellular antennas that work inside buildings or in localized outdoor environments. DAS is good, maybe great, at eliminating dead zones that occur when a stadium full of fans are all fighting for the attention of a few regional macro cell towers. But DAS isn’t going to bring video replays to everyone in attendance. Again, that is why I said that Wi-Fi will be like plumbing going forward. In 5 years you won’t believe a big place hasn’t put in Wi-Fi yet. Because by then we’ll be doing a lot more wirelessly, because Wi-Fi is getting faster and better. And at stadiums, it can even make economic sense.

The second difference at a stadium is that unlike a hotel or other public places, stadiums have a captive crowd that might reasonably be interested in using a wireless network to make their visit more profitable for the owner/operator. That’s why the ideas of video replays to your phone, fantasy football stats via a stadium app, or even simple ordering concessions online to be picked up at an express window make so much sense. With a reliable Wi-Fi network all these options and ones that haven’t been thought of yet become possible. So for sports teams or owners of entertainment venues Wi-Fi might conceivably be able to pay for itself or even become a profit center, somewhere down the road. And we haven’t even started to talk yet about using Wi-Fi for improved internal operations, like public safety, ticketing or luxury suites. There’s money to be made on Wi-Fi networks, but first you have to build them.

That’s why Wi-Fi is coming to the stadium. It’s not so people can ignore the game to get work done. It’s so they can stay connected as necessary, or to enhance the experience they are having, at the few moments they want to do that. It’s a connected world we live in now, and stadiums shouldn’t be missing from that equation. Otherwise — as many of the commenters also noted — it will just be easier to stay at home and watch the game in HD while you tweet or text friends from your couch. Which you do now. If the NFL and other sporting leagues don’t want empty stadiums on those broadcasts, they need to make sure that their paying customers have the basic essential needs of human beings in this century, especially those fortunate enough to have time and money to spend on live-game visits. You wouldn’t build a big stadium without a lot of bathrooms. And these days, you wouldn’t build one without Wi-Fi either.

NFL, Verizon Introduce New NFL Mobile App

New NFL Mobile app; photo courtesy of NFL.

New NFL Mobile app; photo courtesy of NFL.

If you, like me, enjoy watching live NFL action on your Verizon smartphone, things are probably going to be a lot better this season as the league and Verizon have teamed up for a full refresh of the NFL Mobile app, which will also be available to smartphone customers of other carriers, with the only catch being that those customers won’t have access to live game action.

In a phone interview with Manish Jha, General Manager of Mobile at the National Football League, we learned about a whole bunch of new bells and whistles, maybe things you won’t necessarily notice, like improved back-end design and more up-to-date score information. What hasn’t changed for the 2013 season is the $5 monthly fee Verizon will charge its smartphone customers to view live NFL games on Sunday, Monday and Thursday nights, along with NFL network coverage and (perhaps most important) live RedZone coverage on Sunday.

(Don’t dismiss the RedZone coverage. If you live in California, like we do, you can sometimes get a full extra live game or two on Sundays because they are the only ones still going on.)

If your phone is from AT&T, or Sprint, or T-Mobile, you won’t be able to watch live action but you also won’t have to fight to find an NFL app that works well to bring you things like video highlights, replays, scores, and fantasy stats. Jha said the NFL and Verizon teamed up to produce one app to rule them all, the new NFL Mobile app which you should be able to find in the operating system app store of your choice. Also built into the new version of the app are some of the features from last year’s experimental Thursday Night Xtra app, which we never felt was fully baked; hopefully some of its interesting social media ideas are better fleshed out this time around.

“We want to make this the definitive NFL experience for a smartphone,” said Jha. And well should the NFL pay a lot of attention to mobile access: According to Jha, during last season mobile traffic to all NFL Internet sites surpassed desktop Internet traffic, even as that latter figure also increased.

Read that again: Mobile traffic to NFL sites eclipsed desktop traffic last year, even as desktop traffic grew. So now you have an idea why Verizon paid $1 billion to keep the rights for 4 more years as the NFL’s exclusive mobile carrier.

The only thing we still argue about is the league and Verizon’s decision to keep live cellular action off tablets; the Verzion NFL Mobile live access only works for devices they consider “phones,” though Jha even admits that such a definition may be a short-time thing, given the trend of phones getting big phat screens.

“We’re staying on top of trends, watching what comes out of Silicon Valley,” Jha said. “Right now we’re being pragmatic, trying to strike the right balance between serving fans and creating value for our sponsor.”

Google’s Latest Nexus 7 Raises the Bar on Tablets

nex7

Google has introduced its next generation Nexus 7 tablet this week and has incorporated a host of enhanced features as it seeks to maintain the popularity of its top selling 7-inch tablet, which already has a 10% Android market share.

The tablet, built by Google partner Asus, weighs in at slightly more than half a pound (0.64lbs with Wi-Fi, 0.66 lbs with LTE) and runs the Jelly Bean 4.3 operating system. Among the features in the latest Jelly Bean release is parental control features as well as support for the newest energy efficient Bluetooth technology.

The screen is a standout item on the latest Nexus 7 with support for 1080p high definition video that equates to 1920 x 1200 resolution and includes new stereo speakers. The tablet is powered by a Qualcomm snapdragon S4 Pro processor with 2GB of memory and the company said that you get 9 hours with active use from its battery.

The tablet has a pair of cameras with a 1.2 megapixel fixed focus front facing one and a 5MP auto focus rear facing camera. It sports dual band Wi-Fi 802.11 a/b/g/n, NFC and Bluetooth 4.0 with 4G LTE available as an option.

The Nexus 7 comes with two basic storage capacities, a 16GB model and a 32GB version. The company said that the tablet will be shipping in the US by July 30 with an entry level price of $229 and $269 for the high end model without 4G LTE, and then will start a global rollout.

The company has also delivered a product called Chromecast, a $35 USB dongle that enables you to stream video over Wi-Fi to smartphones, laptops and tablets. It plugs into a television’s HDMI port, you then load an app onto the receiving device and then chose what to watch. Initially Netflix, YouTube, Google Play Music and Google Play Movies are all supported.

It will be interesting to see what Apple and Samsung have in store to combat Google in terms of pricing and features when their next generation 7-inch products hit the market.

MSR Special Report: Bringing Technology to the ‘Friendly Confines’ of Wrigley Field

Wrigley Field on Opening Day, 2012. Photo courtesy of Chicago Cubs.  All rights reserved.

Wrigley Field on Opening Day, 2012. Photo courtesy of Chicago Cubs. All rights reserved.

Editor’s note: This is part of a series of interviews with speakers and thought leaders from the upcoming SEAT 2013 conference in Kansas City, Aug. 4-7.

There are baseball stadiums, and then there is Wrigley Field. As a dyed in the blue-pinstriped-wool Cubs fan, I can’t write objectively about the place. It is Mecca, the Friendly Confines, the hallowed outfield walls of ivy. It’s precisely because of people like me that Andrew McIntyre’s job of bringing better technology to the storied ballyard is so much more complex than that of his stadium-technology peers. Wrigley may have one of the greenest fields anywhere, but from an information-technology deployment standpoint Wrigley is about as far away from a “greenfield” project as you can get.

McIntyre, Senior Director of Information Technology for the Chicago Cubs, spoke with MSR recently on the phone to describe the delicate line his organization must tread as it brings necessary technology improvements to one of the world’s great historic sporting venues. In other stadiums, things like a brand-new huge video board would be welcomed, even celebrated. At Wrigley? Renovation plans that include an outfield video board will need to pass muster with national landmark regulations, and survice reactions from a widespread fan base that resists even the slightest changes to the stadium, and work with the unique neighborhood apartment buildings whose rooftops offer views into the stadium.

So when McIntyre said the Cubs need to get “everyone on board” before things like video screens can be introduced, he’s talking about a lot more than people who pull a Cubs paycheck. That extra planning, McIntyre admits, will likely keep the Cubs a bit behind their sports-stadium brethren in certain technology areas, like digital signage. But on many other fronts McIntyre and his technology team are helping the Cubs and Wrigley keep pace with advanced stadium services, like better mobile device connectivity.

Wi-Fi and DAS, with AT&T

Now in his second year with the Cubs, McIntyre and the IT team there has spent a good amount of time putting infrastructure in place that will support future efforts, beginning with things like optical fiber deployments that bring an almost 10-fold speed improvement in bandwidth backhaul.

Andrew McIntyre, Senior Director of Information Technology, Chicago Cubs. Photo courtesy of Chicago Cubs.  All rights reserved.

Andrew McIntyre, Senior Director of Information Technology, Chicago Cubs. Photo courtesy of Chicago Cubs. All rights reserved.

“Some severe infrastructure upgrades were needed here to enable initiatives moving forward,” McIntyre said. “There was historically a lack of investment from the IT side of the house. We’ve been working on a lot of non-fan-facing improvements that are very critical to us.”

One improvement that fans have been able to enjoy for the past season and a half is improved mobile connectivity inside the park, thanks to a neutral-host Distributed Antenna System (DAS) deployment and a stadium Wi-Fi network, built with carrier partner AT&T. “Next time you’re here, keep your eyes peeled for the antennas,” McIntyre said.

Having improved cellular and Wi-Fi connectivity puts the Cubs in the top third of MLB franchises, as by our count only 12 of the 30 major league parks currently offer free fan Wi-Fi services. When it comes to advanced apps and services that such in-park networks might power, like same-day seat upgrades or video replays, McIntyre said the Cubs are paying close attention to pilot programs underway at other parks, and will be “fast followers” when MLB-approved solutions are ready for prime time. (All in-stadium apps in baseball parks can only be run through the league’s At Bat or At the Ballpark mobile app.)

“I don’t think anyone’s knocking it out of the park yet” with in-stadium services, McIntyre said. But McIntyre also said he and the Cubs have met with franchises who are trying leading-edge deployments, including the San Francisco Giants and some European stadiums.

“We’re doing a lot of watching, listening, and learning,” McIntyre said.

Digital Signage as a Communication Vehicle

While most of the heated debate around the Cubs’ renovation plans centers on the size and placement of the proposed outfield video board, McIntyre and his team are looking deeper into a synchronized digital signage strategy, where boards all around the stadium — even, say, a concession stand pricing menu — could become a communications vehicle for the team to send messages out to the fans.

Wrigley Field marquee entrance. Photo courtesy of Chicago Cubs.  All rights reserved.

Wrigley Field marquee entrance. Photo courtesy of Chicago Cubs. All rights reserved.

“One major thing we are investigating is how the signage can change, to become a communications channel,” McIntyre said. Currently, when games go to a rain delay, there’s not a lot of ways for the team to give fans information about when the game might restart, or to communicate weather forecasts and safety instructions. That could change with a digital signage system that can instantly act as a synchronized stadium-wide messaging system.

“The digital signage strategy doesn’t necessarily get highlighted [in public discussions] but it can all become a vehicle to communicate,” McIntyre said.

Andrew McIntyre will be speaking at the upcoming SEAT Conference in Kansas City, Aug. 4-7.

AT&T Deploys ‘Mobile DAS’ to Answer Big-Event Needs

AT&T's Mobile DAS truck

AT&T’s Mobile DAS truck

For many sporting events these days, it’s a challenge to bring the mobile connectivity fans want and need when onsite. It’s an especially challenging task at outdoor events like golf tournaments, where infrastructure may be limited and large numbers of fans may be congregated in hard-to-reach areas, like near tees and greens.

At the recent Crowne Plaza Invitational at Colonial PGA Tour stop in Fort Worth, Texas, AT&T brought some new technology to handle fans’ mobile needs: A mobile DAS, or distributed antenna system, deployment that delivered much more granular cellular coverage to the event than other solutions, like cell towers on wheels (COWs) were capable of bringing.

Simply, by putting DAS head-end equipment into a truck (pictured above) and then by putting up a lot of smaller antennas around the course, AT&T was able to provide more coverage where it was needed, one of the hallmarks of any DAS system. With a few COWs on site as well, AT&T provided a much better level of connectivity than ever before, according to Chad Townes, VP of AT&T’s antenna solutions group.

“Your first choice is always to put in permanent infrastructure, and leave it there if we can,” said Townes in a phone interview earlier this week. But at many places where coverage is needed, like the Colonial Country Club, it simply isn’t possible to string wire and put up antennas. In the past, what cellular providers typically did in one-time situations was bring in the COWs, the cell antennas on wheels. The problem with those antennas, Townes said, is that they sometimes interfered with each other (if parked too closely together), plus they run into the same limitations a regular macro tower has, mainly an inability to handle a big, compact crowd of phone users.

Chad Townes, VP, AT&T Antenna Solutions Group

Chad Townes, VP, AT&T Antenna Solutions Group

Enter the mDAS, which Townes said was “created kind of out of necessity.” With numerous smaller antennas (which don’t interfere with each other as much) AT&T was able to provide much more targeted cellular coverage, a key for events like golf tournaments where fans are often clustered in areas like hospitality tents or greenside stands. Media and tournament officials also put heavy wireless demands on providers, and the mDAS was used to satisfy those bandwidth needs as well.

Townes said AT&T’s mDAS actually got its first test run at the Democratic National Convention a couple years ago, when AT&T didn’t want to sign a long-term DAS lease with the facility the convention was held in. The DAS truck was built, and the antennas were put up the day of the event, with great success. The next question was: Where to use mDAS next?

Inside the AT&T mDAS truck

Inside the AT&T mDAS truck

“Now that we had this asset, what could we do with it?” Townes said. Golf tournaments made instant sense, given their one-time needs for coverage and the infrastructure challenges of an open grassy field. Where else might AT&T deploy mDAS in the future? Maybe at state fairs, or motor sport events held on downtown streets, or big-crowd music festivals, Townes said.

Townes, who is scheduled to speak at the upcoming SEAT conference in Kansas City, said the mDAS was just another way AT&T has responded to the always-changing and never-the-same needs when it comes to providing big-event connectivity.

“Even when you’re putting wireless into stadiums, no two are alike because there are always differences in building materials, or the shape of the bowl,” Townes said. The mDAS, he added, “was just another example of how we had to get real creative” to solve an event’s connectivity needs. (AT&T video about the mDAS at Colonial below)