Report excerpt: MLB’s Wi-Fi everywhere plan nears completion

By the end of the season, Major League Baseball’s $300 million plan to bring Wi-Fi and DAS to every ballpark should be mostly complete, cementing the league’s title as the best-connected sport for now, and most likely for the near-term future as well.

While other sports, leagues and conferences rely on individual teams, schools and stadiums to figure out their own budgetary paths to connectivity, MLB’s unique decision to foot a major portion of the networking build-out bill should reap dividends for fans, clubs, and all the parties involved in the business of in-stadium wireless, said Joe Inzerillo, the executive vice president and chief technology officer for Major League Baseball Advanced Media (MLBAM).

“In general, things are going pretty well [with the build-outs],” said Inzerillo in a recent interview. “By the end of the calendar year, all the major construction will be complete.”

Vague on specifics, but clear on the goal

Editor’s note: This story is reprinted from our latest Stadium Tech Report, the BASEBALL (and Soccer!) ISSUE, which is available now for free download from our site. The report includes a focus on baseball and soccer stadium technology deployments, and team-by-team coverage of technology deployments for all 30 MLS teams — AND all 20 MLS teams. DOWNLOAD THE REPORT now and read for yourself!

Joe Inzerillo, Executive Vice President and Chief Technology Officer for Major League Baseball Advanced Media (MLBAM)

Joe Inzerillo, Executive Vice President and Chief Technology Officer for Major League Baseball Advanced Media (MLBAM)

In somewhat curious fashion, Inzerillo and MLBAM pointedly do not provide specifics on the buildouts, which include both new Wi-Fi networks for parks that didn’t have deployments, and many upgrades for those that did. While Inzerillo did provide a short list of some of the new MLBAM-led deployments, including those for the Kansas City Royals, the Colorado Rockies, the Texas Rangers, the Houston Astros, the Arizona Diamondbacks, the Chicago White Sox and the Minnesota Twins, to find out others we had to call or contact teams directly.

New networks coming in at St. Louis’ Busch Stadium and Milwaukee’s Miller Park are also MLBAM-led projects, according to sources with each team. So even though there is no master list of deployments, MSR research shows that by the end of the 2015 season all but two of the league’s 30 stadiums should have working, updated Wi-Fi and DAS deployments, with MLB primarily responsible for putting networks into eight of the 10 venues that didn’t have Wi-Fi as of last season.

The two-year-plus plan also includes construction of multiple new or retooled distributed antenna systems (DAS) that ensure participation by all four of the major U.S. wireless carriers, since all four also contributed to the overall buildout budget.

When asked for specifics on the amount that each entity – MLB, the carriers or the teams – contributed to the buildout pool, Inzerillo declined to give exact figures, other than the $300 million total.

“The notion is, everybody has an interest in making sure the people attending the game are happy with their [wireless] experience,” said Inzerillo, describing the thinking behind the deal that brought together the league, the carriers and the teams. While some industry sources seem to believe that carriers and MLBAM footed most of the deployment costs, Inzerillo said all entities involved paid some share of the total bill.

“It’s fair to say everybody has some skin in the game,” Inzerillo said.

For MLBAM, the reason behind seeking a solid level of connectivity at all parks is clear, due to “Ballpark,” its league-wide single app for in-stadium use. Though its features differ slightly from park to park depending on technology deployments (such as beacons) and desired uses, MLB’s plans to monetize its own apps depend on their being reliable connectivity on hand – so instead of waiting for teams to get there on their own, MLBAM led the charge, a move Inzerillo said made sense for several reasons, including the ability to herd carriers together and to share rare expertise.

Herding the cats known as carriers

If there’s a topic that scares stadium tech pros the most, it’s having to deal with all the major wireless carriers in negotiating DAS deployments. One of the top reasons some stadiums end up choosing a neutral host for their DAS is so that the neutral host provider can act as a buffer for the catfights that can occur between different carriers’ desires and needs. In MLB’s case, Inzerillo said MLBAM was that lukewarm water between the fire and ice.

Railing Wi-Fi AP enclosure at Seatte's Safeco Field. Photo: MLBAM

Railing Wi-Fi AP enclosure at Seatte’s Safeco Field. Photo: MLBAM

“We [MLBAM] are uniquely positioned to bring all the carriers together, and to give them a ‘safe zone’ to talk about potential issues,” Inzerillo said. Through its short history of supporting original team apps and then its own apps, Inzerillo said that MLBAM also gained a significant amount of internal telecom expertise, a resource not available to most teams.

Having dedicated app or telecom expertise in the form of a full-time employee is something that is hard to do locally, Inzerillo said. “It’s hard to find people with these skills,” he noted, especially for the smaller IT staffs found inside sports teams. The same thing goes for overall deployment expertise, Inzerillo said. “These deployments are big, capital-intensive projects, and they require specialized assets from a human standpoint,” Inzerillo said. “It’d be impossible to have resources [like MLBAM has] on a local staff.”

Construction is half the cost

That said, Inzerillo is quick to add that without the teams’ help and internal expertise, the entire project probably wouldn’t have gotten off the ground.

“We really couldn’t do this without the teams – there’s just a huge effort at each of the facilities,” Inzerillo said. The recognition of the local hard work, he said, is the main reason why MLBAM isn’t putting out a “master list” of its stadium upgrades – “It’s really a team story, and it’s something for them to tell their local markets,” Inzerillo said.

And for all the focus on the latest Wi-Fi antennas and fine-tuned DAS gear, Inzerillo also noted that it’s the other end of deployment – the physical work of putting networks in – that is the hardest part of the puzzle to solve.

“Things that can make or break a deployment are low tech,” Inzerillo said, claiming that half of the cost of any deployment is usually the labor to put it in. “Power, conduits and just knowing how to perform [construction] inside a ballpark cannot be understated,” Inzerillo said.

That’s especially true when you are putting a network in a venue like Boston’s Fenway Park, which is not just old but also has historic construction parameters that need to be worked around. Even Coors Field in Denver, which is a baby compared to Fenway, was still “old” when compared to the era of cellphones, Inzerillo noted.

“When it was built 20 years ago, there were no smartphones and barely an Internet,” said Inzerillo of Coors Field. So things like conduit easements, he said, weren’t anticipated, and added to construction costs.

“Twenty years doesn’t seem old, but compared to technology changes, it is,” he said.

And even though Inzerillo is eager to get to the end of the first long phase of deployments – he talked of “clearing the trees and getting ready to plant crops and tend the garden” – he also knows it’s a job that’s never quite over.

“It’s sort of like painting the Golden Gate bridge, where as soon as you’re done you need to start again,” Inzerillo said. “We’ve got beacons in 29 parks that we’re just starting to use, and now you have to think about things like wearables and the Apple Watch. There’s always new stuff, which is why infrastructure is so important. We’re always going to be thinking, do I have enough stuff in the walls?”

Stadium Tech Report: San Francisco’s AT&T Park lives up to its wireless reputation

Another tough day at the office for MSR. Credit all photos: Paul Kapustka, MSR. (Click on any picture for a larger image)

Another tough day at the office for MSR. Credit all photos: Paul Kapustka, MSR. (Click on any picture for a larger image)

Many times here at Mobile Sports Report we’ve referred to AT&T Park as the “most connected” or “best connected” stadium in baseball, if not for all sports anywhere. But even after multiple network tours and numerous reports from stats given to us by Giants CIO Bill Schlough and his staff I realized that one thing I’ve never done is to roam around the park on a game day, checking to see if its renowned Wi-Fi and DAS networks perform as promised.

Now after attending a recent day game as the guest of AT&T, I can tell you that the stadium that first put in fan-facing Wi-Fi for every seat is still at the forefront of ballpark connectivity, with Wi-Fi and DAS performance that knocks the ball out of the park almost every time. I say “almost” because during my walkaround tour I was able to find one place in the park that had almost no connectivity at all — but I will also bet you that as soon as this story is published, Schlough and his team will likely be out there the next day installing some kind of coverage since he and they have an almost unmatched enthusiasm for making their fan-facing network the best it can be. That, plus a strong partnership with AT&T, gives fans at Giants games perhaps the best stadium network anywhere, with performance so good for so long that it has almost become somewhat of an afterthought, one of the best kind of compliments a network staff can receive.

Nothing beats a strong team

Dynasty is a word fans like to use around Third and King Street in San Francisco, especially after the Giants won their third World Series title in 5 years last fall. Schlough, who loves baseball and the Giants as much as he loves networks, used the press-day gathering to show off his most recent Series ring, a chunk of gold and diamonds that probably gives you a wrist-curl workout when you put it on.

AT&T Park CIO Bill Schlough shows off his World Series bling.

AT&T Park CIO Bill Schlough shows off his World Series bling.

(Point of reference: Schlough offered the ring to me to try on, but as a dyed-blue Cubs fan I refused. “Waiting for one with ‘Cubs’ on it,” I told him. As we say in Chicago, “if it takes forever.”)

You might have heard some of the early Wi-Fi stories from AT&T Park before, but they’re fun to repeat. Then known as SBC Park, after the sponsoring “Baby Bell” that would later revive the family name of AT&T, the Wi-Fi network that debuted in 2004 attracted an average of about 94 fans a game, Schlough said, diehard geeks who would have to put up with people mocking them for bringing laptops to a ballgame. Remember, the iPhone was still 3 years away from existing, and you had to stick a PC card in a laptop to connect to Wi-Fi.

Fast forward to 2015, and now for regular season games the Giants see an average of just more than 13,000 people connecting to the Wi-Fi network, a number that has basically leveled off over the past 3 years, Schlough said. What hasn’t leveled off, however, is data use — even from last season, when fans used an average of 591 gigabytes of data per game, so far this season they’re averaging 915 GB per game. That’s why this season Schlough and team will be busy adding another 400+ Cisco Wi-Fi access points to the park, a total that should hit 1,700 by the time October rolls around.

“We’ll be working hand in hand with AT&T trying to stay one step ahead of demand,” Schlough said.

AT&T Park cabling runs are SRO

AT&T Park cabling runs are SRO

Scott Mair, AT&T’s senior vice president of technology, was on hand to help lead network tours before the game and to talk about how AT&T was using a new device called the EchoBOT to help gauge network performance in the park. Though it’s a bit of inside baseball, EchoBOT — which is basically an off-the-shelf cell phone that sits in a ruggedized box — is the kind of thing that can come out of the smart-person pool at a technology giant like AT&T. Invented in-house, the EchoBOT basically gives network operators an on-the-spot way to determine not just how the network is working, but what the actual user experience is like from the end-user point of view.

With 18 EchBOTs scattered throughout AT&T Park, Schlough and AT&T can get a much more granular view of how the stadium’s network is performing, just another way of using the resources of one of the world’s biggest telecom companies helps the fan experience at AT&T Park.

The one place without Wi-Fi

With our network tours concluded and some crispy chicken fingers inhaled in the comfy confines of the AT&T suite just above the third-base line, it was time to go to work to see if the AT&T Park network could deliver as promised. Since I’ve sat in seats at the park many times and had great connectivity there, I spent my time during the afternoon game seeking out what I thought might be some of the hardest places to bring connectivity. The first, in the second-level concourse, looked like it might be a tough antenna spot, with narrow halls and lots of concrete. But bam, on my Verizon iPhone 6 Plus I got a Wi-Fi signal of 31 Mbps down, 21 Mbps up; on my companion loaner device from AT&T, an LG Optimus G Pro, I got a 4G LTE connection of 26.15 Mbps down, 18.02 up. (For all remaining measurements I’ll just use the down/up convention to save time; I was using the standard Speedtest.net app from Ookla for all measurements.)

So yeah, you can connect while you’re in line to get a hot dog.

Here's the only place we couldn't find decent Wi-Fi. #firstworldproblem

Here’s the only place we couldn’t find decent Wi-Fi. #firstworldproblem

Strolling through the concourse toward right field, I saw several Wi-Fi advertisement signs, letting fans know they should definitely connect to the network. That’s a sign of deployment confidence, unlike many parks that install Wi-Fi but don’t really promote it, perhaps in order to keep user numbers down. After walking outside to get a view of McCovey Cove and the kayaks waiting for home-run balls to clear the park’s fences for a “splash landing,” I found a spot with almost zero connectivity — in the standing-room-only area backed up against the wall overlooking the bay.

With a Wi-Fi reading of 0.93/2.23 on the Verizon device and a 0.94/3.24 on Verizon 4G the SRO perch on the promenade was easily the poorest connection I found all day. But looking around, it’s kind of a silly place to be looking at your phone since from one direction a well-hit ball might be landing on your head and in the other direction there’s great views of the San Franicsco Bay. But still it does go to show that even in the most-connected stadium perhaps on earth, it’s not easy to get a signal everywhere. With no roof overhead and no railings close by, there simply isn’t a place to put an antenna out there. (But I bet Schlough and team will soon come up with a solution.)

Concourses covered, and upper deck too

I kept wandering around the outfield concourse and found decent connectivity at the centerfield Coors Light bar, 9.32/17.31 for Wi-Fi on the Verizon device and 14.21/43.00 for Wi-Fi on the AT&T device, as well as outside the Giants’ social media cafe in left-center, 14.32/32.20 for Wi-Fi on the Verizon device and 13.61/34.70 for Wi-Fi on the AT&T phone. Unless you’re a Wi-Fi geek like me you probably won’t ever see the APs since they are painted to match the structures they’re attached to. But I could see multiple APs hanging off the centerfield scoreboard structure, a piece of architecture that helps to deliver such solid connectivity to the open outside areas.

You can see Wi-FI APs -- if you know where to look.

You can see Wi-FI APs — if you know where to look.

Taking a break in a standing area behind the left field foul pole I got a smoking result for Wi-Fi on the Verizon phone, 22.46 Mbps on the download side and an amazing 52.05 on the upload. (I think it’s important to note that some of the best signals were on a device from an AT&T competitor, a sign that the facility does a great job of ensuring that any customer will get a good signal, no matter where you purchased your phone or service plan.) Climbing the stairwells to the view deck I still got a good signal on the concourse behind the seats in upper left field — 7.24/8.75 on the Verizon phone.

Since it was getting windy and cold (summer in SF!) I ducked back inside and found an empty seat in section 332, near the upper left field corner of the park. There I got a Verizon Wi-Fi mark of 17.52/24.33, and a Verizon 4G LTE mark of 5.62/6.81, again showing that the AT&T Park DAS is also delivering solid performance for customers on other carriers. The AT&T phone at that location saw 12.46/18.79 on Wi-Fi and 12.68/17.06 on 4G LTE. According to Schlough, the AT&T neutral-host DAS, which uses CommScope ION equipment, is so good that many fans don’t even bother to switch their phones to Wi-Fi. The upper deck, or view level, is scheduled to get many of the APs slated for installation this summer, in the under-the-seat enclosures that bring the network right into the seating areas.

Conclusion: Like the Giants, AT&T Park is tough to beat

Here at MSR we get the question a lot — “what’s the park with the best network?” — and I would have to say that like its tenants, AT&T Park is tough to beat. Schlough and the impressive IT team down at AT&T Stadium have a friendly rivalry, and you can’t have the most-connected discussion without mentioning Levi’s Stadium. But the park that did Wi-Fi first continues to improve year in and year out, never resting on its historic laurels. That’s a “dynasty” that is perhaps as impressive as the one built by the team on the field.

(More AT&T Park visit pictures below)

A little hard to see, but if you look closely you can see the Giants showing fan social media posts on the big screen.

A little hard to see, but if you look closely you can see the Giants showing fan social media posts on the big screen.

These signs are up all over the park

These signs are up all over the park

EchoBOT enclosures (white) next to a Wi-FI AP

EchoBOT enclosures (white) next to a Wi-FI AP

View from the Coors Light concourse walk-up bar in center field

View from the Coors Light concourse walk-up bar in center field

Good connectivity here in left field. Maybe the buttons on the hat improved reception?

Good connectivity here in left field. Maybe the buttons on the hat improved reception?

MLB app ads greet you as you walk up the stairs at AT&T Park

MLB app ads greet you as you walk up the stairs at AT&T Park

Nice place for a ballpark, don't you think?

Nice place for a ballpark, don’t you think?

For Giants fans only: I swear that thing weighs about 3 pounds

For Giants fans only: I swear that thing weighs about 3 pounds

MSR editor Paul Kapustka, your man on the Wi-Fi scene.

MSR editor Paul Kapustka, your man on the Wi-Fi scene.

https://duwit.ukdw.ac.id/document/pengadaan/slot777/

https://mtsnupakis.sch.id/wp-content/zeusslot/

https://insankamilsidoarjo.sch.id/wp-content/slot-zeus/

https://smpbhayangkari1sby.sch.id/wp-content/slot-zeus/

https://alhikamsurabaya.sch.id/wp-content/slot-thailand/

https://mtsnupakis.sch.id/wp-content/bonus-new-member/

https://smptagsby.sch.id/wp-content/slot-bet-200/

https://lookahindonesia.com/wp-content/bonus-new-member/

https://ponpesalkhairattanjungselor.sch.id/wp-content/mahjong-slot/

https://mtsnupakis.sch.id/wp-content/slot777/

https://sdlabum.sch.id/wp-content/slot777/

https://sdlabumblitar.sch.id/wp-content/bonus-new-member/

https://sdlabumblitar.sch.id/wp-content/spaceman/

https://paudlabumblitar.sch.id/wp-content/spaceman/