How an AT&T smartphone comes to life: behind the scenes (part one)

Have you ever wondered what happens behind the scenes to get a smartphone pushed to market? If you have, congratulations on advancing to the next echelon of mobile geekhood. We've often pondered the same thing ourselves, but the industry has been historically tight-lipped about the ins and outs of designing, testing and launching a smartphone. Fortunately, our thirst for knowledge was quenched by none other than AT&T, which happily loaned us the time of two senior product managers, the folks that make the magic happen.

AT&T Foundry innovation center walkthrough

On the ground with AT&T's Network Disaster Recovery team

Verizon Application Innovation Center walkthrough

Dante and Chris (last names withheld, per request) have the ultimate geek dream job: they don't just play with the latest prototypes, they're the ones who make sure they get made. Pretty plush gig, right? They're the ringleaders -- the decision makers responsible for making AT&T's smartphone lineup come to life, and their duty is to do it in a way that keeps the carrier at the front lines of innovation. So what drives these two men? What goes into their thought process, and what exactly happens behind locked doors in order to produce a groundbreaking smartphone? Join us in this two-part series as we learn some of their secrets.

The life of a product manager

We met up with the duo at the AT&T campus in downtown Redmond, WA. This three-building facility is where much of the smartphone magic takes place; a large part of the product team and most of the testing labs are based here. Even though Western Washington was rainy and gloomy -- anyone who's paid Seattle a visit will know that this isn't anything out of the ordinary -- the weather didn't seem to affect anyone we met. There was no mistaking that these two gentlemen love what they do.

We'd like to think that being a product manager simply means that you get to play with cool gadgets that nobody else gets to see, but there's a lot more to developing the latest and greatest innovations than just playing around with prototypes: those in this position have a colossal influence on which devices make it into AT&T's lineup, and how well they're received once they get to market. Needless to say, it's a huge responsibility. Dante, the man calling the shots on the development of the Motorola Atrix 4G (which we'll cover more in-depth), lives or dies by how his products fare. He accepts the blame if the phone's a failure, but he's quick to dish credit out to everybody on the team when it's a success.

AT&T is a polarizing company, to say the least. Poll two different people on their opinion of its wireless network, and more often than not you'll get completely opposite answers. But whether you love or hate the company, the gentlemen we sat down with seemed genuinely concerned in how their products are received. "We have a service contract with customers," Chris said. "We have to honor that -- it's sacred to us." What exactly does this mean, though?

In short, it's the product manager's job to produce a phone capable of lasting the entire length of a customer's contract, typically a full two years. "There's a misconception that we want you to buy a new phone every six months," Chris told us. "On the contrary, we're trying to create a lifetime relationship with our customers." The terms "sacred" and "service contract" were brought up several times throughout our discussion, indicating that user experience is one of their primary focus points. Whether or not they actually succeed at this is a burden that rests squarely on their shoulders -- as hard as they may work to make it an enjoyable phone, all of the fingers of blame point back to them if you loathe your handset.

But producing a phone doesn't just happen overnight. The development cycle of a typical smartphone -- going from conception to a full-out launch -- usually ranges from ten to eighteen months, and each handset faces a long and arduous journey along the way. AT&T considers itself one of the toughest cookies in the biz, and won't slap its logo on just any 'ol device. OEMs that want access to 100 million potential customers are required to meet stringent criteria and submit their hardware to intense testing.

One of the most intriguing smartphones of 2011 was Dante's project, the Motorola Atrix 4G. It was going to be AT&T's crown jewel, a flagship product with groundbreaking features. It was the first dual-core mobile device in the US, it offered a fingerprint sensor that doubles as a power button, and the most unique part of the project was Motorola's new game-changing Webtop environment and laptop dock. But a phone like the Atrix doesn't appear out of nowhere, so we were eager to talk to Dante and Chris about the process.

RFP - Request for Proposal

The RFP cycle encompasses the full genesis of the device. It begins with the creation of a formal document that lists the various traits and features AT&T desires. Since it takes so long to crank out a phone, the company needs to predict what the market's going to look like over a year in advance. This means our friends Dante and Chris have to ask themselves a few questions to hone their forecasting skills. What will be considered state of the art by then? How can we offer a truly groundbreaking product at that time? What will be on the low-end? What are customers going to want their phones to do? Answering these questions isn't easy, which is why AT&T has an advanced planning group that looks into all of the chipsets, displays and other components on the horizon.

The length of time a phone takes from conception to launch depends on a few factors: if the project was initiated by AT&T and the OEM needs extra time to work all of the crucial conceptual stuff, there are loads of extra vetting, testing and refining that needs to take place before the final product is ready. However, if the vendor brings a phone to the table that's already in development -- like the Samsung Galaxy S II, for instance -- the testing phase can be considerably shorter. The same rules often apply for second and third-gen models, as they usually use the same platform and UI and have less wrinkles to iron out.

As for the Atrix 4G, it was conceived to be a truly game-changing and innovative product, something AT&T arguably hadn't had since it signed the iPhone exclusivity agreement. Its RFP began in the final quarter of 2009. To offer perspective, this was right when the HTC Tilt 2 -- a Windows Mobile 6.5 smartphone -- was released. AT&T didn't have a single Android device in its smartphone lineup at the time; the very first phone sporting Google's mobile OS was the Motorola Backflip, which didn't launch until March 2010. Yet Ma Bell desperately wanted a game-changing Android device that would revolutionize the industry. It wanted the best idea from each vendor.

Each request includes a list of the various attributes, features and characteristics AT&T is looking for. The carrier doesn't have a specific OEM in mind when the RFP is sent out; instead, it goes to every OEM that's expressed interest in participating. Each one has the opportunity to respond to the document with questions of their own, those queries get formally answered, and the process goes back and forth until the vendor's ready to submit its proposal.

In the case of the Atrix, Motorola met with senior AT&T officials (Chris and Dante's bosses) at CES 2010 to show off its concept, codenamed "Evora." It was far from a polished Atrix, of course -- at this stage in the game, it was just an image of the phone's screen with some electrical circuitry. But every phone has to start somewhere, and the cost to manufacture just a few fully-functional prototypes for each proposal is simply too high; cost naturally decreases with mass production, so most OEMs won't put a live model out until "marriage" (the magical time when carrier and vendor get fully committed to a product and it becomes an official project).

OEM proposals come in droves, and the phones submitted to AT&T will vary from crude drawings on a piece of foam all the way to a realistic dummy unit similar to what you'd see shown off in a retail store. During our meeting, we were given a rare look into a box full of proposed devices (shown above, phones blurred to maintain confidentiality). The one we peered into covered a three-month period and contained at least 60 different units, averaging out to one per business day -- and that doesn't even include proposals that don't come with a tangible portrayal. The team sifts through a lot of candidates before finally settling on a short list of devices that it really likes, and only a select few of those get to taste the sweet privilege of being displayed in front of 100 million customers.

After sharing the Evora concept, AT&T came back to Motorola with feedback: in short, it told the OEM, "here's what we like and what we don't, but we're interested enough to continue moving the concept forward." The two companies volleyed the idea back and forth -- the vendor refining it based on the carrier's preferences and receiving more commentary in response -- until by the time CTIA 2010 rolls around in March, Motorola's crafted Evora to a point where AT&T loves it and is ready to get married. All in all, the time between the original request and the happy couple exchanging vows is around three or four months.

While most products follow this kind of courtship, there are a few exceptions. The BlackBerry Torch 9800, for instance, was a product AT&T got the ball rolling on. Instead of making a mass request to every vendor, it instead approached the OEM with a specific idea: can you make a touchscreen BlackBerry with a full QWERTY slider? Not only did RIM take on the project, it liked the finished product so much that it floated the model to other carriers -- causing the lackluster Torch to spread around the world like a disease.

AT&T takes risks from time to time by throwing handsets against the proverbial wall -- not literally, of course -- in hopes that one or two will stick. These guys know it may not crank out stellar sales, but the only way to hit a home run is to swing for the fences, right? This strategy brings to mind the Motorola Backflip and Flipout, but the Pantech Pocket (seen above), with its 4-inch 800 x 600 SVGA display, is the most recent example of such an oddball device. We briefly spoke with Michael Woodward, Vice President, Mobile Device Portfolio (and Dante and Chris's boss), who explained:

There doesn't appear to be any set rules or parameters for the selection process, but that's unsurprising due to the dynamic nature of the mobile industry. After all, it's difficult to come up with a standard selection process when dozens of vendors are cranking out hundreds of phones every year, and hardware evolves at a breakneck pace. Thus, AT&T weighs all of the proposals and chooses the phone that it deems the best fit for the desired feature set or customer segment. The project managers convene to narrow the field of potential candidates down to a few of the team's favorites, and takes the finalists to senior management as their recommendation. It's then up to executives such as Jeff Bradley -- SVP of Devices at AT&T -- to give the green light. From there, carrier and OEM are bonded together in an oh-so-beautiful marriage.

Product Definition

Marriage license (award letter) in hand, it's now pedal to the metal for the remainder of the phone's development, all the way up until its launch. The goal for the Evora at this point was to get the handset on shelves in time for the holiday season, which made the project the company's top priority. Normally this kind of flagship phone project would take 18 months, but Dante's team was trying to shave over half a year off that timetable. So, the challenges facing the team were absolutely massive.

This is where the intense collaboration and negotiations began. For AT&T to tackle the Evora project, it wanted to exert considerable influence on all aspects of the phone -- its look, feel, and the component hardware as well. With every phone, the final product never turns out quite the same as the manufacturers envisioned, though. Dante explains:

Evora's product definition took place between April and June of 2010. This is when the device went from a crude drawing on a piece of foam to a real-life prototype that's ready for testing. Teams on both sides sat down to hash out the nitty gritty details: form factor, colors, materials, display size, OS and basic pricing. It was rigorous and complex, and the negotiation was incredibly intense in this stage.

Once the concepts and specs were set, the project turned its focus to apps and services. In a nutshell, it's where the user experience gets fine-tuned. The two parties work together to determine the UI -- MotoBlur, in this case -- and every element of the full user experience, down to little things like the available options in the firmware's menu structure. Every aspect of the user experience is examined and no stone's left unturned. An entire team on AT&T's side is dedicated to developing for the UX and working directly with Motorola. Chris sums up the complexities of this seemingly simple process thusly:

There's a fine line, it seems, between coming across as over-the-top and being too conservative -- in this case, an Evora with too much UI saturation versus a plain vanilla Android. 'Course, we believe the Atrix's entire MotoBlur experience should've qualified as too invasive, but admittedly Motorola's recently tweaked its UI to be less in-your-face.

AT&T also encouraged Moto to push the limits a bit further this time -- what's the bleeding edge in our industry, and how far can we push that? As it turned out, that approach resulted in a few significant improvements in the Atrix down the road, as we'll cover shortly. Early evaluation samples of the Evora were looked over, and the two product teams continued to flesh out the finer details and early bugs.

The team was ready to begin the official kickoff within the company, which basically means that the major details about the device were fleshed out and AT&T's internal testing teams could then be alerted to the project. Virgil -- the codename for Motorola's Webtop and LapDock -- remained highly confidential and wouldn't be brought up for yet another few months.

Bloatware

Speaking of overall user experience, August 2010 witnessed the final decision on what apps would come preinstalled on the Evora. Have you purchased a smartphone only to find a litany of preloaded games and programs that are of no interest to you, but you can't delete them? These apps, no-so-affectionately known as "bloatware" and "crapware," have become an ubiquitous part of the smartphone experience. Ironic, given the universal ire they draw from the general public. Yet carriers continue to include the stuff in nearly every single handset. Worse still, very few of them are removable, which means these apps forever remain on your phone, taking up precious storage space. So, what's the big idea?

Many of the apps, according to the gentlemen at AT&T, haven't been deletable in the past because they weren't available on the Android Market -- in other words, once they were gone there was no way to get them back (aside from wiping your phone and losing all of your other data and apps in the process). No matter the reason for their existence, however, eliminating choice doesn't help the user experience. Most customers want the option to do whatever they'd like with their own phones, and don't take kindly to apps that you can't get rid of -- regardless of what findings come up in UX research.

Fortunately, the problem isn't as rampant as it used to be: the carrier now has its own hub within the Market where it can offer re-downloadable bloatware. That's why (at least in part) the Atrix 4G was the first device offered by the carrier to feature deletable preloads. This wasn't a fluke, either -- many of the smartphones in the lineup, including the Atrix 2, now allow branded apps to be uninstalled. Much like its CDMA competitor Sprint, AT&T received a flood of negative feedback associated with preinstalled apps and is working to streamline their numbers (compare the Atrix with the amount of bloatware Verizon releases on its typical Android phone and you'll see the difference).

Believe it or not, there's method to the madness: AT&T put the Evora through a vetting process to determine the breadth and scope of the apps and services to be featured. First, the carrier came to an agreement on a cap. After all, there's such a thing as too much, so they figured out where to draw the line. If there were too many apps, it'd be time to re-evaluate what got placed on the device and pull something out. Dante told us:

Dante goes on to explain that phones with large screens, for example, should feature games and apps that showcase its display size in order to enhance the user experience. Let's Golf 2 and Asphalt 6 were featured on the LG Thrill to accentuate the phone's 3D capabilities. As long as the customer has the option to get rid of these types of apps, that idea holds a lot of traction.

We were curious as to why certain apps were featured more often than others, but there doesn't appear to be any hidden revenue-generating partnerships between carrier and dev -- if any exist, the company's doing a good job keeping them hush-hush. AT&T's app team works directly with both high-caliber development companies like EA and Gameloft, but it also sponsors hack-a-thons and device giveaways to stimulate and encourage smaller developers. The carrier seems to gravitate toward a few preferred apps for the majority of its lineup, but Dante and Chris insist that no special partnerships or agreements are negotiated between them and the developer.

This is the first half of a two-part segment. Join us in the second half as we go into detail on device testing, launch preparation and more!