Andrew Hsu: The Invention of TOUCH & What’s Next

Bill Gates stood on the stage at the (now-defunct) Comdex show in Las Vegas in 2000 with his schoolboy smile touting the new “tablet PC.”  Penned on the tablet in Bill’s handwriting was “Tablet PC is SUPER COOL!”  Behind the stage a backlit sign read “experience the evolution”.

Microsoft evolution never became a revolution because the company’s disparate and factional divisions failed to work together to vision and implement a turnkey experience.

The revolution happened in 2007 with the launch of the iPhone.

(As with most industries) evolution is often interrupted by black-swan revolutions. Sound (voice communications), touch (pinch and zoom navigation), sight (Heads Up Display [HUD]) all changed the way consumer used the phone and is one of the gating factors in technology adoption.

Knowing what technology will help us evolve and what technology revolutionizes is more of a human insight that a science. Ergonomics help us rearrange the digital furniture; however, changing the way we connect with this communication device is profoundly human. What is beyond touch, what is the next revolution?

A Short History of Touch

Although Gates told reporters off stage in Las Vegas that how excited everyone was in Redmond (Developers were checking the tablet out to play with – “a very good sign,” he said) 6 months later warehouses were still full of the Tablets. Q2 shipments had plummeted 25% with a meager 100,000 total units sold.  Mike Magee, technology writer for the Inquirer wrote despondently that “This is another classic case of IT firms thinking they know what technology people will like, and failing to take off the blinkers.”

Touch appeared back in 1971 over a ten year period began to appear in the form of  infrared technology (such as the Hewlett Packard 150) which show up in various military applications. The IR matrix of beams are used to detect a finger touching the screen.

But the IR technology was expensive and the technology gained more mainstream adoption was “resistive touch”.

It was a simple concept. Resistive touch screens were built using two layers of conductive material (Indium Tin Oxide). The two layers were separated by a small pocket of air. An action was triggered when a stylus, or other object, pressed the top layer into contact with the bottom layer.

The limitation was it was like a pin board. You could tell the device where you were move the point of contact. But it did not have multi-touch functionality essential to pitch and zoom navigation.

Mass-market adoption was not an option:

  1. The screen wore out
  2. Required a stylus pen for accuracy
  3. The air pocket made the screen appear hazy
  4. OEMs had to build a clunky hole in the casing (as the top of the resistive sensor had to be exposed to user’s input)

This is the technology that Bill Gates was holding up at Comdex in 2000*. The unit’s resistive touch stylus was used to input into clunky dialogue boxes to input text and commands. The entire project was “resistive”. The Office team refused to build for the unit adding to the painful UX.

*[A technogeek aside: Microsoft’s Surface touch solution uses Frustrated Total Internal Reflection (FTIR)]

Meeting Andrew Hsu

In 2013, I ran an event on connect screens in New York. I wanted to tell a story about the importance of the screen in the evolution of mobile phone design and adoption. I invited Professor Donnell Walton from Corning Glass, as well as representatives from Microsoft’s Surface team, Google Glass and was looking to find a speaker to explain “touch”.  Maybe I could locate someone from the scuttled Apple Newton team?

I found, much to my surprise (like an anthropologist that finds that we did not evolve directly from monkeys) that the precursor to the 2007 Apple iPhone was a skunk works project headed up by an engineer called Andrew Hsu.

Andrew developed and patented a capacitive touchscreen suitable for mobile devices way back in 1999. He developed a system which computes the location of a user’s fingers based on how they change the capacitance values of an invisible matrix of electrodes.  The capacitive touchscreen did not suffer from the various user experience drawbacks of the resistive touchscreen – it does not wear out, it does not cloudy the underlying display, and it does not require a big hole to be cut into the device casing.  But most importantly, it enables natural finger input.

This capacitive touch is not a mouse click. It is not a data poke with a Stylus. Andrew Hsu’s touch allowed us to communicate in a very human way with pointing and pinching space.

Don Norman is often quoted about touch.

“We’ve lost something really big when we went to the abstraction of a computer with a mouse and a keyboard, it wasn’t real . . . swiping your hand across the page  . . . is more intimate. Think of it not as a swipe, think of it as a caress.”

While mobile success is almost always based on interface and usability, it took seven years for Andrew Hsu to convince the industry to adopt the technology. Revolutions come in simple packages: text messaging, Apple’s mobile application SDK, gesture-based gaming.

We talk about the consumerization of technology; touch was the humanization of technology. In a world where data appeared cerebral and uninviting, we suddenly can interface in this data and content as we do with real object. The physical world became extensible and less scary.

From Click to Pinch & Zoom

In 2006, handset manufacturer LG trialled launched capacitive touch with their designer Prada phone. The LG phone had all the correct ingredients – capacitive touchscreen for intuitive finger input, high resolution display, and one of the first graphics co-processors in a handset. Prada brought style to the table and LG brought the insight that touch that would ultimately inspire the new mobile consumer.

But we had to wait one more year.

When Jobs returned to Apple he shut down the Newton project.  This legacy 1993 technology had poor handwriting recognition and had little traction in the market. But Andrew Hsu’s capacitive touch appealed to Steve Jobs UI sensibilities.

As a post Newtonist, Jobs once said “we are born with five styluses on each hand”.

When he introduced the iPhone, we knew that being able to move large format data on a small screen with a pinch and zoom changed the way the consumer saw their mobile device.  Where Steve Job went further than touch was his insight in designing a full edge-to-edge screen that had the dimensions of a letter size piece of paper.  The screen called out to be touched, worked on and paged through.

Although touch revolutionize the phone and lines weaved around the block for new product releases of Apple new “human” interface, the consumer was still nose-to-screen, bumping into lamp posts while elegantly navigating data a hundred miles away.

“Bump” (the file exchange application recently acquired by Google) and other application including NFC payment extending this love of tactile interface by promote social touch between other phones and public devices such as POS.

Gesture: Moving Beyond the Cool?

While touch is an important sense, sight is essential for navigation. The next revolution is to make data come to live seamlessly in the real world.

When we talk about HUD, we think of the new Google Glass and the opportunity to integrate data into our line of site. In parallel, see the world and the data behind it. Integrated cyborg solutions like Google Glass and future visions of embedded epidermal circuit (seen in Total Recall).

Microsoft had the lead in a new HUD interface using gesture.  XBOX Kinects was the one product that Microsoft was seeing growth in the consumer sector. However, the leviathan was unable to make this a multiscreen strategy fast enough.

Moving gesture elegantly to PCs and window phones never happened. There is a Kinect for Windows but it lacks the software for controlling the interface.

The Leap motion controller is a step forward.  A small multiscreen sensor box not tied to console in the dean but with the ability to tether like a dongle to a wide variety of screens and deliver better sensitivity to Kinect. It has multiple commands down to finger level accuracy.

Andrew Hsu still believes that touch is less ambiguous on the consumer navigation intent. “How can you disambiguate between “accidental” and intentional gestures.  The beauty of touch interaction is that you basically get user intent for “free” – a user typically only touches the device when he/she wants to interact with it.  The cases of accidental activation are much lower and easier to reject.”

Arguably HUD is a solution looking for a problem. Like the inspired Seque cycles, the inventor’s goal was to develop an urban consumer transport vehicle but he failed to get significant adoption. The Segue has now found a home with urban tourist touring groups and airport police. Why? It provided an elevated view with minimal multitasking: Ideal for tourists and law enforcement.

Andrew agrees: “What these technologies really need to address is what sort of “problem” they are trying to solve.  That is, with capacitive touchscreens, there were certainly a number of value propositions that arguably were superior to the previous (resistive) solution that helped transform/enable touch input.  Natural gestures (HUD) is still looking for a compelling value proposition”

Google Glass is a platform without a certain home. Will “super cool” it has not inspired the consumer. We have not seen the “a-ha!” that Jobs brought to the touch. We know new more intuitive human interfaces are coming. But we need a Steve Jobs to take the technology and humanize it for intuitive consumption.

Gary Schwartz is the CEO of Impact Mobile. Having been at the frontlines of the mobile industry for over a decade, Gary is the author of two books, “The Impulse Economy: Understanding Mobile Shoppers” and “Fast Shopper. Slow Store: A Guide to Courting and Capturing the Mobile Consumers,” both of which highlight the current state of the mobile commerce space and chronicle the significant impact that mobile is having on consumers, retailers and brands. Gary is also a chair emeritus for the Interactive Advertising Bureau and the Mobile Entertainment Forum NA and global director of the Location Based Marketing Association.

From Singapore to Moscow: Is OPEN a four-letter word?

by Gary Schwartz  (23/07/2013)

I recently returned from speaking in Singapore at the regional CommunicAsia conference and Rasia.com event in Moscow. Innovation was a central theme.

In Singapore I was in a discussion with Google, SingTel and Microsoft. I asked Doug Farber, managing director for Google in Asia-Pacific, if mobile innovation is stifled when there are only a few power brokers that control the mobile ecosystem.

Google has always had a “healthy disregard for the impossible,” Mr. Farber said. Agreed. However, while Google has a healthy internal innovation culture, is it allowing the ecosystem to do the same?

I recall Richard Kramer from Arete Research’s jab that in mobile O-P-E-N is a four-letter word.

There are only a few powers that are trying to lock the mobile ecosystem: Apple, Amazon, Google, Facebook, Microsoft and Du. And while Apple and carrier networks are unabashedly closed fiefdoms, Google’s Android empire may not be truly open.

“But,” Mr. Farber said, “I am the open guy.”

“Open only on the front end,” responded Bill Chang, CEO of group enterprise at SingTel.

Outside of a few global plenipotentiaries, mobile developers have had to pick from the crumbs at the table.

Twenty-five applications command 50 percent of the app store revenue. The billions in revenue do not feed many mouths. The average developer lives on $5,000 per month, which is a bread-and-water diet of $60,000 per year, said Richard Kramer. “Where are the developer’s yachts?”

Hardware innovation has flat-lined and power is exclusively in the hands of Apple and Samsung.

The yearly CES announcements have underwhelmed and recently left the Las Vegas melee altogether. Another handset release with faster “mega” screens does not excite the crowds. With $75 smartphones entering emerging markets, smart has become a commodity.

SingTel’s Mr. Chang said in a market where companies are “cost-cut to death, it is difficult to drive innovation forward. Mobile-first is a challenge.”

Mr. Chang talked about the importance of the CIO is this process. He plays the initialism game that we all do at public events: “The CTO is now the chief transformation officer and the CIO the chief innovation officer.”

But what does that really mean to companies trying to navigate the mobile marketplace?

The CIO never had power, said Michael Thatcher, chief technology officer of Asia for Microsoft. “It is only when something is broken they have power. How can we advance this and stop being tech centric? Stop being reactive. Follow the money.”

But is there money in mobile innovation?

Outside of the big six ecosystem players, there are few that command significant market share. We live in an “app store economy” where we all need to play the piper 30 percent and never own the customer. To make money on Apple’s SDKs or Google’s, dominant big data position is difficult.

The social leviathans are also closed.

Facebook has built a business on connecting people. With its post-IPO revenue focus, it has worked hard to create a media buying tollgate on the impressions and big data that it owns.

In social portals there is little big data sharing and little opportunity to make money as an outside developer. It is a bigger issue for the entire social ecosystem.

The more that these social portals try to leverage these assets to generate revenue, the less socially authentic they become to the consumer. Our social spaces have become more like driving down the public highway.

Social portals such as Tumblr and Instagram have generated such value in the market because they remain authentic – pre-revenue and pre-commercial, of course.

Everything is disintermediated. App stores and social portals are all controlled by Facebook, Amazon, Apple, Microsoft and Google, and somewhere we all have to pay the piper.

Telecommunications is the history of open and then closed systems, from RCA closing down FM Radio and early television.

It is the same recent history of Apple disintermediating the wireless carriers with an “Internet device” and then turning around and using the same iPhone to shut down the mobile Web with a closed App Store.

Are we entering another closed loop where innovation becomes stifled? What does this mean for business?

Google sponsors the ship, “Unreasonable at Sea,” to sail around the world evangelizing entrepreneurism and innovation. How can we make the power brokers more unreasonable at home?

MultiScreen not Omniscreens or Second-Screen

I strongly believe that the space between the screens that allows the brand to connect their consumer experience is critical. I call this “digital Velcro”.  Connecting your consumer screens seamlessly throughout their day is one of the most important challenges we face.

Additionally, one media experience plus other media screen experience equals a multiple of value to the brand: “1+1=3”  This is not a second screen debate. It is a consumer journey challenge that impacts all brands and retailers in their media buying and engaging. That is why we focus on the MultiScreen not the Omniscreens or Second-Screen discussion.

Next week on Tuesday and Wednesday in NYC I will be hosting the MultiScreen Summit. We incubated this event concept in Los Angeles, last year and have expanded the MultiScreen Summit to include Abu Dhabi and Berlin (which will be held in October). Our goal is to establish a global platform to address the growing needs of business to create a seamless digital experiences across their customers, fans, audiences, shoppers . . . screens.

So who owns this? CIO, CMO and CDO? CIO needs to get together with the CMO and the Chief Digital Officer (CDO) needs to become the Chief Disruptive Office (CDO) making backend process screens work with marketing engagement on the consumer screens.

Data has always been important. In a connect device world with location and behavioral
MultiScreen economy and media strategy will become the most important element tying a brands strategy in place.

The question is: How can buyers think horizontally across their consumer screens and add value to their vertical investments? We need to buy media both vertically and horizontally. Small screen buys need to have native mobile strategy attached that go beyond CPMs and drive mobile conversion goal into the cloud and into the mall.data, this consumer journey is more complex than we had assumed. We now know that we need to work harder to drive real shopper insights and shopper marketing goals. Data tools we can use now include location, screen insights and digital relationship building.

As the MultiScreen Summit Chair, I am so grateful to have such an remarkable group of media thought leaders onstage next week at the MultiScreen Summit, June 11th and 12th.  www.thescreensummit.com.

Samsung Abandoning Android? Tis Tizen?

The elephant in the room at the Galaxy S4 launch in New York last week was Android. Not one word about the Google OS. Is all well in Camelot?

Some industry pundits such as ABI Research analyst Aapo Markkanen say there is a clear and intentional distancing of Samsung from its existing OS partner, Android.  Does Samsung want to reduce its almost total dependence on the platform over the next few years? Samsung seems to quietly be building independently on top of the Android OS and may make a jump to a more neutral industry partner by 2016.

Window’s has not offered a compelling alternative to Android. What are other options?

Mozilla (Firefox) and Linux (Tizen) are going head to head to capture next generation developers with their web-based operating system. The Firefox and Tizen SDK and API allow developers to use web-based HTML5.

Tis Tizen

Tizen (a Linux Foundation initiative) may have the edge. Tizen is an open source, standards-based software platform for multiscreen devices (smartphones, tablets, netbooks, in-vehicle devices, and smart TVs). Like Firefox, it provides a cross-screen environment for application developers, based on HTML5.

Samsung abandoned its homegrown smartphone OS,  Bada, early this year and announced that it would start developing Tizen-based devices: “We plan to release new, competitive Tizen devices within this year and will keep expanding the lineup depending on market conditions.”

The implications are significant to the connected screen economy and place application development in a more mature web main-frame on the device. The application store now can exist in a more manageable web environment with bookmark apps and not get lost in widget-design interface promoted by Apple.

Interview: 10 Mobile Big Things in 2012 (BNN)

Business News Network 7.5 mins: 10 Mobile Big Things in 2012 Gary Schwartz, CEO, Impact Mobile, joins BNN to speak about the 10 big mobile things in 2012.

2012 “Top 10″ MOBILE trends & rankings

In a season where every second tweet and Google+ post is a look-back or forward at the “mobile” year, it is sometimes difficult to navigate all the insights. Just Google “Mobile Top 10 . . .” and you will find every blogger, publication and pundit providing their vision of what is newsworthy, trendworthy or simply rankable in mobile.

The proliferation of mobile “Top 10s” is a good thing. It is proof of all the many verticals areas that mobile has intersected. Health, IT, gaming, banking, retail, social, payment, fraud, security, privacy, patent trolling – all now have some form of a top 10 mobile list.

The explosion of the Mobile Top Ten list shows both how disruptive mobile has become and, also, what a massive audience it commands. Top 10-related news headlines seem to drive more hits and be retweeted more than other items.

(Twitter even tweets its top 10 most retweeted tweets. No. 2 being @LilTunechi Lil Wayne WEEZY F “aaaaaaahhhhhhmmmmm baaaaakkkkkkkkkk.” The 2011 top of the top 10 retweets was by Wendy’s restaurants “RT for a good cause. Each retweet sends 50c to help kids in foster care. #TreatItFwd.” Wendy’s raised $1.8 million.)

We seem to all need, what Perry Hoekstra in his blog calls an “Obligatory 2011 Top Ten Mobile Story List.” I presume it helps us simplify this expanding, complex world of mobile. If we can prioritize importance and cut off the discussion at ten, the list can help us make sense of mobile.

Lists also keep us honest. We never are right on half our predictions. Remember our 2010 trends and forecasts? They are out there in the blogosphere for all those with 20/20 hindsight to chuckle.

In December 2010, we all had an opinion on HP webOS-powered tablet plans – that never happened. And where on the list was Google’s purchase of Motorola, ostensibly as fodder for their IP wars against Apple? Missed that one.

But the biggest problem with these lists is their length. In the wasteland of eggnog and turkey dinners, I ask, where are the mobile Clifs Notes? Where is the super-list? Where is the list of lists?

Steve Yankovich, head of mobile for eBay, tells us that the mobile consumer’s attention span is roughly 15 seconds, nose-to-phone. EBay designs its mobile pages for the dwell time at an elevator or the time idling at the red light. Perhaps, in the spirit of mobile efficiency, we better design our year-end prognostications for this small attention-deficit window.

This holiday season, I add to the litany of lists my retrospect and forecast of mobile, but below in 100 words or less.

2011

  1. Facebook timeline set it back in history
  2. Google+ builds circle of trust
  3. Symbian sang swansong
  4. Apple continued to make data a free commodity
  5. Spectrum war showed its dark side
  6. All screens called mobile
  7. Amazon cloud disrupts the mall
  8. Consumer data became jewel in the crown
  9. CarrierIQ poster-child of privacy angst
  10. NFC wallet traded press, not payment

2012

  1. NFC proximity marketing, not proximity payment – yet
  2. X9 (ISO) delivers mobile security recommendations
  3. Apple says “I Do” to NFC
  4. Research In Motion exits stage right
  5. Cloud checkout is optimized and mainstream
  6. Super App = HTML5 browser
  7. Focus on prepaid market services
  8. LTE networks and spectrum become big issues
  9. Mobile privacy hype continues
  10. Microsoft and Nokia enter stage left

Not quite 100 words, but near enough.

Happy Mobile New Year.

Thank Google-ness, it’s Windows?

Mobility is a treacherous business. Cell phones are only used for an average of 18 months before dropping down toilets, falling off tables and general m-wear and tear drives the consumer back into the buyers’ market. For fashion and trend conscious consumer this cycle is probably shorter.

And mobile folk are a fickle bunch. While some may be diehard Apple or Android loyalist, the masses swap out the old for the new in rapid succession. Every time a screen smashes, a phone drops down a manhole or domiciles in a taxi (without you), there is a fresh opportunity for a handset manufacturer and an operating platform to woo the new shopper.

Of course, there is some stickiness. Consumers may be faithful to their Blackberry Messenger group, or the slick screen design of their iPhone, or the price point of their Android.

But when shoppers are back on the market they tend to be unchaste.

It is not just the device. Apple has made sure that the consumer is well aware that if you want APPs there is only one show in town. Developers worldwide are working hard to make sure that Apple and Android operating systems can market themselves and sell based on access to this wealth of content. RIM’s Playbook and Hewlett-Packard’s webOS never stood a chance without a preexisting content library.

Apple’s marketing genius continues to position the iPhone as the aspirational handheld, while Google’s open-android operating system on Samsung, HTC and Motorola continues to drive the bulk of smartphone sales. With Hewlett-Packard declaring defeat and Blackberry (RIM) and Nokia under siege, it looks like a two-horse race.

But recently things have become a little difficult to track.

Apple starts an IP war, litigating against Samsung and winning a stay of the release of its new TAB in Europe. Google’s purchase of Motorola, ostensibly for IP, suddenly makes the giant of the cloud a retail-bound cell phone manufacture.  This forces Samsung and HTC Android loyal hardware partners to reevaluate their strategy and possibly further diversify their operating systems.

The pressure of the looming patent war and Google’s unsettling role in the device market is opening the door for Windows reenter the market.

With the salvo of recent litigation, the Android operating system is turning into a liability.  Open-Android does not come bundled with an indemnification for the hardware vendors. Windows Phone’s does! This is becoming an expensive problem for handset manufacturers. Samsung and HTC have found themselves paying for their own legal defense and paying on any ensuing royalty settlement.

In the background, Windows has been quietly enforcing its IP rights globally. Goldman Sachs analysts estimated that Microsoft will make nearly half a billion dollars in 2012 on royalties of $3 to $6 per Android device from vendors like Samsung and HTC.

Microsoft’s “Android” royalties is estimated to be TRIPLE the revenue earned on Windows Phone licenses.

With all these mobile machinations, Windows 8 may be powering your new device pick on the shelf next year. Microsoft may allow for light computing with more robust utility than Apple or Android.

Next time your phone-du-jour falls down the stairwell or is left at the bus station, a phone powered by a Windows operating system maybe the new flavor of the day.