Real World Ajax - NYC

Real World Ajax in NYC, Monday 3/13 was a blast. I gave a talk on Designing for Ajax. It is a look at the most common patterns used by Ajax applications and the good design principles that underpin them.

You can catch a video interview with me about what is going on with the patterns, ajax and developer services at Yahoo.

I especially enjoyed

• Christophe Conraet from Adobe demo'ing the extra cool Adobe Ajax-Flex Bridge. I have already found an internal app within Yahoo! that this might be a good fit for. Essentially I can now mix Flex and Ajax freely on the page. I can manipulate flex (flash) components via JavaScript. And with the connectivity capabilities of flash you can do some pretty cool Comet style interactions (collaboration, white boarding, push style technology)
• David Heinnemer Hannson's Rails talk. David just keeps adding more and more cool stuff to Rails every day. His example of driving multiple rails applications via a flash bridge (again Comet-style technology) was really cool.
• Kevin Hackman of Tibco gave a very interesting tour of early Ajax apps in the enterprise (1999-2003). Its really cool to see what incredible work was happening before the "Ajax Christening."
• Scott Dietzen, CTO of Zimbra gave a great look into the world of Zimbra. I really like the Zimlets. Great way to extend their product. Scott, Ross and the others at Zimbra are a very talented team.

I also had interesting conversations with Backbase by CEO Jouk Pleister and Rob Gonda the new editor-in-chief for Ajax Developer's Journal.

It was good to touch base with JackBe founder, Jacob Derechin. The exciting news for them is they have added three really talented folks from Sun Microsystems (Deepak Alur, Dan Malks, John Crupi). These are the authors of the Core J2EE Patterns book. I had an great talk with them about the world of patterns. JackBe is an interesting Ajax toolkit for the enterprise. It is the only full framework I am aware of that can handle really low bandwidth/latency connections (and I am talking 9600 baud Ajax!)

So others have done a great job of summarizing. Here are a few articles on the conference:

• Compelling Discussion in NYC
• Ajaxian Article
• Wrapping Up Real World Ajax
• Backbase Article

Jeremy Geelan and Dion Hinchcliffe are great hosts. Check out their upcoming seminars in San Jose, CA in April as well as back in NYC in June.

All I Really Need to Know I Learned from a 128K Mac

On Monday I had the wonderful opportunity to speak at the Real World Ajax Conference in NYC. During my talk on Designing for Ajax, I mentioned how I got my start in the world of user interface and user experience -- writing the Macintosh game GATO in 1985.

Several attendees came up afterward and told me that they had played and enjoyed GATO. That's always fun to find someone who has enjoyed something you created.

One of those that spoke with me was Blake Patterson. I didn't recognize the name but after an email followup I connected him with one of his sites that I have frequented, ipodhacks.com. I also found out he runs ByteCellar a site that covers the history of early computing. Blake requested that I give some of the history behind creating the game and the experiences of early Macintosh development. I pointed him to some comments I posted on Andy Hertzfeld's Folklore site.

Blake wrote up a nice summary of those comments. You can read his article here.

After reading his article it really came home to me how defining the event of writing that game was in my life. Like the famous book, All I Need to Know I Learned in Kindergarten; I can say that really All I Need to Know I Learned from a 128K Mac. So what did that Mac teach me?

• Passion. I fell in love with computing on that puny Mac. I caught the passion of folks like Guy Kawasaki and Steve Jobs for this insanely great computer. Heck that phrase "insanely great" I continue to use to this day.
• It's Possible. Look when we started development we were green. We did not know the "C" language. We made horrible mistakes. It looked like we would never succeed. We had unrealistic deadlines. But one by one we solved our problems. And we did it.
• Event-Driven programming. The Mac introduced me to the world of event-oriented development. This underpins everything today. It is the re-introduction of the possiblity of finer-grained events via Ajax that makes it possible to build cool apps on the web.
• Object-Oriented programming & design. While C was not object oriented, the Mac had an object-based feel to it. It got me into reading the book on SmallTalk, the purest Object-Oriented language.
• Model-View-Controller. The powerful pattern that taught me the concept of separation of concerns, pub-sub concepts and the power of abstraction.
• User-Centered Design. The original PC GATO was divided into many screens. When we sat down to do our brainstorming on how to write a submarine simulation game we thought a lot about how submarine crews need access to their controls/panels. We interviewed a GATO class captain from WWII. That drove us to design a display that never lost the most essential controls. You could flip between a map and other less important controls but never lost the gauges, controls and dials.
• The Importance of Understanding the Deep Magic. What I mean is after GATO was released we went under the hood. We disassembled the ROM code. We documented it. We reverse engineered everything we could. We learned exactly how memory management, regions, event management and so on worked from the bottom up. This need to demystify the complex was first learned on my Mac.
• Where to Optimize. Learning from the master, Bill Atkinson. His drive to have a solid design approach and then optimize in the 5% of the code was a lesson I took forward into many projects in the future.
• Use the Tools. There weren't many tools available on the Mac. But we put MacPaint, MacDraw to use to create backdrops and vector shape files to create the graphical aspects of GATO. I have since always tried to use tools in novel ways to get my work done (see my VISIO toolkit as one example).
• Being Pixel Perfect. Design is made up of nuances. Apple always has gotten this. I fussed over every pixel in drawing the GATO screens (although I will admit the explosion routine was horendous -- it was created the night before our first public demonstration.) To this day it drives me nutty to see something one or two pixels off.
• Minimalism. Heck we only had black and white. It forced you to think about designs without having tons of colors and shading, etc.
• Principles of Interactive Design. Directness, Immediate Feedback, Modeless Design, and so on. Writing one of the first complex games utilizing Windows, Menus, Icons and Pointers (WIMP) was an exciting playground to learn within. We had the Mac & Lisa to thank for inspiration.
• Pay Attention to Real World Constraints. Hey it was a 128K Mac. Its one thing to design for the sky, its another to design for an extremely memory constrained system.
  

That's the list of things that immediately come to mind. I am sure there are others. For folks who did not live through the Mac & Lisa revolution I hope this little walk down nostalgia lane can capture the magic I felt when working on GATO and my lowly 128K Macintosh.

I should add that the other two developers/designers were James Rhodes and Sean Hill. James was (and I am sure still is) one of the finest developers I have ever worked with. Sean was a master at marketing and creativity. Sean later led the game unit at Sphere (formerly Spectrum Holobyte). As I understood it he was lead for the highly successful Falcon game.

Google Acquires SketchUp

About a year or so back I fell in love with an application.

At the time I had been pondering what should the ideal user interface prototyping tool look like. In my mind what I wanted was a way to doodle ideas and make those ideas come alive. Once alive I felt I should be able to quickly re-arrange the sketches and interact with them or allow users to interact with them.

The key thought was I don't want to burdern this interface sketching tool with requirements generation or code generation. I just want it for quick ideation. And I would like it to be approachable to those who are normally tool adverse.

In researching the space (at the time I had a desire to design, build and produce a product) I stumbled across SketchUp.

SketchUp was the perfect example of what I was struggling to envision. SketchUp was created by a garage startup to go up against the likes of AutoCAD. Its mission was to bring 3D sketching to the masses. I remember the first time I played with my son, who was 7 at the time, was having a little timeout on the bed next to where I was working. I heard a "WHOA, Dad that's cooool. Can I play with that?" Now I thought geez this is an awesome serious tool. Real architecture, landscaping, even game design can be done with this sketching tool and here my son who is only 7 sees it as a fun. And after his timeout, yes even he could do some cool stuff with it.

I remember having a conversation last year at the IA Summit in Montreal with Jesse James Garrett about this tool. At the time I was trying to gauge the need for a tool like this. He remarked that he had heard Alan Cooper talking about it being a great example of wonderful application design.

I forget where I read about it, but in an article describing how this little company (@lastSoftware) was able to withstand even AutoCAD's response, they said the real secret to SketchUp's success is "it is just a blast to use!"

And today that application I fell in love with just joined the ranks of Google. Congratulations Google! I am wondering what's cooking with MeasureMap (another favorite of mine), Writely and SketchUp joining the fold.

BTW, I still would love to see a SketchUp for UI prototyping. Wish I had time to work on it. Maybe someone will catch the vision.

Slides from eTech Talk

I gave my eTech talk today at 12:15pm. I followed an incredible sweeping history of computing by George Dyson.

I kept thinking, wow! how to follow someone like Mr. Dyson.

But the talk seemed to be well received. Most importantly several folks really got excited about the vision that I was putting forth. In a nutshell, a pattern library becames a vocabulary for a tribe. It becomes a nesting place for exposing solutions to help create a passionate design & development community.

You can get the PDF here (5 mb). If you want all of the animations here is a MOV version of the Keynote (3.6 mb). (Note: Links have been updated as of 3/9 12:24am PST. Prior to this they pointed to an incorrect version of the talk)

I had prepared the speech on paper. However, my extemperanous nature took over and I gave the talk from the gut. But just for heck of it here is the full text of the prepared speech.

The Written Form of the Talk (with the accompanying slide noted in brackets)
[interactive language of attention]
I was very fortunate to attend Kathy Sierra's talk on Monday Morning -- Creating Passionate Users. So many nuggets I am still ruminating on. But one thought in particular is apropos to our talk today. The "Power of a tribe" Its really about thinking about who your tribe of passionate users will be around your product or services.

Its not so much thinking about how to get their attention but how to foster a tribe. For once you have a tribe you have attention.

[attention. tribes pay attention.]
Tribe members pay attention to the tribe.

I was also struck by the quick peek we had into the second Life tribes. Wasn't that an incredible story about the tax revolt. Second Lifers setting property on fire to protest taxation. And guess what? It got attention. And it changed the rules.

[tribal platform.]
Yesterday, I heard Jeffrey McManus who leads our Yahoo! Developer Network describe the suite of web services as a "Participation Platform". May I stretch it a bit and call it a tribal platform?

[Yahoo! Developer Network.]
The Yahoo! Developer Network has been a shining example within Yahoo! (and we believe outside as well) of the importance of openness in this clamor for garnering the user's attention. It has been the backplane to our openness, sort of a canvas of openness on which we can express even more openness.

[internal tribes]
It has had a good social impact within Yahoo! Its platform of participation has created tribes within Yahoo! that are motivated to share and connect with tribes on the outside. And we have our tribe days too - called hack days. These internal mashup contests, the best Yahoo! Music Engine Plugin, the best Konfabulator (Yahoo! Widgets) day. These kind of tribe days captivate the imagination and attention of our best designers and developers.

In fact a good example of this is just yesterday Chad Dickerson, Edward Ho, Jonathan Trevor, and Karon Weber unleashed CheckMates here at the conference. You can check it out at chaddickerson.com. Its really just a tribal sort of thing.

[two recent examples]
Well with all of this tribal goodness going on inside the halls of Yahoo! it just had to work its way out into the wild.

Two very recent examples I would like to share is the release of the Yahoo! Design Pattern Library and the Yahoo! User Interface Library.

[design tribe]
So at Yahoo! I have two roles. One is as Ajax Evangelist. The other is the Design Pattern Shepherd. I am organizationally within User Experience. I am fortunate to work for the likes of Erin Malone and Larry Tesler. Both who believe strongly in the power of the tribe.

[design tribe. new models.]
Well, this last year has seen the rise of Ajax and the revival of JavaScript and Flash. We are no longer bound by the page refresh model. With this change, new user interaction idioms have surfaced on the web. The exciting thing about these new "patterns of interaction" is that they make it easier to create engaging interactions. And they also go a long way in making information relevant. Engaging interactions help get the user's attention. Relevant information goes a long way in keeping the user's attention.

[attention. engaging.]
Getting the user's attention is all about engagement. It's about creating a compelling reason for interest. It's about what we call the Wow! Factor. There are points in the user's interaction that are pure moments of opportunity. Opportunities for engagement. Opportunities for getting the user's attention. Moments for creating interest. Interesting moments.

[attention. relevant.]
But it's not just about getting the user's attention. It's also about keeping the user's attention. It's about creating relevancy -- creating more reasons for the user to stay interested. By having intelligence more near the user can experience a deeper delight with the interface. Not just an initial euphoric Wow!, but a deeper more satisfying delight.

[attention. loyalty.]
The final stage of all of this goodness is pure, unadulterated love. It's that "sit around the campfire singing Kumba-ya" kind of attitude. It is when attention turns into habit. The goodness of wow and delight blend together to create loyalty. The blending of engagement and relevancy.

Now here is where as an Ajax Evangelist I would normally make an altar call to come to this Ajax goodness. But I am not going to claim that a little Ajax powder is going to guarantee you a slice of the user's attention. All I can say is that there are new ways of interaction. And these new ways of interaction create the possibility to be more engaging and more relevant.

And that is what this design tribe is struggling with. How to communicate, design, document these rich interactions.

[immediate]
First, its about the immediacy of information, logic and presentation that can be brought near just in time. Ajax makes it possible to provide what is needed in the nick of time.

What immediacy brings to the table is the ability to be relevant. No longer does information have to be stale. Patterns exist that illustrate how to bring information near. Relevant, just-in-time information is attention-getting & keeping.

[direct]
And what about directness? If the user has to go through a lot of indirect steps to find information or accomplish their task, how long do you think you will hold their attention? But if you can provide direct editing, drag and drop and other desktop mechanisms the user may find themselves more confident and engaged in your site.

[inviting]
And how about the power of an invitation?
Nothing gets someone's attention more than offering an appropriate tip at just the right moment. And how about politeness? Doesn't that rank high on your attention meter? These patterns throw out the welcome mat. If done correctly, they provide relevant instruction just when you need it. Enticing the user to engage.

[without boundaries]
Want to lose the user's attention? How about erecting hurdles for them? How about making them jump unneccessarily from page to page and turn a simple process into a laborious one? That was the old model. It centered around page-to-page interaction. But these patterns bias staying in the page. They help keep the user engaged by not giving them a point to decide to bail out.

[light footprint]
Users will lose interest if tasks seem to hard. The newer styles of interaction center around keeping a light footprint. Making it easy for the user to interact.

[cinematic]
And don't forget animations (in measure). Transitions can communicate relationships. What has changed. Symmetry of action.

[rich in objects]
And finally, if all content has the same level of richness, what kind of interest will be created? Little or none! But take some content and turn it into an object. Now its bloggable, shareable, searchable. Rich Internet Objects like trip planner make it fun to create content. Similar work is being enabled by projects like microformats.

[pattern list]
This is just a few of the themes of interaction that have emerged recently on the web. And these are the patterns that make up much of these interactions you now see on the web.

This study of what patterns were emerging on the web and what patterns were emerging across Yahoo! are being catalogued within the Yahoo! Design Pattern Library.

[pattern library]
Recently, 2/13, we released a public facing version of the Pattern Library. While it currently only has a handful of patterns in its initial release, we plan to fill out this library over the course of the year.

Why? It's really about surfacing a vocabulary. It's about creating a language around getting and keeping the user's attention. It's about ways to create Wow, Delight & Love.

Every good tribe has a language. This is one dialect of the design tribe's language.

So when you create a language connotations arise and metaphors and similes and double entendres arise. Lots of various birds come to roost here.

So its also about creating a nesting place for all of the other ideas that surround design solutions. It's about understanding and discussion the context of a problem. When to use a pattern. How to use a pattern. And the rationale. But it's even broader. By having a library, a language we can talk about getting and keeping user's with special needs. We can consider accessibility concerns. We can consider internationalization issues. Issues with degradability. All the things that affect getting and keeping the user's attention.

And even more exciting from its branches we can hang the fruit of code. Good and honest implementations of the patterns.

[developer tribe.]
So the the new model also affects the developer tribe. In this case it is a struggle to find the best code solutions to interaction problems

[developer tribe. yahoo! ui library]
For this tribe we have released Yahoo! User Interface Library.

[another ajax framework?]
So why another framework? Well, inspired by the acquisition of Oddpost (now Yahoo! Mail Beta) in 2004, Thomas Sha started the development of a Yahoo! Ajax framework in Jan/2005. As it flourished internally the decision was made to expand the tribe.

Once again the belief was that by using the Yahoo! Developer Network's Participation Platform we could share and benefit at the same time.

And we see a real synergy between the various tribes: mashups, designers and developers.

So in conclusion, the real purpose of all of this is to surface a vocabulary and expose solutions so that the language of the tribe is more visible and the path to full tribal membership is clearly marked.

[surface, expose]
The patterns are important because they surface a language. By exposing services and the code that can implement the designs we are providing a few tools that may help you get and keep your user's attention.

eTech Happenings - Monday 3/6/06

I am at eTech this week. I will try to blog about some of the events as I observe them.

Creating Passionate Users
I attended Kathy Sierra's workshop this morning, Creating Passionate Users. Wow! What a treat.

I went back over my notes and found it very hard to summarize what I had heard (since there were dozens of excellent thoughts to chew on.) But here are a few of the nuggets I came away with.

• People want to "kick ass". As Cooper says, "User's don't want to appear stupid." It's important that a person be able to move from the "I suck" phase in mastering anything to the "I rock" phase.
  
• Constant learning potential is the key to passion. The promise of mastery and its benefits along with continual stimulating nuances to a craft make for passion. It's nice to have a long growth potential.
  
• The brain has a crap filter. We have to speak to the BRAIN (physical, chemical responses that can't be help) rather than the mind. For it is in speaking to the way we are wired that we can get to the mind.
• Conversation trumps formal lecture. The brain engages in a conversation. It goes into stasis in a lecture.
• Do what it takes to get users past the "I suck" phase into the "I rock" phase. You can motivate users to do this by providing:
• a way to recognize expertise
• a meaningul benefit
• a clear path to getting there
• Learn from game design. The cycle of build interest -> challenging activity -> payoff.
  
• Learn from screenwriters. The heros journey.
  
• Life is normal
• Something happens to change that
• [helpful sidekick shows up]
• Things really suck
• Hero overcomes bad things
• Life returns to normal-- but the character is changed.
• Ask two questions when you are writing.
• What will be the user's journey?
• How will the protaganist be changed.
• Dignity is deadly. Once you try to appease the critics you will be driven to mediocrity. If you are doing something write someone will criticize.

And the final thought is when writing, designing a user experience, etc. it is not about you (the author, creator). It is about how users feel when they interact with your organization/book/blog, etc.

Ruby on Rails
And speaking of passion, I got to hear my friend David Heinemeier Hansson give a workshop on Ruby on Rails.

I have loved this language and framework since the first time I played with it. I still do. I just wish I had some time to do something with it. Note to self-- make time!

Lots could be said, but one thing that stuck out was the concept of "partials". In Rails you can express snippets of HTML as partials (read templates). You then have access to this snippet to include in your code. Kind of like JSP tags only a thousand times lighter and in native HTML.

You can also create rjs files (rails javascript). Its a way to create javascript with a ruby on rails syntax being able to access the DOM via rails variables or by selecting elements with CSS selectors. Then there are page manipulation mechanisms to apply animation effects or hide, show or replace html. What's cool is the code is generated on the server side in the controller where it is natural to think of accessing application state. It controls what gets generated on the client side in JavaScript land.

There was also some interesting discussion of using Microformats as a data source. David is working on ways to transform a microformat into your application specific HTML markup. Interestingly I did a blog interview with a group from the UK along with Kevin Marks of Technorati who is doing a lot of the work with microformats. We also talked about this over dinner. It is somewhat the argument for JSON. JSON expressess data. JSON is native to the browser. Microformat express data (albeit in XHTML). Microformats are native to the browser. So in both cases the load is light on parsing and using them to render views.

Bruce Sterling Keynote
On the Internet of Things. A look at the visionary world of when all things are acessible via the "internet". And by all things I mean all things. Like your keys or your shoes. Imagine, he says "googling" for your shoes :-)

Its the first time I have heard Bruce talk. He is an amazing speaker. An incredible wordsmith. His approach to writing science fiction is to first write a visionary technology manifesto then craft a story around the emergence of the technology into the future.

Sites to Check Out/Books to Read
So here is the list of interesting things I want to investigate further:

Books:

• Flow: The Psychology of Optimal Experience
• Mind Hacks : Tips & Tricks for Using Your Brain (Hacks)
• Creating Passionate Users (coming summer 2006)
• The Wisdom of Crowds
• The Media Equation : How People Treat Computers, Television, and New Media like Real People and Places (CSLI Lecture Notes)
• Shaping Things (Mediaworks Pamphlets)

Sites:

• eeggs.com
• ipodlounge
• Maker Faire
• instructables
• second life
• iconbuffet
• fluxiom
  

By the way, I will be giving a 15 minute talk to the conference about the Yahoo! Patterns & Code Library. That will be Wednesday, 12:15pm.

Open Yahoo!

I've had a few people ask us why we went public with our libraries.

Having been intimitately involved in the bringing the pattern library live I can tell you first hand the simple reasons they have gone public. And these are good reasons for others to do the same.

• Goodwill. Sharing back with the community always pays back in rich dividends.
  
  
• Feedback. Lots of eyes and lots of hands make the libraries better. We recognize that there are a lot of smart folks outside the halls of Yahoo!
  
  
• Dialog. Brings issues to the forefront for discussion. Like accessibility for design; common design libraries and language.
  
  
• Reuse. Best corporate reuse strategy is open source. I saw it at Sabre with Rico. Same at Yahoo. Already groups are using the libraries more now that they are public. And it energizes people to participate in something bigger than all of us.
  
  
• Standards. If some of our better design ideas spread then it is good for our users and in turn good for us. Relearning different ways to do the same thing is only fun for so long :-)
  
  
• Mindset. Yahoo! has a lot of folks in management, design & engineering that have benefited greatly from the open source community. This gets in your blood. It becomes part of your value system. So it becomes a natural reflex to do it this way.
  

That's basically the reasoning. No big surprise.

And yes, we don't share everything. We have lots of really awesome research going on. Really cool stuff coming. Sorry that won't be in the next release of the libraries ;-) But I think that is understandable.

Yahoo! Libraries Released - Ajax & Patterns

I am excited to announce that just a little while ago, we (Yahoo!) released two new (previously internal) libraries to the public.

The Yahoo! Design Pattern Library is being released under a friendly Creative Commons License. The goal is to share common patterns we see emerging on Yahoo! We hope that by opening up our design patterns that we can both share our thinking and garner your feedback.

At the same time we are releasing the Yahoo! User Interface Library. This is an industrial strength Ajax/DHTML library that we are currently using in lots of places all across Yahoo! Its got drag and drop, animation, event management, position handling, widgets, etc. And its all under the BSD license!

And we are announcing a new blog for both of these libraries. The Yahoo! User Interface Blog discusses both of these libraries in detail. You can choose to subscribe to all of the articles there, just the design side or just the development side.

And finally, we have a Yahoo! groups forum for discussing patterns and the UI library will be part of the current Yahoo! Developer Network JavaScript forum.

Go check it out!

Fun in LA

LA Conference
A few days back I had the delight of speaking at the Adaptive Path, Designing and Building with Ajax along with Jesse James Garrett and Dan Saffer down in LA.

First, thanks to all the wonderful designers and developers I had the honor of meeting. Its been a while since I spoke in a venue like this and I thoroughly enjoyed it. It seems that you also felt the conference was a wonderful success. It is my hope that the mental framework that we laid out for understanding the world of Ajax design & development will really provide an ongoing boost to your success. Don't forget to share those successes with me! Love to hear them.

Second, thanks to the AIGA for their wonderful mixer at the Westwood Brewing Company. So many creative and talented folks in LA to learn from even in the most casual setting. And, wow, the IxDA F2F at the Liquid Kitty was awesome. Thanks to Danna Hudson for getting this together.

So what did I take away from my time there?

• Subtle design. I met Ian who works in motion graphics. He taught me a little about cinematic transitions & effects in the world of commercials. One rule of thumb he mentioned was when deciding on the amount of color saturation or cinematic effect, he generally chooses what he thinks looks right, then halves it. This rule has always stood him in good stead. Hmmm. Perhaps we should apply this to flash & Ajax effects.
  
  This follows the rule of thumb I learned several years back from Kevin Mullett's book Designing Visual Interfaces in which he describes a technique for removing clutter. Get rid of all but the most essential information for any screen. Then slowly turn up the "contrast" knob, dialing in more visual treatment. Stop just when it communicates.

• Documenting rich interactions is still way too hard. Ok, I already knew this. Just re-inforced this fact. While we covered several real-world approaches to documenting Ajax applications, designers and developers still struggle way too hard in this area. Fine-grained interactions (interesting moments, microstates) must be documented in context of the page and flow. And all of these have to flow along the time dimension. Its just a hard problem.
  
  Powerpoint, Captivate, Breeze, SnagIt, Visio, Photoshop, Dreamweaver, Keynote, Illustrator, Fireworks, Word, Quicktime, SnapZPro, Director-- each has its pros and cons but nothing is really crafted for our field. My next step is to check out Microsoft Sparkle. Will it be the holy grail? Or is it just a slippery slope into Redmond? :-)
  

• Teaching with visuals and a mental framework really works. I put a lot of time into my part of the presentation (spanned 4 topics). I have been ruminating for months on interesting ways to think about Design Patterns and Ajax Operations. I wondered if I could impart what has been rattling around in my brain. It seems (based on survey feedback and talking with attendees) that it worked. I love Keynote. When you use transitions or animations they are cleaner, more elegant and when used right reinforce what you are teaching.
  
  This is true with interfaces. A good mental model and using a visual language for the purpose of communication works for user interfaces and for presentations.

• We have much to learn from each other and from other disciplines. I met people with backgrounds in graphic design, animation design, motion graphics, information architect, classical architecture, software engineering, and so on. Each come at the problems from a different angle. I have a list of books that I am going to plow into that I am looking to inform my design skills from orthogonal directions.
  
  This caused me to reflect on the wealth I have around me at Yahoo! in our User Experience Team. My manager is an awesome photographer (and designer!) I work with a PhD in Rhetoric (and professor of writing at Stanford), a lady who designed the animation tools at Pixar, a guy who designs complex strategy board games (like Settlers of Caton) in his spare time, visual designers (like Luke Wroblenski), a classically trained architect, software engineers turned designers, talented musicians, and so on. Wow! what unique perspectives live inside the heads of those around us.

Soon, a City Near You?
So, hopefully we will be in a city near you. Looks like Austin and NYC or Philadelphia might be next. Keep an eye out at adaptivepath.com.

A few notes on upcoming speaking engagements:

• Feb 6 - VeriSign Developers Summit (private), Miami, FL
• Mar 2,3 - Yahoo!'s Front End Engineering Summit
• Mar 13 - Real World Ajax - SysCon - NYC
• Mar 21 - Eclipse Con - Panel - Rich Interaction Toolkits - Santa Clara, CA
  
• Apr [TBD] - Designing & Building with Ajax - Adaptive Path - NCY or Philadelphia (tentative)
  
• May 10 - Adaptive Path - Designing & Building with Ajax - Austin (tentative)
• May 11-12 - Ajaxian Experience - San Francisco
• June 7 - Adaptive Path - Designing & Building with Ajax - Amsterdam (tentative)

If you are at an event that I am speaking, please catch up with me so that we can chat. I love to hear from folks who read my blog.

Nine Tips for Designing Rich Internet Applications

Recently I was asked to provide input into a presentation. The question was asked, if you were going to provide some quick basic guidelines for designing rich applications what would they be.

Here were the nine thoughts as they originally came to my head.

1) Make it directly interactive
Instead of page to page interactions think direct interaction. Use in context editing as much as possible. Use drag and drop only where it makes sense. Barring a selection model, put tools as close to the objects being edited as possible. Cooper states it as "Where there is output, let there be input."

2) Make it inviting
Use hover to invite users to the next level of interaction. If the interface responds well to light events (like hover) it can be used to entice the user to interact.

3) Use lightweight, in-context popups instead of page transitions where possible
Although they will eventually get over-used, lightweight popups can be your friend. Think of them as annexed areas for your page.

4) Use real-estate creatively
As mentioned popups help. But slide outs have long been allies in desktop tools, they can be an aid in the world of the web.

5) Cross page boundaries reluctantly
Think of a page switch as a context boundary that the user may or may not want to cross. Think of it as a place that many of your users will lose interest and no longer follow you.

6) Create a light footprint
Make it extremely easy to interact. Rating movies or news with just a click on a star with no-refresh is awesome. Checking hostnames without leaving the page is an excellent way to keep a user engaged. Shopping by clicks that only add to a container on the page (instead of going to a new page) are like impulse aisles in the grocery store.

7) Think of your interactions as storyboards
As the designer you are the director. Think about the event states as acts in a play and your interface elements as actors. Get them all moving towards telling your story. Putting the frames down on a storyboard is a great way to rehearse your script. Think of the interesting moments as opportunities for engagement.

8) Communicate transitions
Keeping the user informed during lightweight operations (that don't leave the page) with spinning wheels, busy or progress indicators keep the user engaged with a living page.

9) Think in objects
Instead of thinking about content and pages, think about Rich Internet Objects. The travel log in Yahoo!'s Trip Planner is a good example. Once created it can be searched for or shared. This will help you create more interactive applications and make the user's work recognizable and sharable.

These are not exhaustive. Even as I go to publish this I can think of other tips to include... but I will resist adding to the list. Perhaps you have some tips/principles that have helped you solve design challenges?

Designing and Building With Ajax

In December, Adaptive Path started a new series of conferences specifically addressing how to design and engineer rich Internet applications with Ajax. At that conference Ben & Dion (ajaxian.com) did a wonderful job of presenting the world of Ajax from the technical perspective.

I am excited to announce that I will be following in their footsteps and will be presenting along with Jesse James Garrett, Dan Saffer and Ryan Freitas at the Designing and Building with Ajax conference in LA on January 26th and in Miami on February 8th.

Based on feedback from the first conference we are revamping some of that curriculum and I am very excited about what we will be presenting.

So who should attend? Well we are desiging the material for both designers and web developers. If you are solely focused on design there will be lots of great ideas and tools presented that will enrich your work. If your bag is development, I aim to whet your whistle with some technical goodies as well as attempt to de-mystify the world of Ajax and DHMTL. And if you wear both hats-- well you will have a double-treat.

Why should you attend? Well, I think one really unique aspect of this conference is that instead of hearing 5 or 6 different speakers talking about individual topics, we are weaving a story through the day that is meant to build a mental model in your head to help you take a problem from conceptualization to design to implementation. The other unique aspect is presenting design and development in a wholistic manner. This is the context in which problems must be solved. This is also a great way to really understand the technology-- in light of our user's goals and needs.

Oh and one more thing. At the conference in LA, there will be some exciting news announced!

Would love to see you there!

Interesting, Seducible, Teachable, Engaging Moments

James Govenor pointed out a connection between my blog about Interesting Moments and Kathy Sierra's blog on Is it Interesting? (see also her discussion of the tie-in). Her angle is about engagement and delight.

Also, Joshua Porter from User Interface Engineering pointed me to an article by Jared Spool from back in 2002 on The Search for Seducible Moments.

As a parent I learned a long time ago there are times when a person is especially receptive to learning. These are called Teachable Moments. If one is prepared for those times, more can be accomplished in a few minutes that a lifetime of nagging would ever accomplish.

Putting these thoughts together, we should look for the interesting moments of interaction that open the door for us to seduce and engage our audience.

I have written about some of these moments in general. Like the thought of providing an invitation to the user through lightweight events (like hover). Getting users to review a movie by making the interaction with the movie rating control fun and engaging is one example.

I am curious, what sites, products or features have you found engaging-- or even seducing?

Distracting or Narrowing: Looking a Little More At Live Search

Ajax opened the door to immediate actions. One of the first types of problems that immediate action was applied to was in the area of search. Here are a few examples.

Google Suggest
Google Suggest was an early demonstration of the concept of Live Search. As the user types in the Google search box, the top search terms that match the user's input are shown in a drop down. If I am looking for 'Tom Cruise' only 5 characters are required to match this search term ('tom c').



I have never used Google Suggests for anything more than a demonstration of how live search works. I rarely need help in finding search terms based on four or five characters I have typed. Don't get me wrong, I think it is wonderful code, snappy response and perhaps has some purpose. I just have never found it personally useful. I mean if I am going to look up Tom Cruise, I perhaps might find it useful that it completes it for me and spells it correctly. But I think I know how to spell Tom's name.

Here are a few thoughts I have on this.

• If I am typing a term I probably already know the term I am typing
• The drop down can be annoying as it distracts me from typing
• It is really only useful for common searches. Since I find myself typing fairly qualified search terms; and there the suggestions start to trail off rapidly (at the end of the long tail)
• Once I find matching terms I just get a second pile of results. So in essence I have searched twice.
  
• The usefulness seems to be if I just want to see what is hot that starts with a few characters ;-)

I am just not sure how often I am searching for search terms rather than search results. Did it really help me narrow on results or just distract me along the way. (Hey sometimes distraction is intended for a purpose-- don't see its value here.)

If you have found Google Suggests useful please let me know. I am sure I am missing something. I would love to understand common use cases in which this is the case.

Yahoo! Instant Search
Yahoo!'s Instant Search takes a different twist on this. A balloon popup shows up with a single best match (based on Yahoo! Shortcuts). Kind of like a 'google suggests' for 'feeling lucky' only your lucky day is on Yahoo!

Typing 'tom cruise' yields the following:



Ok, the difference here is a candidate result is shown based on what I have typed. But here are the caveats.

• I have to type fairly specific, complete search terms. 'tom c' does not cut it. 'tom cruise' does. And do I really want the imdb database result for Tom? Interestingly, the Y! Shortcut is his fan page, which is what I was looking for in this case.
• The balloon can be distracting since I am personally conditioned to multiple results to look at. Giving me just one seems to make me think harder about whether this is the right place to go. I tend to compare results before picking a term to drill into.
• With the balloon in place, hitting enter just takes me to the results. Which seems reasonable but there is amiguity about where I am going when I hit Enter. Ctrl+Enter is supposed to go to the link in the balloon. But on the Mac with Firefox it does not seem to work.

I think the instant search is a slightly better idea than showing search terms that match the characters I type. Perhaps if Google Suggests showed synonymous search terms around what I am trying to search for (hard problem) it might be more useful.

It is interesting that the instant search provides a nice interface for those with screen readers. The popup triggers visibility and is read by the reader. Those without sight can know the best result found instantly. Nice side effect.

Now I am on the fence overall about its narrowing capability vs. the distraction it causes me. For popular items it seems to have a 50/50 chance finding me exactly what I want. However, I tend to be someone who is searching for more obscure items so to me it is distracting (though cool!) The fact that it narrows to 1 item causes it to have this sharp distinction between distraction vs. narrowing.

Do you like or not? If I understand correctly, it did well in user tests. What say ye?

Kayak Auto Complete
Kayak.com is a good example of live auto complete. Say the user is wanting to fly into New York city. They seem to remember that the airport they want is called LaGuardia (not sure how to spell it or what the airport code is). When I type 'New Y' I get the following results:



Ok, this is nice. I hate remembering the airport code. And I hate having to go to another page to find out that the site could not figure out what I meant. Jakob Nielsen states as one of his ten usability principles, Error Prevention. Providing the feedback instantaneously to cue me to what the system thinks is great. Saves me time and makes me feel like I am narrowing in on the right thing. This probably ties in with Jared Spool's research on The Confidence Game which shows that users are happy to click and click as long as they feel confident they are reaching their goal. Here I have the best of both worlds. Raise my confidence and do it with the minimal effort!

Hey Travelocity, FareChase, Orbitz, Hotwire, Expedia-- this is good stuff! Get this feature in your sites!

Without a question this helped me narrow to my result and did not distract me.

Of course I have compared apples and oranges. Both Google & Yahoo are dealing with an enormous body of information that it is trying to give you instantaneous feedback on during your typing. Much to infer. Kayak has an extremely limited (though hard to remember) set of data. Perfect for an instantaneous narrowing opportunity.

Yahoo! Mail & Why it Dumped LiveSearch
The new Yahoo! Mail Beta (still in limited release--and no I can't get you in the beta! :-) has a nice search box for finding content in your mail. It works extremely well. When you search for some content you thought came into your mail messages, the results show snippets of the conversations with the matched text highlighted.





Now you will notice that the search box has a little "Go" button beside it (and when activated, the an inline progress indicator shows). So how can I call this live search? Well it really isn't. However, they started with a live search in the Alpha version. The user would type a few characters and then the search would trigger. However, when the product went beta they went with a more traditional "submit" model. In other words instead of key events (or losing focus) implicitly triggering a search, the user has to now hit the "Go" button to get a search initiated. Why the change?

The primary reason was performance. Supporting the scale of users within the Yahoo! Mail network would be an enormous hit on servers. However, I think there is also something to be learned from this on the user experience side.

Now this is not scientific. It is my observation. But when confronted with a live search box that will return results (not suggestions) I immediately expect a button to trigger it. So at first I am a little unsure. Ok, that can be worked around by learning the interface. But next as I start to type a string, say "liberty mutual", it would start matching 'liber' finding things about liberals, liberality, liberty and so on. Now it starts changing as I continue to type. I get through the first word 'liberty' and now I find mail that discusses liberty in the U.S. and elsewhere as well as finding an email from my insurance company. Finally I type the whole thing in and get what I am looking for.

Now here is the point. How helpful was the constant feedback loop?

Did the feedback help me narrow to the message I am looking for or did it distract me during the task of entering what the term I knew I was going to type?

I vote that it was distracting.

I feel like I am in a race with the instantaneous search engine to get my term in before it starts returning the wrong results :-(

Thankfully it was replaced by this more traditional (yet Ajax-based) solution.

Amazon Zuggests
Using the Amazon API, Francis Shanahan created a way (Amazon Zuggests) to search amazon using the similar approach of Google Suggests and Yahoo! Instant Search.



This is really nice since it shows the results to my search terms as I type. Since the results usually have cover thumbnails and some text, it is easy to use it to explore to find books on a topic without ever leaving the page.

But does it really help me narrow quickly? Well, yes & no. Since it seems to not try to search too fast (on my pausing to type) I don't get into the race I felt like in the early Y! Mail Alpha. I can type a word, say "football." What I get back is a bunch of books on football for NFL and college. What I really want is college football. So I continue typing, adding the word "college". Now I get a bunch of college football results.

By not narrowing too quickly (to one result as with Y! Instant Search) and not trying to return results to fast (like Y! Mail Alpha did), the feedback loop is nice and allows me to refine by typing keywords.

Summary or What I Think I Have Learned So Far

To kind of sum of the lessons I think I see here. Live search is useful when

• The results are direct results-- not indirect results (as is the case with Google Suggests)
• The results are narrowing and not diverging (as in the case of showing more search terms in Google Suggests)
  
• The results are narrowing and not distracting (as was the case with Y! Mail Alpha, Google Suggests and potentially Y! Instant Search)
  

There are other ways to refine search. Nice examples on the Yahoo! site include the new Yahoo! Shopping and Yahoo! Travel which allow you to select & unselect filters on the left and see the results on the right in real time. One of the early examples of this was on the kayak.com site in refining flight searches.

So I am interested in feedback. What other general rules am I thinking of? What other examples come to mind? What do you like or hate about live search examples?

Feedback appreciated.

Storyboarding Interesting Moments

In a previous post, I discussed the idea of an Interaction Matrix. I described its purpose as documenting the event states that occur within drag and drop, inline editing and other types of interactions.

However, the term event states is a rather dry and sterile way to think about this really important concept. In fact the term comes from the programming world (which I lived happily in for many years.)

Recently, I was at a talk where Nate Koechley presented about the new vocabulary of user experience in the world of Rich Internet Applications. He used the term Interesting Moments (which was coined by Eric Miraglia) to describe the event states within an interaction that are points of user engagement or interest.

I really like that term. Wish I had thought of it first ;-)

Thinking of storyboarding interesting moments within an application, an interaction, or a widget simplifies our thinking.

It actually turns event states inside out and focuses them instead on the user. It asks the question, "What is interesting to our user?" and "What is needed to engage them (invitations) and aid them (feedback) through our story?"

Nate also pointed out that designers have the role of director. As such we have a cast of characters (our interface elements) and the timeline (interesting moments) on which to play out our interface.

Let me know what you think...

Tags: storyboarding interactiondesign prototyping article

Visio Wireframe Toolkit for Download

In an earlier post I mentioned a wireframe toolkit I wrote for Visio to make wireframe creation go much faster. Specifically I discussed one aspect of the kit-- animating wireframes with Visio. I wrote a fuller article that is just getting around to being published on boxesandarrows.com. It should show up this weekend.

In the article, I provide a reference to the fuller wireframe toolkit and they (at boxesandarrows) have agreed to host the installation files. Since I have not really explained the wireframe toolkit nor provided a way to download it before, I felt it would be important to provide a blog article about the kit and how to get it for your own use.

Caveats
Please note. The kit was fine-tuned for our frameworks at Sabre (some of which became the openrico.org project). As such the code generation is only beneficial to those with that framework (sorry, not Rico). However, those that are industrious can open up the Visual Basic Editor in Visio and find the code that generates JSP and Java/Swing code and simply replace it with code to generate PHP, ASP or whatever suits your fancy.

Also, the style has been reduced to a mostly color-less variety suitable for wireframes. In the original we had our demo CSS style implemented in Visio. The color-less variety was a really quick hack I did to dumb it down.

I don't have any proper documentation. I have included a movie presentation on how some of the features look when someone is using them.

And finally, I have spent no time in the last 5 or 6 months refining it. Now that I understand how to create Visio shapes and manipulate them I would do some things differently. I keep thinking it would be nice to clean it up, fully document it and release it. That though was what kept me from sharing it before. So, here it is. Hopefully, some will find it helpful.

Wireframe Prototyping for Visio

Visio is widely used by interaction designers and information architects for creating wireframes. At my previous employer, Sabre, I created an extensive wireframe toolkit for Visio that included such features as:

• Complete web component stencil library (including standard components, table, menus, tabs, and tree)
• Fast, intelligent snap-to layout using pre-programmed Visio connectors
• Provide a way to visualize rich interactions (animating a wireframe)
• Intelligent components that have built-in behaviors
• Generate requirements documents from wireframe artifacts
• Generate code from wireframe artifacts
• Automated annotations and callouts

Download & Installation

The stencil library is available for download.

1. Make sure you have Microsoft Visio 2003 installed on your machine.
2. Download the zip file
3. Unzip this into your C:\My Documents\My Shapes folder. (My Shapes should have been created by Visio when you installed it.) You should have the contents of the zip directly in the My Shapes folder.

Now that it is installed, you can navigate to this directory and double-click inlineEditWireframeExample.vsd to experiment with the photo editing example. If you want to create a new Visio wireframe using the stencil library double-click the Wireframe.vst file.

Notes

• Make sure you allow macros to run (change your security setting to avoid the annoying messages... sorry I know how to fix this, just haven't gotten around to it.)
  
• The templates were originally built for enterprise web applications. Some parts may or may not be applicable to your application. I welcome feedback. At some point I would like to rework to be more generally useful.
  

Boxes And Arrows Article

Look for upcoming article (11/20/05 ?)

Presentation

• You can download a presentation of what the Visio stencil library contains. Download it as flash (4 MB) or as a quicktime movie (this contains all of the embedded videos -- 5 MB). I recommend the .mov version.
  

Enjoy!

Y! Trip Planner & Web 2.0 Content Objects

A couple of weeks back, Yahoo! announced Trip Planner

If you haven't had a chance to use it, I strongly encourage you to get your hands on it. In a nutshell, you can plan day trips, weekend trips, vacations into a trip plan. You can build the trip plan collaboratively with a group of fellow travelers or if you are a control-freak, all by yourself ;-)

At a later time I may blog about the details of the interface. Others have gone into lots of detail. And the team has lots of really cool features planned. On top of that, the team is full of talented folks and they are jazzed about this product. I am too.

Trip Planner is a good example of how an older idea can be transformed as it moves to become a Web 2.0 tool.

I used to use Microsoft Trip Planner. However it was limited since the maps and travel guide information were based on a snapshot on my CD. I loved it but it was severely limited in what it could give me as well as rapidly growing out of date.

There are a lot of things I love about this product:

• Collaborate on trip planning
• Share trips publicly
• Pulls trip planning into a single location
  
• Can find other's trips
• Uses the ever expanding Y! Maps products (map view)
• Has enormous tie in potential to other Y! products (watch for upcoming features!)
• I just love travel. Hey I live in the Bay Area. Lots of places to explore!
  

Web 2.0 Content Objects
But I think what has really excited me about this product is how it illustrates the power of Web 2.0 Content Objects.

In Microsoft Trip Planner the trip plan was a file on my disk. I could send the file to a friend and if they had the same application they could access it.

But think of the power of the web as a platform for these Web 2.0 style files. I can create trip planning content, share it publicly, and allow others (of my choosing) to collaborate with me on it. And once good trip plans surface over time, there will be less and less need for me to create trip plans from scratch.

This is what del.icio.us (and later furl.net and myweb2.yahoo.com) did for bookmarks. Shareable and findable.

Microformats
microformats.org is an attempt at defining the underlying data format for sharing user generated content. I don't want to discuss the merits of whether it should be in HTML or XML format. I just find the idea of movie reviews, restaurant reviews, trip plans, news clips, and a host of other content being in a format that is shareable, bloggable, findable to be really exciting.

Challenges for User Experience
So that leaves several challenges for us in the User Experience world. Here are just a couple of issues.

• Discovering Content Objects. Progressive disclosure (giving the user just the amount of features they need at the beginning and growing with the user) will help. Finding other trip plans while just searching around travel related items lead the user to explore.
  
• Getting folks to share. We all have suspicions about sharing information on the web. Do I want people to know where I travel, what I like to do? To some of us it is not an issue. To others that is terrifying. How do we create assurances (and more importantly we had better do the right thing with their data!) that there can be some anonymity? Trip Planner allows you to choose to share or not. But perhaps there should be a way to share anonymously. Have Trip Planner only post the outline of the trip and no user generated comments (only from other sources).
• Understanding collaboration. At Y! we struggle with who your community is. Is it Y!360, address contacts, IM buddies, flickr friends and family? And do user's really understand how they would work with other people (or how many will do this?)
• Avoiding feature bloat. We are building full-scale applications. Yet most user's engage us ligthtly. How do you make feature rich, discoverable applications? Providing "feature invitations" unobtrusively at the moment they need it.
• Getting users to understand how other tools might work together to solve their problem. Trip Planner uses Y! Local, Y! Maps, Y! Travel, Y! Search, Y! Travel Guides. It will use more properties in the future. How do you take a slice like trip planning across all of these products? Obviously you have to treat the other properties like services and build a complete application out of this slice. That leads to understanding how to present robust applications (without undue clutter, etc.) as a web application.

I'm sure there are other issues. As well as other solutions than those I briefly touched on above.

What do you think the challenges are for these types of applications? For these types of shareable content objects? Other examples you have seen?

The Blurry Line
And, one more thing. I notice that Microsoft has Streets & Trips 2006. Looks to be very full featured. It seems to pull its content from the web. This raises a number of questions about what is the right approach. Do I build a product like this as a desktop application or as a web application? Sounds like a good topic for another blog...

Animating Interactions With Photoshop CS2

Sometime in the next month my article for boxesandarrows.com will finally appear. It discusses using Microsoft Visio for animating rich interactions. I discuss this briefly in a prior blog.

Photoshop CS and CS2 introduced some new features that make it easy to simulate complex interactions including animated transitions.

Layered Comps
Photoshop CS brought us the wonderful world of layered comps. You can now name the layer configuration. The layer order, visibility, layer styles, and element positions are saved.

It is sort of like being able to take a photo snapshot of the current state of the design.

Its purpose is to create alternate designs and give them a name. Then the designer can walk through the different comp designs with a customer to choose the best one among them.

In the world of rich interactions, we can take a snapshot of the user interface at each interesting event state or user interaction.

Animation Palette
Photoshop CS2 brought an animation palette. By adding layered comps to the each frame of an animation, you can play the various layered comps. Think of it being able to play back a set of complex interactions.

A cool feature is the ability to create tweening between frames.

An example is I could have two rectangles positioned and named as a layered comp called rectangles. The next comp could switch these rectangles. This could be called swapped_rectangles. Dragging these two layered comps into the animation palette will create two steps to the animation: rectangles in their original position and rectangles in their final position. Not too exciting, right? Now select both frames in the animation palette and tell it to tween and allow it to create 10 frames. Now we have 12 frames total.

When you run the animation, notice that the rectangles look like they are swapping. The in-between frames make the animation look convincing.

Capture the Movie
Simplest way is to use a video capture tool (like SnapZ Pro for the Mac) to create a movie file of the interaction as it is played in Photoshop. The movie can be distributed to illustrate the interaction.

The combination of layered comps, animation, and video capture create a simple way to illustrate cinematic effects and complex interactions in a page.

Interaction Matrix

Recently, I have been exploring drag and drop and inline editing for the web.

At Yahoo! I am fortunate to work with some incredibly bright folks. We get together for brainstorming sessions to discuss interactions like these that have application all across the Yahoo! network.

When I started capturing our sessions and writing up documents for these two interactions, I was really struck with how non-trivial it was to define exactly how a drag and drop or inline edit would operate in every case.

Let me take drag and drop to illustrate my point.

With drag & drop there are at least ten interesting event states.

• Page load
• Mouse hover
• Mouse down
• Drag initiated
• Drag over valid target
• Drag over invalid target
• Drag over parent container
• Drop accepted
• Drop rejected
• Drop on parent container

And there are a number of elements that get involved in the interaction sequences. Let's call them our interaction actors. I count at least seven of them.

• Page
• Cursor
• Tool Tip
• Drag Object
• Drag Object's Proxy (ghost of object being dragged)
  
• Drag Object's Parent Container
• Drop Target

That means that there are 70 possible visual states or transitions in a single drag and drop sequence. If you eliminate ones that don't make sense (e.g., tooltip on mousedown) you end up with about 40 possible states.

So defining this in a meeting is non-trivial.

To simplify this, I created the concept of an Interaction Storyboard Matrix. For simplicity I created it as a Microsoft Excel document. Each row corresponds to an actor; each column corresponds to an event state. Each cell in the grid represents a different visual state or animation providing feedback or affordances during the interaction.

Here is what the drag and drop interaction storyboard matrix looks like:


Events and Actors Reveal Interaction Storyboard Frames

So how do you use it?

Just fill in the non-gray cells with how each actor will look or behave during each event state. The interaction storyboard matrix is just a way to show different visual channels against the backdrop of event interactions. Think of it as a choreographed dance and each state being a difference scene in the play.

Feel free to download both the drag and drop template and the inline editing template.

Let me know if you find this useful.

Special thanks go to my fellow Yahoos. Matt Leacock, Kiersten Lammerding, Lance Nishihira, Eric Miraglia, Rob Aseron, Kathleen Watkins, Eric Costello, Anja Krombholz, Ericson DeJesus, Greg Rosenberg and many others...

Popups With a Twist

If you lived through the desktop UI boom of the 80's and 90's, you will recall the history of popup windows. You will recall the rise and fall of their popularity. And now with the web we are seeing popups growing in popularity. Are we headed for the same problem? Or are these popups different? While I believe that once can create annoying popups, what we are starting to see on the web are popups with a slightly new twist.


Example of a Lightweight Popup from Yahoo! Local

So here is where I want to say (with a nice, slow, southern, raspy drawl), "Back in my day, we didn't have no popups."

But in reality we did. With the advent of windowing we kind of went popup crazy. We had moved from a world of ASCII terminal based applications that went from screen to screen (or if you were clever you could tile in an area of the screen using some IBM font characters to draw the popup border and create a simulated dialog) to a nice environment with menus, windows, mice, and icons. With that sudden power applications started adding popups everywhere.

In Alan Cooper's original About Face book, he used the term, Idiot Boxes, for unnecessary dialogs that interrupt the user's flow asking pointless confirmation questions or asking for information that they system should already have.

One key design goal that I have always strived for is to minimize modes, context switches and windows for the user to manage. Direct manipulation helps to realize this goal. But another technique is to avoid going to other screens or pages to get information from the user.

In the web world, for the longest, we were plagued with two types of popups:

• Javascript alert box. Your basic ugly dialog box
• Javascript open window. Which created another browser window (that you could leave fairly undecorated)

Later, two varieties of popups came on the scene

• DIV based lightweight layers
• iframe based layer

iframes provided some advantages and disadvantages. With iframes you can specify a separate HTTP connection and an iframe can cover native browser components (e.g., in IE/Win the combo box shows above any DIV layer and while you can mess with z-ordering to resolve it, other issues are introduced.) The problem with iframes is that they can be slow. The HTTP request and render cycle can cause a second or more delay to populate an iframe's content.

The use of a DIV that is absolute positioned and given a higher z-order index, styled with a title and perhaps made draggable has come into vogue of late.

And for good reasons.

• They allow the user to input more information without leaving the current page context
• They allow the user to look at more information without leaving the current page context

Notice the interesting twist.

On desktop apps, dialogs seem to be disruptive. In web apps, popups (when done correctly) can actually minimize the need for breaking the user's concentration by bringing in information in context quickly and easily. (Obviously abuse will not be far behind.)

In my previous post I illustrated the Hover Detail. Luke Wroblenski commented about using Hover Detail + In Context Tools to make more complex related commands readily available to the user without disrupting flow.

This is really intriguing! These really are not your father's popups!

Its the same basic idea. Just popups. But with a light twist. So what is this light twist that makes these web popups so much better than the old style? Here are some of the things that I think make them better.

• Their goal is to minimize disruption to the user
• They are virtually instantaneous. No waiting like in the old days.
• They can be styled to feel lightweight
• They can be styled to match the page they are part of (this makes them feel like they are part of the page, but yet not at the same time)
• They have enough familarity with normal popups that users know how to move and close them (hopefully!)
• They are often triggered with just a mouse hover and dismissed with a mouse out
• They remove the fear of where a link might lead me.
  
• They remove the need to load a new page
• They often bring information at the point of interaction and at the point of curiousity
• They remove the need for getting back to the context by using a back button
• They provide information in context-- a powerful information architecture principle.
• Couple with Ajax, they provide a world of information at the user's fingertip (or mouse point ;-)
  
• And they just feel great! No, seriously, after living in the land of page to page navigation it is refreshing to be able to pull content in quickly.

Several nice examples are in order.

Yahoo! Instant Search

Yahoo! Instant Search Uses a Lightweight Balloon Approach to Show Shortcut Matches

Netflix

Netflix Movie Details

MarketWatch

Lightweight, Quick but Substantive. From MarketWatch. Illustrates Information & Commands in a Single In Context Popup

Musings on Mouse Hover

I've been thinking a lot lately about mouse hovers. Yeh, I know I really should get out more often ;-)

A mouse hover is a really simple event. When you get a mouseover event you do someting. When you get a mouseout event you usually restore things to the original state.

Mouse hovers were virtually unheard of in the desktop user interface world. I remember writing some very sophisticated applications (games, graphical drawing packages, etc.) and only using mouse hover for some very basic operations. Tooltips & mouse coordinate feedback are two that come to mind.

It was a while before rollovers showed up. In the original Macintosh Finder and in Windows 3.1, rollovers were not used to indicate an object was actionable. It was the advent of the web and the need for discoverability that really opened the door for the lowly hover to be come to the forefront.

In the desktop world, hovers were not that immediately useful. It seemed gratuitous and redundant in those early days to do something on a mouse hover. Traditional desktop apps have always been structured around user interface controls providing the structure with data poured into this mold. Web sites & applications started with content. With hyperlinks and images embedded in the page, it was not obvious what the 'control' or navigation structure of the page was without some visual clues. Over time rollover effects that made it into the web found themselves migrating to the desktop.

Windows 95 flattened toolbars to remove the visual speed bumbs created by beveled button borders. In its place, the mouse hover showed the border and gave the essential clue that the object was clickable. As more and more folks have got used to exploring with a mouse hover, it became a more acceptable a way to discover functionality.

Ajax, DHTML and the Lowly Hover
Now that interactions are even more dense, the hover has become our friend to discoverability. We are introducing new idioms to the web space. Things like drag and drop and inline editability are not immediately expected. The hover can provide vital clues to the behavior of an application at the moment the user is curious about it. Hovers are also the lightest event for the user (they just move the mouse.)

Let's look at some places that hover is showing up more and more (not an exhaustive list!)

Hover Invitation
We can use the hover to cue the user what is going to happen next.

Inline editing
How would you know that parts of a web page's content is editable? We could be overt and just say it on the page. But that gets busy if there are lots of places that are editable. We could make everything look editable. But that also makes for a cluttered page. Hover lets us do just-in-time functionality feedback.

In flickr, hover over the photo title and you will see the color change to indicate the area has some interaction to offer. Coupled with the tooltip the user is invited to click to edit the title.


Flickr's Photo Title Before Hover


Flickr's Photo Title After Hover

Pretty obvious if the user ever hovers over the title.

In backpackit.com, the 37 Signals folks take a similar approach. They shad the area editable with a gradiated yellow background and add a lightweight inline, incontext toolset. It becomes obvious that you can edit, delete (trash can not shown below), etc. the content.


Backpackit Note - Non Editable


Backpackit Note - Showing Editability

In Yahoo's trip planner (travel.yahoo.com/trip) you get the same style of in-context toolsets on hover.


Y! Travel's Trip Planner - Itinerary Item


Y! Travel's Trip Planner - Showing Editability

Ratings
Yahoo popularized ratings. Just check out how the hover helps guide a user to rate a movie.


Yahoo! Movies Ratings - Not Rated


Yahoo! Movies Ratings - Invitation to Rate

By visually engaging the user in rating possibilities the user only needs to click to participate. This lightweight approach to engagement is the advantage the hover has. Hover requires little effort. The next step is just to click.

Hover is nice because it allows you to hide lots of controls and show them only as needed. Showing them in a light enough manner makes it much more engaging to participate.

One more example will suffice from Yahoo! Local's Reviews.


Yahoo! Local Review Rating - Before Hover


Yahoo! Local Review Rating - During Hover

Flagging Mail
In the new Y! Mail, hovering over a message's column for flagging shows a lightly shaded flag. Clicking turns the flag on.


Yahoo! Mail (Beta) - Unflagged Mail


Yahoo! Mail (Beta) - Inviting to Flag

I think gmail's star could have been greatly improved with the addition of a hover. Although I don't believe that Google believes in the hover event at this point in time ;-)


Gmail's Star - No Hover

In all of these the hover creates an invitation, a call to action. It shows what the user can do right now, right where they are at.

I call this the Hover Invitation.

Hover Zoom/Focus
There is another use for hover. Since the mouse can indicate where I want to look, the hover can say, hey I want to focus here... I want more detail.

Getting More Detail
local.yahoo.com does this with the map. Try searching for 'restaurants' in some locale and you will see a map to the left. Hovering over the map enlarges it. Leaving the map area shrinks it back to the normal size.


Yahoo! Local Map Inset - Before Hover


Yahoo! Local Map Zoomed - After Hover

This pattern is about Mouse Focus or Mouse Zoom. In this case the hover is the interaction. While it might lead to more interactions, the Hover Invitation is a specific call to action, an invitation to click.

However, there are problems with this approach. Once the area is zoomed up it can cover other important areas, controls on the page. The user also has to move a lot further to exit the now enlarged area to get it to shrink down in size. You have to be certain that the user really wants to interact with an object to zoom on hover.

This is also common with photo organizing software. Hovering over a photo automatically enlarges it slightly.

A very slight variation on this is enlarging an area on hover to make it more apparent it will be the object selected. The Macintosh OS X Dock does this. Although with mixed results. Part of the problem is the area is just magnified. If the icon is 32x32 and it enlarges to 64x64, you still only have 32 pixels of mouse movement to traverse the object. This causes people to zoom past tools by accident.

Hover Detail
And yet another variation is the hover detail. By hovering over something I get a separate view that shows up in a lightweight popup allowing me to see or read more data about an entity. Not too much unlike a the hover zoom, but with more emphasis on letting me peek at more information without changing the page at all.

You can see an example of this in netflix.com.

Netflix Hover Detail

Also, it is used to get more of the story in news.yahoo.com.

Yahoo! News Hover Detail

Going Forward
Of course, hover will be abused. Once could imagine a page with no controls showing and they only show their action on hover. This is the idea of mystery meat interaction a wonderfully hideous approach to engagement.

But used wisely, the hover can

• Engage the user at the point of interest
• Reduce clutter on the page
• Give specific previews as to what will happen if the user clicks
• Provide a lighter way to zoom in on things with little effort
  

Take-Away
Use hover to create a more lightweight but engaging way to cue the user to hidden functionality. Use it as a way to provide just-in-time details. Use it to keep your page visually simpler providing what is needed when they are most curious.