Reader Comments

Loving the Madness of Good Design: "Institutionalize us all"
– how to get your organization excited about usability.

After fighting just such battles for many years, I won't do it anymore. Seeing a job description with the words 'evangelize usability' or one of its many variants causes me to run in the opposite direction. In this day and age, any company that has a need for evangelizing just doesn't get it and I want nothing to do with them.
Sylvia Lowden, Metanoia Design

Great article! If you replace usability, UX, et al, with "ergonomics", what's in this article is exactly the battle and the approaches I've been using for some 25 years. There is light at the end of the tunnel, it may at times be only a candle, but it still illuminates.
Don Morelli, Donald L. Morelli CPE

"User Experience" meets "Beauty is Truth, Truth is Beauty"
– the foundations of "user-experience" design.

I dig your new year resolution. Don't we all love a good kaleidoscope where science meets art to create beauty. All the best in this coming year with your new resolve, Eric.
Vikram Chauhan, UTV New Media Ltd

Where is a good chart when you need it?
– Guidelines for creating online decision tools.

Challenger was not a "debacle", it was a tragedy. No chart will substitute for management's lack of backbone.

Having said that, yes, there is a huge difference between data and information. Showing people data does not substitute for giving them information.
Ann Pinion

Interactivity is what gives the details in your Treemap (Depth of field) example. Interactivity supports evaluability - additional information can be given on request.

In his classes Tufte emphasizes that information comes to life when people can explore it, and discover what's most interesting to them. Here, too, interactivity is key.
Marc Schluper

Do we always need data to make a decision? As the sign on the wall in Albert Einstein's office read... "Things that can be counted don't always count. And things that count can't always be counted." Perhaps the high level folks at NASA and a few other places need to read that sign?
Evan Rolek

Can you avoid political fallout from your standards effort?

But wait. Those users are all human. They all have only one focus of attention apiece. That's why we want the mechanisms to stay out of the way. We don't go to the theatre to watch the projector. Indeed, if we ever notice it, it detracts from the content.

See Raskin's summary of "The Humane Interface" at Nitpicker.pbwiki.com and note that we can only multi-task if all but one of the things we're doing is automatic so our attention can remain on the other. Ban modal dialog boxes. They guarantee to break your train of thought.

Don't demand credit card numbers without blanks, they are on the card for a reason, it's easier to get it right that way. Don't even have a save button. You mean if I don't remember to click on this button, you're gonna take my twenty minutes of work and throw it away? That would be cruel. Don't do it.

I like Lean and Agile, but that only gets you bug free code, you have to measure and test to get code that doesn't get in the way. Jef shows how, but nobody seems to have noticed yet. When they ever do, they'll find that the result is absolutely addictive.

Wait till you see what happens when you accomplish links by rollover in an auto-zooming world. It will take over the WIMP GUI if it ever gets into use at all. Pie menus haven't yet, either.
Richard Karpinski

This sounds very familiar for every organisation I have worked at.
Richard P.

This article has some good information. Parts of it seem a little confronting to Eschenfelder.
Carli Connally

It's interesting that Dr. Schaffer refers to "older adults" as "they" instead of "we". ;) Is it because WE older adults who have technical knowledge don't think we need simpler interfaces? That's not what the article indicates.
Ann Pinion

I would think that the "more errors" is more the result of rigid exploration techniques than the exploration taking place after the errors. Otherwise the erros would indeed indicate less technical knowledge?
Dana Reynolds, Walmart

I loved this article. I have an intense interest in this subject, being one in the older category and having three adult sons who seem way more savvy with information technology. I was glad to see that at least people in my category often were able to persist and get successful consequence, even if their user experience was less satisfying. I'd love to see more articles like this and more research along these lines. Excellent!
Christine Van Brunt, Paychex

I am very interested in your findings. Some of it seems intuitive. However, the title suggests that differentiation of results was determined by both age and experience. It is clear that your control groups had different age ranges. However, how was their experience level "qualified" for the participants? Or – in fact, is it merely presumed that the older age group had lesser experience with the technology. As a correlation to that, I think "'older readers" could take objection to results. The essence of the findings may be more about familiarity and experience with the tools than it is about age.
Dale Sauro

Very interesting study. I believe that just about everything I've read about designing for "older adults" can also apply for designing for younger adults. Providing strong feedback and letting users know they have erred – and providing information about how to correct their error – is important no matter the age of the user. As far as web applications are concerned – I believe older users have an easier time with applications dealing with information they are familiar with – such as financial accounts or health insurance – than younger people. Providing meaningful feedback and education on errors – both technological errors and information errors – is helpful for both groups.
Ashley Rovenski, Humana

Great experiment and presentation of findings. Cheers!
Brian Jordan

Reality is that people vote on emotion, I call it the emotion-effect. The problem with the emotion-effect is that folks don't think clearly and logically; they tend to listen to emotional rhetoric and therefore vote purely on their emotional state.

So please vote with what I call the "logical-state", and not get caught up in the emotion-effect. My 2 cents! :)
Danny Heise, NIH

I think the real reason polls are off is because people don't like to be bothered. I don't answer the phone if I don't recognize the number. I've never been approached for an in-person poll - I would probably refuse because I don't want to take up the time. I really don't like polls - the only one that should be taken is after all the votes are counted on Nov 4.
Dee

I just wanted to say thank you for the recommended reading list. I thoroughly enjoyed Presentation Zen and put the lessons learned to immediate use - quite successfully I might add. One member of my team commented: "By far the best PPT I have seen come out of IMX. It says a ton without using a bunch of words. Cool."
Robert Wolverton, Intelimedix

I read Stumbling on Happiness recently and found it very interesting to say the least. Thanks for the recommendations; I'll most likely look into Presentation Zen and maybe The New Unconscious if I'm feeling brave.
Mike Wheaton

For those interested, the elements of SUCCES referenced in the Zen book are fleshed out in the very readable Made to Stick, a bestseller from 2007 by Chip Heath and Dan Heath.
Gary Pendergrass, L-3 Communications

– How visuals can draw and drive attention.

One of the main goals of any ad is to draw us in as it competes with all other visuals for our attention. Since direct eye contact is very powerful in drawing spectator interest, the first ad might actually be more effective. The spectators who study it might pay relatively less attention to the product, but there will be many more spectators who notice it than the other alternative as a direct result of its direct eye contact. I'm not sure how the research was conducted, but based on the eye-tracking heat images, it appears spectators were probably presented with the ad and nothing else. Another interesting research would be to present an audience with a variety of ads or a complete web site that includes this ad. We can then try to weigh in the number of participants who fixated on the product of this particular ad in each of its two alternatives. I believe the results will be able to approximate real life experiences even closer.
Raz Levinhar, Amdocs

While I find these results very interesting, what I do not feel was addressed is the key question of "which one resulted in more sales?".

Did the image of the woman looking at the audience appeal more to men? Does looking at the product generate a better response with women?

I feel this article speaks to how designers can elicit a desired behavior, but I think it is incorrect (based on the information disclosed) to imply effectiveness of the marketing piece.
Mitchell Heller, Heller& BAC, Inc.

I think it would be interesting to see an additional study on where the user would look if the model was looking at the product and the product was shifted up with the tagline below it.

Just for grins...
Jeanne Wilson

The article by Kath Straub is fascinating. I attended her class in Austin at the Government Technology Conference and was very impressed with her and HFI.

The article "Look Composed" reminded me of something I have been thinking about: Is marketing getting too good? At what point does psychological manipulation cross ethical lines?
Ann Pinion

I think this is a very interesting observation and worth application. In fact studies have been made in advertising but this kind of user response "proved and tested" is definitely worth all the money and resources spent on user testing.
Laura Dsouza

Very interesting study. That's what every director or creative designer should be aware of. If they understand all this, I am sure the ROI for their client will definitely increase.
Prashant Poladia, Monior Group

Very interesting. Informative and captivating. Keep up the good work. Appreciate more such information.
Anand Nair, GAC

What if you gave some of the people the placebo and told them they were part of a control group and would get a placebo? Maybe there should be a real control group that gets no drug or information (yeah, we just abduct them and apply the electric shocks! :-)

Are there any studies to indicate how long this effect (perception based discrepancy with reality) will last? People repeatedly buy the expensive cold remedy but what about in a long term, continuous usage situation. I suppose as always, some do, some don't.

What might be the evolutionary value of this kind of behavior? Do we have this phenomenon to thank for things like courage, honor, morality, religion, political parties, ...?
Leo Hartman, Canadian Space Administration

I love to apply these articles to the strategies of my side business outside the software world. I am a professional musician. My colleagues and I always hear comments about how expensive it is to hire music groups for events. I could charge a lower rate and I would still be as good. There will always be people who can not afford what I charge no matter how low it is.

In some cases, the packaging is the only thing that is different, but that can be enough to sway my purchase decisions.

Interesting article as always!
Diane Albert

Like the old saying goes: "Perception is reality."
Don Demrow, Citi

Amusing and enlightening...thanks for sharing!
Bonni Dennison, TD Canada Trust

The paper on testing multiple design alternatives is intriguing but it also raises a lot of questions. Usability testing's primary purpose and strength have always been about observing how successful people are in using a tool. Asking them how easy it is to use something is notoriously unreliable. The paper seems to put a disproportional weight in comments.

Also, people will learn about the non-interaction based aspects of a task while they perform them. Things like verbiage, capabilities, domain knowledge, even getting a better understanding of what's being asked of them. What they learn on the first task (first design alternative) affects their confidence and knowledge when performing the second and third tasks (other design alternatives). I couldn't find how this bias was addressed.

Lastly, I'm surprised that the authors were expecting solid design suggestions from the participants. As they state themselves, usability tests produce findings: what works, what doesn't, perhaps why. You still need an interaction designer to translate these into solid design improvements.
Zephyr

Response from Kath Straub:
Ah... your observations are quite sharp and insightful... and certainly ones that serious practitioners should be considering, addressing, and adjusting for in their own work. After all, usability testing should be a form of controlled experimentation... If only applied research could be well designed laboratory research...

But the pendulum does swing both ways. At what point do we worry that controlled, balanced experimentation shifts us away from ecological validity in our findings? After all, as I explore a site (in my real world), I do get exposure to it... and that exposure has an impact on how I execute subsequent tasks.

To arrive at a (reasonable?) local minima, it's important that practitioners understand and apply rigorous scientific method to their testing designs. That is your point, right? And we agree on this point. I would be curious to know what proportion of the active user experience practitioners today have taken (and still remember the key points from) their experimental design course(s)?

That said, the point of the paper – at least the one we are hoping practitioners will take away – is that testing multiple designs (against each other) can yield a very different outcome than testing a single design (against itself?).
End of esponse from Kath Straub

Great article on health care online. Very timely for a large project that we are doing right now. Thank you!
Jennifer Blake, Sutter Health

Good article. This confirms trends observed by other research on the same topic. The Mayo Clinic is another source used by consumers seeking health information from a neutral third party.
Hector Fernandes, Premera Blue Cross

The information was useful & I quite agree on the shift in trend from 2000 to 2005.
Subhrajit Majumder, Web18 Software Services Ltd

I think it's debatable as to whether the doctor really does know best. If you are talking about an allopathic paradigm, yes. But is that the best paradigm? For many people, the answer is no. A more holistic medical practice recognizes that treating symptoms through surgery or pharmaceuticals is often not the treatment of choice... educating the clients to change their lifestyle is far more effective in the long term.
Don Child, Health is Your Destiny

I wonder how much of a time gap there were between when users played the virtual racing game and actually took the Vienna Risk Taking Test. I would speculate that if the users took the Vienna Risk Taking Test two weeks after playing the game (instead of right after), their risk-taking behavior would probably be equal to those who played the soccer game.
Thanh Nguyen, BusinessOnLine

Yes, I agree with the Author(s) findings. Especially in the paragragh discussing facing one's fears through graphical intervention or positive role playing. You know, just look up the word "playing" in the New Oxford American Dictionary.
Eric Stark, U.S.P.S.

I am curious how people who play shooter games or strategic games in which players build their own worlds and fight other worlds would score in the Vienna Risk Taking Test.

That is, is there a correlation between aggression and risk-taking? It is also curious whether people's personal risk-affinity in games is correlated to the results in the test. Do people who take great risks when playing such games take similarly great risks in life.
Dimiter Simov, Lucrat

The neural pathways for simulated or imagined activities are very similar to those for the actual activities. using those pathways over and over builds efficiencies that makes those actions, attitudes, and emotions more natural. The Army has been very successfully using simulations for many years in order to overcome soldiers natural resistance to killing other human beings.

Watching violent media and playing violent video games has also been shown (via fMRI) to reduce impulse control and critical thinking.

This effect is far more pronounced in children who's mental processes are more flexible than adults. Of course many media producers, including the MPAA and video game manufacturers, deny that media affects behavior.
John Moore, Human Experience Design

How timely. I was just thinking about how "games" can be substituted for risk management in-service programs and what type of games could be developed to do this. Do you know of any that have already been developed connected to risk management?
Glenna Schindler, Healthcare Services Group

I have for years read the debates over which number of users is statistically significant, and yes the minimum of 5 has always been a safe bet. But seriously I don't see how increasing the number of tasks is more beneficial. I am a designer for applications for which I generally test a scope of tasks per feature and there is no need for more tasks when my goal is to a test specific set of features that can be completed with a finite set of tasks. Also the article references sites; in my experience the types of tests we perform to validate the usability of a site are often different than the types of measures used to test the usability of product applications. I think this distinction needs to be made.
Shilpa, HP

Can you please comment on the selling of this idea to clients – three groups of 6-12 participants. This would be helpful because every different user groups you recruit adds to the cost. Is it advisable to separate out as common tasks across groups and special tasks per specific group? Somewhere it also connects to the maturity of usability practice within an organization. Your recommendations can help practitioners in companies sell this to their own management or clients. If an end-to-end software solution provider needs the bandwidth to address usability in projects, are there more automatized tests or techniques that can be provided?
Raju

Instead of placing users into arbitrary categorizations based on age, wouldn't it be more efficient to address this as an accessibility issue and not just a problem of "old" people? I'm personally offended by the implication that just because someone is "old" their vision is failing or they're becoming less intelligent than others who are less "mature". Yes, there are users who are visually impaired, have learning disabilities or possess motor control issues. But those things are not a condition of age! I feel these age classifications are poorly conceived and the author could have done a better job.
Dan Ford, Sigma Micro

Was it really necessary to label the age groups as "newly old" through "very old"? Wouldn't just specifying the age ranges have been enough? I stopped reading the article at that point because I found the labels offensive.
Lauren Hansen, JPMorgan Chase

In the section "So who's really in charge here?", I can recall tons of stores that use this method. McDonald's; having the inside unappealing to the customer so they will not stay, making it the "get and go" environment they seek. Americus Diamond; I can't walk in there without feeling the romance due to the music which results in me buying something for my girlfriend :O!

Businesses also tend to use the state-dependent memory method. Businesses will have an advertisement of happiness which tends to bring back memories of times when we were happy. A good example of this might be Kodak. The way they have brought the "Kodak moment" to people. The information you guys provided was very insightful – thanks!

p.s. I can explain EXACTLY why I made each purchase decision... I am a teen, who will eat anything/everything. :)
Matt Floyd

While it may be possible that the subconscious mind plays a role, the word "unconscious" by definition means that it is not aware at all and therefore cannot play a role in decisions.

Even to the extent that the subconscious is involved, it does not consist of something separate and mystical apart from the conscious mind. The subconscious consists of conscious decisions that are automated.

In the case of the German wine, if you had had German wine before and despised it, it would be in the realm of your consciousness and no amount of German music would make you consider it.
Wendy

Excellent information !!!!!!!!!!!!!
Martina Lexner, IBM

Closely related (but approaching it from another point of view): the "top ten" book of Barry Schwarz "The Paradox of Choice" – a must-read for those interested in choice and decision.
Paul Hubert Vossen, I-SQUARE

I enjoyed the write up about trust on the Web. My sense is that usability certainly plays a big part in trusting a company when you visit their Web site. After all, if you can't find what you need and understand what you read on a Web site, you're certainly going to wonder whether the company that created that site knows or cares anything about you!

But I think trust on the Web goes beyond navigation, layout, fonts, colors, and content. I often look at testimonials from customers (with real names), and I look at whether the company offers any personal information about its executive staff. That's something that I really appreciate about the HFI Web site – the "big wigs" all provide their pictures, along with friendly, personal information about themselves. This makes them look and feel like real human beings who actually care about something more than profit and loss statements – they care about people. And I can trust someone who cares about people!
Martha Roden

Interesting piece, thanks for putting it together.

I think some general principles about the nature of trust may help here. In my work (sources cited below) I have found useful the Trust Equation--a formulation of the components of being trustworthy: (Credibility + Reliability + Intimacy) / (Self-orientation).

In this approach, credibility has to do with issues like clarity, reputation, credentials, transparency – things that help us believe what is being said.

Reliability, by contrast, has to do not with words but with actions. It talks about track record, predictability, breadth of usage, and again reputation.

Intimacy has to do with security – am I comfortable sharing this information with this person, will they know how to treat it, will they know when it should be passed on and when held confidential, discretion, sensitivity.

But the fourth factor – the one in the denominator – is the most powerful. It has to do with motives. Why are you doing what you are doing, and to whom are you paying attention? If the answer is, yourself, then I do not trust you. If the answer is 'me,' then to that extent I do trust you.

Most of those traits – the one exception being reliability – are traits that affect interpersonal relations, not institutional relations. Trust is largely human and personal, not corporate.

This suggests that the ideal Web site would have some combination of the following attributes:

1. Clear in its logic and visuals
2. Annotated, sourced, linked
3. A track record of past success
4. Testimonials that are identifiable
5. Clear statement of benefits to users
6. Transparency about economic model of the seller or web host
7. Customizable
8. Transparency about all information
9. Easy access to all models of communication (e.g. links to email, phone, blog, online)
10. Make comparisons available to comparable sites

Others I'm sure will draw further implications from the general principles. (I am a speaker and consultant, co-author of The Trusted Advisor (Free Press, 2000), and author of Trust-based Selling (McGraw-Hill, 2005). More at www.trustedadvisor.com)
Charles H. Green, Trusted Advisor Associates

One of the major drivers of trust is customer satisfaction. When your company does not deliver the product as seen/promised (as is the case in many e-commerce sites), users lose trust in you, however user-centered your design! What is crucial to remember here is that trust is not just limited to the interface but spreads across everything related to the product.
Meghana

I'm surprised, or maybe not so surprised, to see no mention in this article of source citation as a trust factor in Web design.

I think this poses an interesting question when it comes to online content in general. Do design factors compensate for the legitimacy of information when it comes to trusting health, financial, and other research information on line?

Too often I think Web designers sacrifice citing sources for design. I appreciate that this Website provides clear citation for its sources.
Laura Damkoehler, ELM Resources

This is very interesting! Two comments:
(1) There has been some research conducted on the effect of "frame" on decision-making (i.e., whether the question is posed in a positive or negative frame). My colleagues and I are currently involved in an experiment to determine the effects of frame on decision-making.
(2) In the context of Ho and Tam's study, the "negative" tone deals more with "you should buy now, 'cause you are about to lose something you'd like," rather than a bad/negative attitude on the part of the seller (or agent). We all deal with the pressure of a "negative" pitch to buy, when we ask ourselves, "shall I buy this item now, since it's the only one of its kind available? (...and someone else may buy it out from under me if I don't get it myself...)" and/or, "shall I buy this now, 'cause the price will be higher the next time I come back?" Note: Amazon.com seems to have implemented this "negative pressure" quite effectively (and not offensively) with their "Gold Box" strategy.
- Thanks for the food for thought in this article!
Bonnie Hautamaki, Alion Science and Technology

So, it's an interesting study, but since when is usability the same thing as pushing sales? To me, there's clearly a difference between making it easy for users to pursue what they want/need/expect from your site and applying sales pressure. I feel like if this is what I'm interested in, I should be reading self-help books that sleazy sales people might read to trick people into buying. I'm sure the same principles will apply. Ick!
Diana Ryan, Rockwell Collins

I thought Susan’s article was fantastic: a concise, timely and relevant summary of academic research that may influence future directions in our field.

I wanted to also point out that in market research in Australia, at least, they are starting to use neuroscience to gauge reactions to things (e.g. advertisements) in much the way that Susan alludes to in her response to Question 5. See this link, for example.

I have concerns about the usefulness of such data and the quality of the resulting analysis (much as I do with eye tracking) but I thought you might be interested nonetheless.

Finally, thank you for a consistently great read.
Jessica Enders, The Hiser Group

One thought on the generalizability of the West and Lehman paper. They used SAS employees from marketing and UI design groups. I realize they did this for expediency's sake. However, I wonder whether less techy, communications oriented people would provide as good a qualitative feedback in their typed in automated test comments. I was little suspicious when I saw a few of the verbatims seemed more descriptive than what I might expect test participants from the public at large might say. I might not expect references such as "easy to find the dialog" and "in order to commit the changes" from the public at large. Also, at least in the examples given, some were very descriptive task focused - almost what I'd expect a usability engineer to do when describing a problem. It'd be nice to see the study done with a broader recruiting profile.
John Bierschwale, Quest Software

People looking for another perspective on "QUESTION #4: USABILITY TESTING = THE THINK-ALOUD TECHNIQUE?" should read relevant sections of "Blink" by Malcolm Gladwell.
David Robertson, Petro-Canada

– how too much educating is not always the best way to get your point across.

It is interesting having looked into this urban legend, however I approach this theory from a marketing point of view and argue, it's readable and diffferent than what the brain is expecting to see, so it draws the eye. I'd be interested in feedback regarding the theory that because the text is different than what the brain expects, the eye is more likely to be drawn to the text.
Sarla Holmes, Gordon Institute of TAFE

Kath, thanks for addressing this issue. This email has been bugging me for a long time. Your third point, about the "adaptability of people" is the reason poorly designed interfaces are still with us. A very well written article, Ron.
Ron Sova, Sova Consulting

Useful article. Many times in my field, QA Usability Testing, I hear this same argument– "but the design works", "we can find the information", "it works as designed". Strenuous arguments including logic don't always convince. A couple of examples of ways "they" might have trouble using the interface do work! As my late, great grandma said, "It depends on whose ox is being gored!"
Elianna James, NetLibrary

That darned language study:

I have actually sent debunking e-mails once or twice. But the debunking e-mail has to be right. The usual version of the "study result" is that as long as you leave the first and last letter of each word in the right place, the order of the other letters does not matter. If you stick to sentences made up of short words, this is quite difficult to disprove with an example – with a four-letter word, only one "wrong" spelling is possible. You have to use an example with long words to show that the length of words can have a sacgnnnftiiit [significant] effect on the practical value of the conclusion.
Peter Biddlecombe, KODAK Dental Systems

Here's something I agree with violently...
Howard Tamler, Amdocs Product Group

Good article. Dr. Schaffer makes a good point when he relates the jumbled words to business consequences. I think a jumbled interface will have a negative effect on the interface, very unprofessional.
Kathy

– users' expectations of where items should be placed on your Web page.

I feel "Login" and "Logout" are also significant links worth a mention. To speak about my own experience, I expect the "Logout" link to be located in the top-right corner of the screen, similar to the Gmail Web site. I think it is very logical for us to expect the "Logout" in this place because we think and read from left to right. May be for an Urdu Web site, the user expectation will be exactly opposite.
Narendra Singhal, Reliance Industries, Ltd.

This follow-up study is tremendously helpful for practitioners. I'd like to see "login" added to the list of common Web objects studied in the future. Thanks for this very applied research.
Michelle Bejian Lotia, University of Michigan

A simple tool for helping others understand that it's not where they "think others will think" site visitors will look – but where they *actually do* look. Making the science of usability easy for project stakeholders to understand as critical to design success is job #1!

Thank you for sharing this.
Rebecca St. Martin, Web Sites That Fly

Anticipating people's expectations is the hardest part of creating an intranet site. I have help developed and organized some for both Deloitte and Arthur Andersen, including working with Jim Voorhies at Deloitte. There truly is a fine line between location, graphics, content and speed. All four must be in play.
Christopher J. Dove, Baker & McKenzie

When an Enterprise application needs to be developed, it is absolutely imperative that the Menu or internal links should never be displayed on left or right. It should always be in Top [Horizontal], this gives lot of real estate and a flexibility to improve the usability.
Chetan Murthy, ArisGlobal

Thanks for this great article. Sometimes it is difficult to persuade a creative client that his idea for a "cool way to set up the navigation" may not make users very happy. Over and over, I have pointed to your research for support. The best design is often the most obvious, clear design. Thank you!
Kimberly Kubalek, Kubalek.com

Hmm... having the 'participants' paste in items is, I believe, less precise than than using eye tracking to actually see where they look when you prompt them to do a particular task. Maybe these results have not changed because of your methodology where you got what you expected to get?

Thanks for the food for thought.
Scott Lary

Participants used in the research were undergrad. students. The research tell us the nature of the structure those sites have. Result for 'About us' are bit surprising. Maybe it comes from students looking at more college and university sites where they do have HTML links at the bottom. Back-to-home is on the spot. Wondering if the researchers probed further to tell us if it was the logo.

I hope people looking at this research would understand the user profile and not generalize the research.

I am not sure about the number look. What are we referring to here.
Prasad

– the interaction of Aesthetics and Usability.

Interesting article. "...a beautiful interface that seemed like a good idea at the time becomes less appealing if it's not usable... However, when getting stuff done matters, perceived usability – judged through usage over time – is what matters most."

The one thing that isn't mentioned is the other side of this coin, which goes to the heart of the differences in approach between usability work and design work. If a site that is imminently usable but unattractive to the user is compared with another site that is equally usable but more attractive, which gets used more?

Until usability practitioners and designers can join forces to create the best of both worlds, we continue to fail our users.
Jim Voorhies, Deloitte

As a Chief Creative Officer of an interactive agency, we're trying hard to identify PrEmo models and try to see how impact on a mood once an action is taken. As interactive projects are more reactive than any other communications pieces, I wonder how can a system be built that responds to users moods...
Sergio Barrientos

One of the great things about Don Norman's book is that he speaks of emotional responses (not of beauty) as something that happens at 3 different levels. These levels exist along different time periods and thus means that while visceral emotional responses can be tested in most usability studies, behavioral and reflective emotional responses occur over a much broader time frame and thus a lab setting does not afford this level of testing, unless you repeat and compare testing results over time.

Now I would also argue that aesthetics exist at many levels within a User Experience and the presentation layer is but one piece. I recommend people check out my "Whiteboard" column in the May/June 2005 edition of ACM <interactions> on the complexity of aesthetics in interaction design.
David Heller, Synaptic Burn

– recent research on the effect of designer gender on Web site design.

Thank you for the gender/age/education differences insight into design. As an educator of high school students this is a key point in helping me and my students decide how to present critical content is our school web page projects.
Brian Kerr, Vancouver School Board

– the importance of effective detailed design.

I found this article very interesting. I have experienced the Window-to-five set-up often in such areas as skills selections on Web sites, as well.

I definitely agree with Dr. Schaffer. The use of a particular presentation method has to be dictated by the user mental model and the data taxonomy (AKA the content model). It seems that some designers simply build a list of data items without thought to categorizing them. The repetition of seemingly identical data points in these instances will cause even more frustration. Thanks for the information and for letting me put in my 2 cents.
Michael Bradshaw, Indian Health Service

Right on! Unfortunately there are tons and tons of poorly designed Web pages (and interactive forms on all sorts of other, non-Web platforms) that suffer because of such poor selection-list design. Little things DO matter, a lot!
Tony Austin, Asia/Pacific Computer Services

Yes, we can never again make that selection casual. Sure, the study shows surveys can be biased. But this can be said also in the case of a general opinion topic. But in case a user has to select his education level – he has to anyway go to the right option in the radio button one or in the scroll down window. So there are so many dimensions to design, and each design solution is a unique case altogether and very difficult to standardize.
Laura Fernandez, Times Group

This article was very useful for me! I am teaching the Usability Essentials and told my class about this new research finding on labels being even more relevant to navigational usability than structure. Thanks for this!!
Marjolijn Verbeek, Capgemini

– the relative importance of user-centered labels and structure.

Actually, except for the state streets that run diagonally, Washington DC is a logical, diamond-shaped grid with each quadrant mirroring the one opposite to it. The Capitol is in the centre of the diamond. Horizontal streets are alphabetized and vertical streets are numbered, all starting from the center point (the Capitol). So the Capitol is boxed by 1st and A Streets NE, SE, SW, and NW and the letters and numbers increase accordingly as you move further away in any direction.
S. Dwyer

– How to select the best menu presentation style for a given application.

It should also be noted that when users interact with folding menus that require the extra click to see all the choices, they will be less inclined to try menu items which they are less familiar with. This does not make them better users because they are not regularly experimenting with new ways to improve there routine. It can also be said that as they continue on a routine with limited choices they may in fact get so efficient at it that their speed may rival the better ways of doing it. So this may be a mute point.
John Smith

– Evaluating the "test-five-users" guideline

Regarding the "Is 5 Enough?" debate in your newsletter.

If you are the only interaction designer and usability professional in a company of 400 people, AND no one really thinks usability tests are important or wants to free up additional resources to conduct the tests... then believe me ... 5 is definitely better than none! Five people still find more usability problems than zero people.
Martha Roden, Usability Engineer, CoCreate Software Inc.

As always, I enjoyed the newsletter. I guess I can understand the frustration with the recurring question of sample sizes for usability studies, but it can be an important issue, and the various explorations of it have enhanced our understanding of some of our practices (or at least led to some interesting discussions).

I've ordered the Faulkner paper, and am looking forward to reading it. It sounds similar in method and results to a paper that I published in 2001:

Lewis, J. R. (2001). Evaluation of procedures for adjusting problem-discovery rates estimated from small samples. Journal of Human-Computer Interaction, 13, 445-479.

I was a little surprised that you didn't mention the paper that I published in Human Factors in 1994:

Lewis, J. R. (1994a). Sample sizes for usability studies: Additional considerations. Human Factors, 36, 368-378. Then again, I shouldn't be surprised because it doesn't get mentioned that much. I don't promote my work as well as Jakob Nielsen, and Bob Virzi was the first to publish the results of a Monte Carlo study on the topic (HFES Proceedings in 1990, then the HF article in 1992). I also think that I might not have received as much mention because my message wasn't as optimistic or simple. Basically, the message was that the sample size you need depends on the problem discovery rate (p). If p is large, then you don't need very many people to discover the problems available for discovery. If p is small, then you need a larger sample. An ROI simulation indicated that the appropriate target for the proportion of problems to discover also depended on the value of p, with higher values having a break-even point at around 98% and lower values of p having a break-even point at around 86% (it's important to keep in mind that these values might depend on the assumptions made for the simulation -- but I think they are still informative). The data I reported in Lewis (1994) had p=.16, so to get to 86% problem discovery, I needed to run 12 participants (I actually ran 15). It's interesting (perhaps coincidental) that this is the number that Dr. Schaffer suggested in his part of the newsletter.

Another finding from Lewis (1994) that might have been discouraging to practitioners was that I didn't find any relationship between problem impact (or severity, using a behavioral scale rather than observer judgments) and frequency. In other words, my data showed severe problems being discovered at the same rate as non-severe problems.

Something that people always seem to forget (and this is something that I heard Jakob Nielsen try to drive home at a UPA panel session) is that the use of small-sample usability tests is bound to iterative testing. If you're not iterating, then small samples don't make much sense. By the time you finish testing, you should have tested a relatively large number of participants -- not just five! The reason to run multiple small samples (especially in the early stages of evaluating a design) is that certain problems emerge early and there is often general agreement among human factors engineers and developers regarding the reality of the observed problems and the need to fix them. Why would you continue running participants when you know in advance that they have a very high likelihood of encountering the known problems? Continuing to test under those circumstances is potentially wasteful of resources, slows down your ability to find out if the fixes you have in mind will actually fix the problem and not create any others, and there might even be an ethical consideration about needlessly frustrating participants.

As you repair problems, the likelihood of discovering additional problems should fall until the problems are appearing almost at random -- what I sometimes think of as problem foam. The early problems seem to have more structure and consistency than the later problems (which will usually, almost by definition, have a lower likelihood of occurrence than the early problems). Thus, the more mature a product becomes, the larger your sample size will need to be to discover the remaining problems (but, hopefully, there are fewer problems available for discovery -- and I don't think that any of us believe that it is possible to discover all usability problems). This suggests a strategy of starting with small samples, then increasing sample sizes as you continue iterating. Another advantage of shifting from small to large samples is that as you approach the end of the development cycle, the importance of global task measures (task completion rates, task completion times, user satisfaction measures) becomes more important -- especially if you have set up targets for these measures to use as a stopping rule for the iterative test-redesign-retest process, as larger samples lead to more precision in these types of measurements.

It's a shame that practitioners sometimes latch onto simple but misleading rules of practice. Thanks for tackling these topics in the newsletter.

James R. (Jim) Lewis, Ph.D., CHFP Senior Human Factors Engineer, IBM Pervasive Computing Division

– Can thoughtful information design help users make better consumer choices?

This was a great issue. Case in point, compare the Toyota car site with the Honda car site. I was looking for a car recently, and mainly want to choose between Toyota and Honda. The Toyota car site allows me to rank all their models vs. cost or mpg or seating. The Honda site only shows price in a comparison screen. Living in LA makes mpg critical information. I felt much more inclined to go buy a Toyota car because I feel like their site gives me the decision criteria I need in an easy to find format.

Toyota: click on the top bar Vehicles, then Model selector
Honda: and click on the Honda icon on the right, and then select the all models link on the right
Dara McLaughlin, Project Webmaster, Boeing IDS

I've long used a "compensatory decision process" without knowing it was called that, and for large decisions will use a spreadsheet. I see "satisficing" as either consistent with that process, or at the very worst lazy, but not naturally at odds.

It seems to me that comparison shopping has value – either positive or negative – in the decision process the same as other factors listed: It takes time, and can be either enjoyable or not. Depending on the perceived value of time in a situation, people will behave differently.

Comparison shopping for food items by price has a positive value for the lower income shopper. Comparison shopping for ingredients has a positive value for the health-conscious consumer. It has a negative value based on the time investment for the person on lunch hour, where a decision NOW means more time for the individual.

Comparison shopping for hobby items such as fishing reels almost always has a positive value, because hobbies are essentially pastimes and the shopping process extends the pursuit without substantially increasing the cost.

The goal should be for a Web site designer or business owner to first define their clientele and facilitate the process accordingly. The Tire Rack is a good example. The consumer can simply identify their car and their climate, and get two or three recommendations and an order form. The enthusiast can spend days searching the database of specifications and reviews, and even see how the choice will look on their car.
Ray Williamson, Consulting Engineer

– Recent research on how download times affect user perceptions of your Web pages.

The Aug. '03 Update was very interesting. We all know that "slow is bad," but some of the specific relationships between expectations, perceptions and reality are eye-opening – for example, the perception that slow sites are less secure. I hadn't heard that one before. The August update made me think of two other studies that neither patently agree nor disagree but, rather, support the notion we hear a lot in usability land: It Depends.

+++

From The Truth About Download Time, by Christine Perfetti, User Interface Engineering:

"It seems that, when people accomplish what they set out to do on a site, they perceive that site to be fast.

If people can't find what they want on a site, they will regard the site as a waste of time (and slow). But, when users successfully complete tasks on a site, they will perceive their time there as having been well spent.

We're wondering: When users are complaining about the download speed of your site, what are they actually complaining about? Are you better off making the site load faster or ensuring that users complete their tasks?"

+++

And from Aesthetics and Usability: A Look at Color and Balance, by Laurie Brady and Christine Phillips:

"This appears to support the idea that user satisfaction is related more to successful navigation than aesthetic appearance. However, when asked to predict which of the four sites the users thought would be the easiest to use, they ranked the aesthetically pleasing site the highest."

What I like about this article is the notion that good structure *actually* helps users be successful, while good appeal helps users *feel* they will be successful (which may impact their actual success on the site and their post-use perception of the site).

Just my two cents.
Pat Malecek, CUA User Experience Manager, A.G. Edwards & Sons, Inc.

Good summary, you neglected to mention/cite my new book, "Speed Up Your Site: Web Site Optimization" which is all about this subject and cites the papers you cite (and other newer ones). The first chapter is devoted to this subject (summarizes and distills current research)
Andy King

– Recent research regarding designing for the "elderly."

Thanks for the article on designing websites for "the elderly".

I was a little disappointed that your article failed to mention the importance of designing sites to use some of the accessibility features of modern web browsers. I'd like to point out that in my experience very few sites are designed to give visitors the control to change the text size through the browser. I'm refering to features such as IE's View, Text Size... option. Using this option sometimes changes some of the text size; sometimes none of the text sizes. Designing a site to provide this control is relatively easy, yet few designers seem to be aware of it, or maybe they're avoiding it for some reasons I don't understand.

It's possible to design a site that looks good and can be used effectively when viewed with point sizes of 8-10 or 12-14 as selected by visitors with different levels of visual accuity. I'd like to see more discussion of the use of Cascading Style Sheets (CSS) to acieve these and other design objectives to satisfy different visitor requirements. How about it HFI?
Roger Edwards, Standard Insurance Co.

– How should you lay out your Web site?

Isn't it more than a little ironic that this article was published on a centered fixed-width page? I find it strange that with all of the resources at the fingertips of these major corporations (NineMSN, IBM, Cisco, HP etc...) they have all preferred to implement left justified fixed-width solutions. Surely they have the research and development power to be able to make justified (no pun intended) and informed decisions. Through your research did you think of contacting some of these companies to try to find out why this approach was taken?
M. Wanstall

Reading a "stretchy" site on a large monitor is not comfortable. On the other hand, restricting the size leaves you with huge white margins on that same monitor. There is a third way, so to speak, that I use on my sites. By placing the entire page in a table set with a % width, you can compromise between ugly margins and readibility. Set it to, say 80%, and it will have acceptable margins on the large monitor, whilst still adapting to the user's display. Not so clever, I know, and there's a catch with this: on a small display (e.g. 800), there are still margins on 20% of the screen – a waste of valuable real estate. One solution I've found is to use a 1px white / transparent spacer at the very top or bottom of your page, just inside the table. Set the image width to 790, and your site will never shrink below this level. The result: a Web site that displays at full screen on an 800 width, but balances margins with readibility at above. The user gets some control over the size, but within the boundaries set by the designer. Screen width: ------------- 800 = 790 1024 = 820 1280 = 1024
Brendan D.

I am surprised that the research has concluded that all three layout resulted in equal efficiency... unless of course that was the intention... Let's consider some out-of-the-lab scenario: these Web sites that are designed for 800x600 or whatever, assume their page will occupy that whole space, but often, in a windowing environment, the user needs to view two windows at the same time. Anyway, for whatever reason, the user may want to view the Web site in a window smaller than the designers have hoped for and in that case, the fluid layout adjusts the text/content, whereas the fixed size layouts would now require the user to do a considerable amount of scrolling to get to some information. So how about running the same research again, this time limiting the browser window area to something like a quarter of the monitor area? Let's see if they are still all equally efficient?
Roni Korenshtein

– What is the best font size and font style for Web sites?

Coming from a DTP and WP background, I have become interested in the use of typefaces and have done some work on students' preferences – mostly on paper rather than on screen. Given the choice, there seems to be a wide range of fonts and sizes students say they find easiest to read – some due to real visual problems, but not all. In a recent series of tests Arial 12 gave better comprehension results than TNR or the dreaded Comic Sans. Next step is to test Arial against Verdana and Georgia. I have not been looking at speed of reading though – only at comprehension.
Ann Shaw