Banner Blindness: Old and New Findings
After much soul-searching, I've now decided to take a different approach and publish our new findings, despite their ethical implications. In reality, it's not possible to suppress research results because anybody who bothers to run the study will get the same findings. There are no secrets of usability any more than there are secrets of astronomy. If you point your telescope at Saturn, you will see that it has rings. And, if you conduct a series of usability studies, you will discover the same insights as we do -- assuming you employ the correct methodology.
Many people without a grounding in behavioral user-research principles use bogus methodology and thus get misleading findings. Poor methodology is especially common for eyetracking studies, and thus most published studies in this area are wrong.
For example, unskilled researchers often ask users to simply look at a page, rather than have them encounter it as part of a task flow. Users naturally look at things differently depending on the context. For example, if you want to know how users look at the elements of a form, you can't just present the form on a stand-alone page and ask them to fill it out. Instead, you have to present the form in the context of a meaningful task that they might attempt in the real world. That is, users should encounter the form in response to particular actions, such as deciding to check out from an e-commerce site.
