A question I hear pretty frequently is, “Why a redesign of the admin panel so soon after 2.5?” Those who have attended WordCamps in the past few months have already heard the answer, but for the people who haven’t had that opportunity, this post is for you.
When the community response to the 2.5 admin redesign was mixed, it seemed like a good idea to do usability testing to find out which issues were based on actual interface problems vs. which complaints were just a result of not liking change. To prevent bias, a third party was contracted to conduct usability testing, Ball State University’s Center for Media Design, Insight and Research division. Try saying that three times fast with a mouth full of peanut butter. Or fitting it on a business card. To save time, we’ll just call that third party CMD, since that’s what they call themselves.
The plan that was developed involved multiple rounds of testing, as well as the creation of two prototypes, hardcore! The first phase involved a usability review of 2.5 by CMD, the results of which were discussed with lead developers. A quick prototype was created that addressed some of the lightweight issues, so that the test participants could use both 2.5 on their own blogs and the prototype on a test site. Results would be analyzed and compared, leading to a second round of suggestions. A second prototype would be developed over a week or two, which would then be tested with the same participants, and a final report delivered. But you know what they say… the best laid plans of designers and developers often go awry.
After the first round of testing, it was clear that a prototype delivering the kind of fixes that could be coded in a week or two wouldn’t make much of a difference overall. We all decided a more ambitious prototype was in order, one that would experiment with a new approach to screen real estate and attempt to address as many of the issues from 2.5 as was possible with a few extra weeks of time. A rapid design process was followed by an even more rapid development cycle. The second prototype is what you know as Crazyhorse.
The second round of testing blew everyone away. The research team had never seen such consistent results. Tasks were completed faster, participant opinions rated it higher, understanding of how interface elements worked was greater, and it wasn’t even a fully functional application. Of the test participants, every single one said they would choose the prototype over their current administrative interface, and it wasn’t even pretty (those of you who remember the original Crazyhorse will vouch for this).
A presentation on the process from start to finish was part of the schedule at WordCamp 2008 in San Francisco, and the slides are available online, but as always the slides only tell you so much without the videos, live demo and verbal narration that went with it. (Use Google and you can see audience videos of the presentation.)
Here, then, is a PDF of reasonable size that you can download and peruse at your leisure that outlines the usability testing project in some detail. I wanted to include some eye tracking videos, but the file was so huge it would have been ridiculous for anyone to download it, so I stuck with eye tracking outputs called gaze trails to illustrate the findings. I also tried to pare down the text to the more salient points, since more than 50 hours of test video really does reveal an insane amount of data. I also cut out the section about designing Crazyhorse in the interest of staying under 25 pages. Hopefully you’ll think it’s a good balance. I’ll try to put together a separate document on the design process of 2.7 in a couple of weeks that will include the early Crazyhorse material.
So, if you want to know what we learned from the usability testing this summer that caused us to create what is now 2.7, go ahead and read the report.
WordPress 2.5/Crazyhorse Usability Testing Report (PDF)
Have you seen the getting-prettier-all-the-time menus in 2.7-almost-beta? They really are. Getting prettier all the time, I mean. Once we drop in the fonts and do a little brushing up of edges and colors, the menu system is going to be smooth. The last thing we’ll need to do to is replace the icons we’ve been using as placeholders. Currently, the menus are using icons from 
