User Testing D7UX - Screencasts and Writeup

Here are the first three user tests that have been performed on D7UX stuff so far. One is about a month old, the others more recent. As implementation is constantly adding new things, the interface changes. Two were real user-tests performed with me as a host, one is done by the user itself. All participants of different skill levels.They have been using Drupal before, but none is an expert. All the current and future videos can also be found on the d7ux vimeo group

How does this differ from my famililar 4.7

The guy in Video 1 runs a big website that interestingly hosts Screencasts largely in the area of Photography and is using Drupal 4.7 (yes) on a daily basis.

[video:http://vimeo.com/6241014]

Where am I?

The participant in the second video, "User-Testing Admin header Drupal 7" is a rather non-webby person but takes care of the basic content on her Website and makes money off it. 

[video:http://vimeo.com/5994325]

Ah, let's have a look at the new Drupal version...

 "D7UX first try" was performed by the user itself, so it is her personal walkthrough. 

[video:http://vimeo.com/5657241]

What's the outcome?

Let's try to get to recurring patterns in these videos, as they "prove" something does not get its way to the user.
One weakness of my testing personnel I admit: all of them knew Drupal, and some knew Drupal too well. So they expected things to be in familiar places. More interesting would be people with very little or no Drupal experience.
What may be a cause for some distortion is none of the participants was a native english speaker. Very apparent in the "Where am I" video, Iris missed some things just because the english terms  "edit" "add" and others did not immediately trigger the right connotation for her.

Where am I
While this is surely the more a problem, the more novide the user is, it is the primary one. You start out in the interface of Drupal, and you want to write something. Now you want to know where you put your new article and to go back to some place you recognize as "start" or "home". As Drupal does not have any intitial structure it is no wonder this is not there. Surely the "Where am I" Video participant had the greatest problems with this, but the pattern is apparent throughout.

How do I get out of this
Similar to the where am I: the user just completed a setting in the admin area and wants to go back. Two out of three users have missed the "Close" button on the far right side, despite repeated hinting there. This proves the far right area is not one in focus, but the users might learn this. However the button appears not to jump to the eye enough. No other hints are given how to leave the overlay, and since your regular site title is not there, you have a bit of a problem. Lost in the black :)

How do I go to the front page / live site
Again, this is about the same, with the difference that now the user does know where he/she wants to go. Well, there is a teeny tiny Link "Home" or "Back to live site" in the upper right corner... If you find it.

Confusing: what is overlay, what is admin theme, do we have three areas now?
What is the difference between areas where we have an overlay and thus a black border, and of those where we don't? Though not articulated, this adds to the general lack of orientation and should be considered. How can it be clearer?

Overlay is not recognized as an overlay
Now this is serious. Is an overlay that covers the entire page still an overlay? It definitely looks nice. Yet not one out of three participants recognized it as such.

Right top part of the screen is out of focus:
We had that before. Strangely it appears to be a bigger problem with the close button than with the tabs. Maybe the tabs are more of a familiar concept. The second "Save" button also gets widely overlooked. This may be a problem of Users being used to Drupal and knowing the save button is always at the bottom.

Water pit teaser text field
The biggest problem was there for the most novice user. Some people start to write inside that field until they realize: hey, there is another one beneath.

Enable themes: ouch
User enables all themes. Surprise, surprise: what does enable mean? Next gets trapped by making Seven default: suddenly the backend looks like the frontend, we are lost.
As the user of the first video said, it might be reconsidered to put "Appearance" that only contains one single item, and is probably ever once used in the site lifecycle, in such a prominent place.

Positive outcome

Edit mode is intuitively understood
Only two user had the chance to use this. Apparantly it is the most self-explaining of all parts of the interface we have. The dotted lines add much more visibility to it. Which does not mean people know what to do where they get, once clicking the edit link :P Some issues were raised in the tiny icons being to close to each other: you might trash an article by chance.

Naming of top level Categories is good
Especially the Links "People" and "Appearance" triggered instant reaction in the "familiar 4.7" Participant. Especially interesting, since he was used to different terms. The search for common-language-terms that differ slightly from common "web slang" that would e.g. have used "users" instead of "people" appears to be a success.

Shortcut bar is clicked frequently.
Though I only before the last test had the idea to put some icons in for testing, people seem to like the shortcut bar the more, the more they are novice. Still Focus appears to lie a little lower in the main content area. The natural starting point with biggest attention seems to be somewhere between the header and the text of your first article, on the left side.

Some easily use Freetagging
Quite astoundingly, the "How does this differ from my familiar 4.7" User easily entered some tags, as if they had always been there. The others didn't, but weren't trapped and just skipped it.

Clear diffentiation between Admin and site area
While people have a hard time switching back and forth between the two areas, they appear to recognize the difference effortlessly.

Seven looks nice and is good in scannability
The seven theme itself does a good job in helping orientation on the pages. The economic typography and especially the arrow-like bullets on item lists make life easier.

Wrapup

Everybody draw his own conclusions. I had quite some surprises in this first rounds of testing. I fully take the blame on manipulating them on some items I just wanted certain reactions :P. Still for the most part people who see this interface for the first time react in a completely different way than I expected.