I've seen in here and there someone mentioning that usability of Plone is very good. Lot's of people - me included - like the fact that in Plone you don't have separate content management interface compared to some of Plones rivals. That counts for something when we're talking about good usability. Still that is only one quite small part of the whole picture. So what else is there? What do people like in Plone and where are the rough edges for end users? If general consensus is that Plone does have good usability, where is the actual proof of that?
On plone.org I found one page in developer documentation mentioning following: "Plone differentiates itself on usability. The intuitiveness of the user interface is what attracts people to Plone the most."
I interpret this sentence meaning about the "one view for all" approach. What bugs me in this is that this whole sentence about good usability is about how the UI works compared to Plones competitors. Saying that our approach has more intuitive UI than our competitors is one thing - proving that our claim is true is another thing. Yet I think the most important thing is missing from here. Proving that the UI we have has good usability in itself. Is the user experience better than our competitors and good enough for our end users or is it actually, really and without any doubt so good that it wouldn't be better without rethinking whole thing and designing even more intuitive UI?
In launchpad.net/plone it reads that "Plone is easy to use. The Plone Team includes usability experts who have made Plone easy and attractive for content managers to add, update, and mantain content."
Good usability is not just that someone says or writes it. You'll need to prove it too - and not only by comparing how our product does against competitors products - getting the end users opinion weights so much more.
On plone.org I found one page in developer documentation mentioning following: "Plone differentiates itself on usability. The intuitiveness of the user interface is what attracts people to Plone the most."
I interpret this sentence meaning about the "one view for all" approach. What bugs me in this is that this whole sentence about good usability is about how the UI works compared to Plones competitors. Saying that our approach has more intuitive UI than our competitors is one thing - proving that our claim is true is another thing. Yet I think the most important thing is missing from here. Proving that the UI we have has good usability in itself. Is the user experience better than our competitors and good enough for our end users or is it actually, really and without any doubt so good that it wouldn't be better without rethinking whole thing and designing even more intuitive UI?
In launchpad.net/plone it reads that "Plone is easy to use. The Plone Team includes usability experts who have made Plone easy and attractive for content managers to add, update, and mantain content."
Good usability is not just that someone says or writes it. You'll need to prove it too - and not only by comparing how our product does against competitors products - getting the end users opinion weights so much more.
Why I'm bringing this topic up? I'm doing my masters thesis about Plone usability and in the past week I've been searching usability studies about Plone to get some background material. When I started this I thought it'd be easy to find at least few through reports - knowing that people have mentioned Plones good usability quite often in separate posts. To my surprise I found only few short blogposts (Plone Usability Makes It As Easy As ABC, Usability of CMS Home Sites) and few evaluation videos, but none of those are scientifically accurate enough to be material in masters thesis. Basically I didn't find any actual usability study made about Plone.
I hope you don't get me wrong in this post. I'm being over critical about this matter as the subject is maybe too close my job and interests. I've been working with Plone about 4 years now and this includes developing new products, giving end user support and administrating sites which gets millions hits/month. I'm trying to forget all this in my masters thesis which should to be scientifically approvable study. If I won't find any proof about the good usability I have to think (scientifically) that it is all just words and nothings been proven - yet.
If you happen to know some study what I've (obviously) missed I'd be more than thankful to know about it.
Hi Jukka,
ReplyDeleteFirst, the Plone user interface in its current state is almost 9 years old. It's still a lot more intuitive than pretty much all our competition, but that doesn't mean I'm super happy with it. :)
That's why we are re-thinking the entire approach starting with Plone 5. There have been some informal user testing of Plone done by people on various blogs, but nothing that is really high quality.
We also pretty much know about all the issues we need to fix and where Plone needs to go in the future. Luckily, the work on Plone 5 has been going on for quite a while already on the infrastructure side, so we're very close to being able to start implementing the new approach.
But you are correct that "nothing has been proven" in the scientific sense — it's more a "we tried giving people CMS X and CMS Y, but they were more comfortable with Plone" kind of feedback. Your best bet may be to talk to integrators out there that have been through these situations and implement multiple different solutions, like Groundwire.
Plone always gets good feedback for its UI, even though it's starting to show its age. Fortunately, we're fixing that.
The other audience that should be able to help out here — but isn't — are the "CMS analyst" companies. Unfortunately, they usually focus more on specs and abbreviations and emerging standards in the chaotic world of content management. So you won't find much about ease of use there, they usually just devote a paragraph or two to it. (I mean, have you ever used Documentum? Holy s—t, that's one of the worst UIs I have ever encountered, anywhere ;)
I do have a track record with Google and now Firefox at Mozilla, so the sentence about the team including usability experts hopefully isn't too far off. :)
But I'll be the first to admit that Plone needs a massive overhaul in the usability department, and I hope to involve more people too. If you're willing to do some usability testing of the upcoming Plone 5 as part of your thesis, I'd love to cooperate with you.
Usability testing Plone 3/4 is likely to yield consistent, but unsurprising results, and our energies should really go into testing the future approach instead, which fixes the mental model and I believe will be a game-changer for CMSes out there.
Thanks for the very comprehensive reply!
ReplyDeleteI'll answer to last things first. When I first time started to plan my masters thesis (this happened about three years ago) I was going to do an usability study about usability differences between Plone 2.5 and Plone 3. Working with Plone caught me for few years and I'm now giving my masters thesis an another go. As the original idea is now quite old I was going to just study usability of Plone 4 which is the version we're developing our new sites/products for in hopes that at least we'd get some use for the data of my study.
Now you got me really interested about the Plone 5 UI. I'd love to include it or parts of it to my study. Problem is that I have very tight schedule - I should have my thesis ready before June and as I'm working full time at the same time I should already be preparing the test setup. The question is that how ready is the Plone 5 UI and what are the parts where usability test would provide useful data?
Here's some clarification about my post. What the scientist side of "me" tried to achieve with my post was to break the "Plone has great UI" attitude and find the actual data for it. You know how people think in university world when they're trying to do research :) I don't think that your (or anyone else) merits wouldn't be enough for being called as usability expert - it's just that it doesn't count in the research where results should be based on actual raw data.
I was surprised that even if we have for example Joel Burtons great pdf about Building A Humane CMS With Plone which is full of small things how to make Plone more usable, I couldn't find any study about it. And as you said "Usability testing Plone 3/4 is likely to yield consistent, but unsurprising results". Results are unsurprising if we know what the results are going to be, but to know that there should already be results which makes the whole study unsurprising. Also the fact that Plone is quite popular CMS in education world made me wonder why anyone hadn't studied it's usability before. What I was looking for was comprehensive study of Plone usability as of it's current state so that there would be actual data what users prefer and what not. Data about tasks that require too many steps, or if they think there is too much "noise" or unneeded fields etc. I've spent quite amount of time myself answering to our users support requests and I've got at least some picture what they find difficult in Plone UI and what has been more than difficult - inline editing or object locking in < Plone 3.2 versions pops to mind first.
The working "me" would be the first one to admit that Plones current UI is the best I've seen so far. Not that I'd actually seen that many CMS around - only Drupal, Sharepoint and Plone (I don't count Wordpress as a full CMS). Sharepoint was 5 years ago totally IE only system and I haven't looked at it since my trainee perioid. Drupals separate view for managing the actual content felt like huge step backwards after I had used Plone. First time I used Plone was quite a smooth experience, so there we go :)
But as you mentioned the work for the new UI is already being done and I think my efforts with already existing Plone versions wouldn't be that useful than if I'd conduct usability study about Plone 5 UI. I hope my schedule won't be problem with that.
http://blog.fourdigits.nl/researching-plone-ui-done
ReplyDeleteThank you anonymous! I just recently browsed in fourdigits website, but for some reason that post didn't caught my attention. Gonna give that pdf a good read tomorrow.
ReplyDelete