SDL Web Content Editing Survey - Background on the Survey Approach

You may have seen Nikki's post about the SDL Web Content Editing Survey or received the following invitation from Support.

If you haven't, feel free to read about and participate in the survey, especially if you regularly use the rich text formatting (RTF) feature in SDL Web or Tridion.

Question Format

As you take the survey you may notice something unique about the format of the questions, as seen in the following screenshot.

Most of the survey question describes a feature then ask how you would feel if the feature was available as well as unavailable.

After a description and possible screenshot of a feature, the survey presents two separate single-choice questions:

  • "If a future release of SDL Tridion enables you to (accomplish some goal), how would you feel about this?"
  • "If a future release of SDL Tridion does not enable you (accomplish some goal), how would you feel about this?"

The answer options vary being helpful to being a major problem:

  • This would be very helpful to me
  • This is a basic requirement for me
  • This would not affect me / I have no opinion
  • This would be a minor inconvenience for me
  • This would be a major problem for me

These pairs of questions are intentionally similar but reversed. Asking both about having and not having a feature help us to not only confirm which features editors use, but what impact they would have on satisfaction. It would been easy enough to ask if or how frequently editors use a certain feature. But that won't help us know how important a given feature is to editors independent of use.

The Kano Model

In Jared Spool's Mind the Product presentation last year, to explain his point that "Not Sucky ≠ Delightful," he cited the Kano Model as a useful view of customer satisfaction. This model groups product or service features into general categories such as:

  • The performance pay-off, the features that contribute to satisfaction the better they are
  • Basic (must-have) expectations, the things customers bring with them sometimes outside of the organization's control
  • Excitement generators, the things that would delight customers in a surprising way

As an example, he points out that a working toilet and hot water in a hotel are basic expectations (he's also mentioned Wifi was an excitement generator, at least before it started becoming a basic expectation).

The survey format helps us discover which RTF features matter to content editors and in what way (linear/performance, basic or must-be, or excitement generator). As Spool pointed out, it can be difficult to get the basics right without checking. Getting the basics wrong leads to customer dissatisfaction while spending extra effort on them won't bring delight.

At an aggregate level, in the example report below we see that survey participants feel between indifferent to positive towards having some XYZ feature. On the other hand, participants are mostly between indifferent to negative about not having XYZ.

The answers in aggregate help (the above might imply having XYZ is better for customer satisfaction than not having it), but more specifically we want to plot the answer-pairs for a given feature in a table similar to the following.

  Dysfunctional Question
(feature not present)
Functional Question
(feature present)
  Like Expect Neutral Live with Dislike
Like it Q E E E P
Expect R I I I M
Neutral R I I I M
Live with R I I I M
Dislike R R R R Q

Legend (note there are several ways to label such Kano Model survey results as seen in this Google images search):

  • Questionable (a "wrong" answer, which is still helpful in seeing which survey questions or features may be confusing)
  • Excitement-generator (also referred to as an attractive feature)
  • Performance (also referred to as linear, because the better/worse this is implemented, the better/worse the customer satisfaction)
  • Reverse
  • Indifference, where survey participants don't care either way*
  • Must-have or must-be (basic expectations)

See this post from Folding Burritos for excellent guidance on how to create such a survey and apply the Kano Model. After gathering the answer-pairs we should have a better understanding of what survey participants expect.

*If you're a fan of Gerry McGovern, you'll see that leaving in all features is probably not a good idea.

What's Next?

This survey is yet another way to conduct research and adopt a data-driven design in our design process. We've also conducted UX research visits and asked implementers for their thoughts on rich text on Tridion Stack Exchange meta. I'd also like to take a look at actual data, which is perhaps something the technical community could help us with.

We'll post a follow-up with some observations and results of this survey later this year, which we'll use as input into initiatives for rich text format area improvements in a future release of SDL Web.