Friday, February 29, 2008

February 29th: Bonus Day!

Every four years something remarkable happens; we are given an extra day. Today is that day; do something with it. Here are some options.

  • You probably have the latest Intercom or Technical Communication sitting on your desk. Find a good article in each and read it.
  • Your inbox has 476 emails in it. Clean them out.
  • Download trial software and play with it.
  • Organize the rest of your year.
  • Go around the office being extra nice or cheery.

You get the idea. Today is a free day, a time-out from the 365 day grind. Make it special and in doing so make yourself special.

Happy bonus day!

Thursday, February 28, 2008

Currents Is Almost Here--
In times of tight budgets (and why are we ALWAYS in times of tight budgets) the Atlanta regional conference Currents is a great professional development bargain for those in the neighborhood. So put March 14th (workshop day) and 15th (sessions day) on your calendar.

The quality of the presentations is world class:
  • One of the speakers, Dr. David Yates from Auburn, has an article in the just published February issue of Technical Communication.
  • At least four university programs are represented: Mercer, Auburn, Georgia State, and my alma mater SPSU (also the venue of this year's event)
  • Speakers from major corporations will share their insights: IBM, Mirant, ComponetOne
  • I can see that at least two of the presentations (mine and Carol Barnum's) are part of the 55th conference in Philadelphia.
  • STC leadership will be there: Mark Clifford, STC 1st VP (upcoming president in June)-- and local boy 2ndVP candidate will be there kissing babies :-)
  • International keynoter: Jean-luc Dumont--just saying his name is worth the price of admission

And the networking is always great. So get your head out of the read-me you're working on, or that configuration guide, and rub shoulders with fellow professionals and talk about your craft. Celebrate your professionalism!

For full details go to http://www.stcatlanta.org/currents.htm

Wednesday, February 13, 2008

ROI--
Those wishing to go beyond Kirkpatrick's 4 levels that I discussed in my previous blog, can talk about Return on Investment (ROI). In short, does the dollar value of the benefit exceed the cost, and if so, how does it compare to other investments a company could be making? Cutting to the chase, what do companies get for their investment in technical communications (measured in financial terms)?

First, the math: ROI = $Results/$Investment.

You face three challenges when you try to do an ROI justification (or claim an ROI victory):
  1. Isolating the effect that technical communication had from what everybody else did. For example, did the Help desk calls go down because the Help was better or because the product's design improved?
  2. Putting a dollar value on the benefits. In the case of improved efficiency, this is easy enough: (time saved per transaction) x (number of transactions) x (labor cost). Putting a dollar value on increased customer satisfaction scores gets a little foggier.
  3. How does the return get harvested?

Challenges 1 and 2 get a lot of attention in the ROI literature so I'm going to skip them. I think #3 is the elephant in the room that nobody wants to talk about.

The R in ROI is "return" and can occur in one of two ways: Checks coming in get bigger, or checks going out get smaller.

I see lots of ROI examples and arguments around scenarios where better documentation reduces the search time of the user. I know that Information Mapping's web page used to have a calculator program that would help you arrive at the dollar amount for what you could save your company. Let's say you do such a calculation and can show a return of $175,000 a year for an initial training and document conversion cost of $100,000. That's a good ROI. But what do you do when the sponsor asks for her $175,000? After all, she did her bit and forked over the $100,000. Where does it come from?

"Uh, people did more work."

Great, did we charge more for the extra work, is my $175,000 in income? Can you point me to it?

"Uh, well, they were more efficient and did the same work in less time."

Great, did my payroll go down $175,000? Was headcount reduced as a result of this innovation?

And this is the problem with many ROI scenarios. They look great until you try to lay your hands on the money. So what's my suggestion? Don't go there unless you can actually produce the increased savings or income in real ways that can be pointed to. If DITA reduces translation costs, go for it. If you say you're going to make content more reusable, be prepared to see headcount cutbacks in future budgets. Hey, you were the one that said you would be able to do more with less.

Bottom line: Don't get into the ROI game because it makes you sound more business-like. Use it only if you can cough up the bucks when your investor asks, "Where is my return?"

Friday, February 08, 2008

A Model for Assessing Value--
I've been thinking and talking a lot lately about our value as technical communicators. I can see that my thinking has been influenced by my background in instructional design, specificially by Kirkpatrick's model of evaluation. In his model, he states that instruction can be assessed at four levels:
Level 1: Reactions
Level 2: Learning
Level 3: Transfer
Level 4: Results

I think we can easily apply this same model to how we evaluate our communication products as well as our contribution to our sponsors.

To summarize the four levels in the context in which Kirkpatrick proposed them, I put them in the context of a training course on how to correctly operate a drill press. Let's assume the training had been triggered by excessive scrap rates coming out of the machine shop.
  1. Reactions. How did the students react to the training itself? This is generally assessed through a course evaluation sheet, e.g., course met my expectations, instructor was knowledegable, yadda, yadda, yadda.
  2. Learning. Did the students learn anything? This is generally assessed through testing or lab observations. Students could be obseved by the instructor who certifies them using a checklist of targeted drill press competencies.
  3. Transfer. Did the student go back to the job and apply the new knowledge or skills correctly and effectively? Employees coming out of the training could be assessed by the shop supervisor who observes if the they are applying the right techniques.
  4. Results. Did the training solve the business problem that triggered it? Did the scrap rate go down?

By the way, Phillips adds a fifth level, ROI, which would compare the cost of the training to the dollar value of the reduced scrap and compare it to other investments that could have been made.I'll blog on that one later.

So let's see how Kirkpatrick's model could be applied to technical communication.

Level 1: Reactions--We see this in reader response cards and in usability tests where participants are asked to rate various aspects of a product or document. Lots of the research done on fonts or layout stop at this level of evaluation, e.g., which document looks more professional?


Level 2: Learning--To me this translates to: If the user read the document, did he or she understand it and did it apply to the task at hand. For example, did the Quick Start card work in the lab when we specifically asked users to use it?


Level 3: Transfer--Oooooh, the toughie. Did users improve their performance in real life based on the documentation? For example, did real patients comply better with their medication protocol when given redesigned instructions?


Level 4: Results--Did we achieve the business goal the document was meant to achieve? For example, did Support Desk calls go down, did medical claims decrease, did user registrations increase on a redesigned web page, did percentage of transactions completed go up, etc.

I think we need to emphasize levels 3 and 4 more. I might be wrong, but I haven't come across a research study on fonts that tested if users completed tasks faster or made fewer errors depending on which font face the Help was written in. (So why do we fight so passionately about it?) So I would like to see research in our field increase its emphasis on user performance. (Level 3)

I would also like to see more discussion about the how better informed, better performing users make positive impacts on an organization's business performance. (Level 4)