The AIIM Blog - Overcoming Information Chaos

8 Ways to Learn from the Wisdom of the Crowd

Written by John Mancini | Sep 27, 2010 8:46:56 AM

To the frustration of information architects and managers, end users have a habit of spoiling their hard work the moment they start using the system. The symptoms are many and varied: carefully designed folder structures devolve into chaos over time; despite training on document version control, copies of the same file keep appearing with version numbers, dates, and initials in the file name (e.g., "Proposal v.2 MP Comments.doc"); templates are used inconsistently or even ignored; automated workflows spawn undocumented manual workarounds... Aren't these precisely the kinds of things that our expensive, carefully planned ECM system was supposed to eliminate?

Yet in all of the apparent chaos, your users are giving you clues on better ways to organize information. For instance, when users ignore your folder hierarchy, they're telling you something important about it: maybe it doesn't make sense to them? Maybe they don't understand the semantic cues because the system's vocabulary doesn't jibe with theirs, or perhaps there is ambiguity that forces them to make decisions they didn't bargain for (e.g., Does an Employee Non-Disclosure Agreement go under Contracts or under HR?).

Instead of treating users' behavior as a collection of bad habits to be trained away, another approach is to learn from them: let your users' self-organizing behaviors teach you about what is meaningful to them, what their semantic cues are, how they really think about their content. When users find ways to work around structures you've provided for them, they are saying it doesn't work for them. How you respond will depend on your organization, the rigidity of your rules and regulations, and your willingness to let the crowd help to steer the ship.

Folksonomy, commonly represented by user-supplied tagging, is a popular example of capitalizing on the "wisdom of crowds". Letting your users develop their own informal classification structures via tagging is a useful step, but it really speaks to something more fundamental: users of content can be and often are the best judge of how to organize it.

How To Learn from the Wisdom of the Crowd

1. Learn to Tolerate Some Chaos, Within Reason.

Trying to control and lock everything down will stifle your ability to learn from your users, and will likely fail in the end anyway. Letting users self-organize will ultimately provide better insight than focus groups or surveys if you're willing to put up with some messiness at times.

2. Mine your Data.

What kinds of tags are people using? What structures do they come up with organically? What vocabulary and semantics do they use? ECM systems tend to promote the semantics of file cabinets, folders, etc. What kinds of things can your users come up with? Some ECM packages come out of the box with useful reporting tools, but you may find it more useful to integrate into your Business Intelligence environment (if you have one), to foster ongoing evaluation and analysis of your users' habits.

3. Periodically Take Stock of the Best Organic, Ad Hoc Ways of Organizing and "Graduate" Them into Formal Methods.

For example, pick the best categorization schemas from successful collaboration projects and turn them into templates, making them easy for others to use.

4. Listen Carefully to Feedback and Complaints, but Don't React in Haste.

Give your users time to self-organize without excessive tinkering. Problems (as long as they're not killing productivity) can be catalysts to so-called "workaround behaviors", which can give you a roadmap for better solutions. Workaround behaviors can also alert you to changes in processes that your original planning hadn't anticipated. I once worked on a workflow where three signatures were required in order for a document to be accepted, but a change in policy eliminated two of the signature steps. To compensate, users began assigning all three signature steps to themselves, and because the inconvenience was minor, they never raised it as a systems issue.

5. Talk to Users About What They're Doing Outside of the System.

What manual workarounds are taking place? You can sometimes learn a lot from the features and functions of a system that aren't used. Not every problem has a technical solution, and at times the implementation of a new system will paradoxically lead to more offline, untracked behavior as users circumvent processes they perceive to be clumsy or inadequate. But if you know what's happening outside the system, you at least have the opportunity to address the challenges that are leading to workarounds.

6. Give Users Tools that Let Collective Wisdom Cumulate.

A great example of this is a predictive search field, as used by Google and YouTube. Merely by typing in a few characters, users get a list of the top search terms that match. Not only can this greatly enhance the speed and efficiency of searches, but the lists are self-maintaining: the more they are used, the better the predictive value of the lists. Note, however, that there may be privacy concerns with revealing search terms this way, so make sure you don't violate any internal policies or regulations.

7. Provide Venues for Users to Share Their Techniques.

This could take the form of a user group, where users are encouraged to talk about their methods for organizing and finding information. Alternatively, foster a rewards system, and even friendly competition. Perhaps you can keep a public list of the "Top Taggers", with visualizations showing how the current user stacks up. Remember that the users are providing you with valuable information, and encourage ways for them to provide information to each other.

8. Learn from What Doesn’t Work, and Don’t Be Afraid to Cull Dead-Ends.

Many organizations implement internal message boards or forums, only to see them gather dust after a few weeks or months of use. On the one hand, you want to provide tools to foster collective wisdom, but on the other, it’s important to recognize when something isn’t working and to try to find out why. Why did people stop contributing to the company message board? The answer to that question may tell you a lot about the users and provide clues to what will work better in the future.