Enterprise Search | Intelligent Information Management (IIM)
The wealth of unstructured text contained in most organizations offers the potential for both overwhelming levels of information overload and fascinating levels of information discovery and new knowledge construction. This will likely require changes in the information literacy and sensemaking skills of knowledge workers in the workplace of the future.
Symptoms of information overload include greater tolerance of error, a false sense of security due to uncertainty reduction or overconfidence, stress, and an inability to use information effectively for decision making. There are even examples where poor performance in finding information led to fatalities because key documents were missed when searching.
However, there have been few (if any) empirical research studies on the impact of increasing information volumes on search task effectiveness in the enterprise. Prior to the 1990s, the focus of the librarian or information professional was mainly one of searching and finding information on topics. The focus today appears typically on filtering and selecting, as it is relatively easy to find (some) information on virtually any topic. Whether what is found is the most relevant or useful is another question, of course.
What is Information Overload?
There is no universally agreed definition of information overload, although most definitions are tied to the relationship between decision making and the amount (and characteristics) of information the person or team is exposed to. Information overload has been described in context to work tasks, as a perception on the part of the individual where the flow of information is greater than that which can be managed effectively. Other definitions imply people may not always be aware when a task is being hampered by information overload.
In contrast, others argue information overload has been conflated as both cause and effect, where perceptions of information overload may be more related to filter failure, identifying the importance of Information Architecture. Others argue it is not just filter failure, but also information abundance that are the two signatures of information overload. In this model, information overload is where the abundance of information causes a breakdown of an intention, where that intention can be imparted to a person and/or machine.
The Impact of Information Overload
In recent studies within one of the largest multinationals in the world, it was found that increasing information volumes negatively impacted search task performance by 50%. In other words, professionals missed twice as many high-value items present in their information assets than when searching significantly less volumes of information. Crucially, business professionals were not aware of this poor performance and were, in general, satisfied with how they had performed.
It was also found that in their enterprise search engine (corporate Google), an increase in the amount of content indexed (60%) over the period of one year, increased the search 'failure rate' (for the same queries) by over 20%. Where search failure rate is where a user makes a search and does not click on any result (or 'bounces' back after 30 seconds) most likely indicating nothing presented by the search engine fulfilled their need. This is most likely caused by more 'competing items' that match the search terms entered by the user. This evidences how a machine algorithm that once worked ‘quite well’ in terms of its intention, can also be negatively impacted by increasing information volumes.
Coping with rapidly increasing information volumes may be a wicked problem. There may be no solution, but there may be levers to be pulled, which can make things ‘better’ or ‘worse’. Recognizing its impact (on people and machines) and putting various theories into practice may be a first step in that journey. After all, there is nothing more practical than a good theory.
