How do we get any business insight out of all the information we are gathering? Here's an action plan:
Ignoring the “how” for now, imagine what piece of information, business intelligence, customer understanding, or incident prediction would transform your business. Does the data exist that you would need to analyze in order to find this key parameter, and where is it?
If it is in-house, how accessible is it? Is it in a sufficiently clean and managed state? Does it involve connecting more than one dataset together?
If it exists outside of the business, how readily available is it? Would you need to subscribe, or simply capture a publicly available data-stream or social feed?
If your content data is dirty, duplicated, or inconsistently tagged, consider using a data cleaning or migration package to apply better policy rules.
Once you have the content data under management, look at automated extraction of identifiers that you can use as keys to connect with unstructured or transactional datasets.
Looking at the volume of the data you have identified, or the speed at which you need to analyze it, or the complexity of textual analysis that you require, do you need a stronger toolset than you already have?
Treat external (or internal) social media monitoring as a special case, and discuss it with your PR team.
How will you acquire the skills needed to carry out the project? Using external consultants will get you started but may not be cost-effective in the long term. Recruiting new staff or training existing staff may work better, but look for “data entrepreneurs” who can combine technical skills with business acumen.
Consider how best to source suitable analytics tools. In-house development around open source products could leave you reliant on expert individuals. The purchase of a best-of-breed tool for a single requirement may be the simplest way to solve your problem but will not help others in the business with their needs. A productized analytics toolset may be the best answer, particularly if it is well integrated with your existing content and search systems.