Building Information Governance in the Age of Data Lakes
Chris Foley

By: Chris Foley on January 2nd, 2025

Print/Save as PDF

Building Information Governance in the Age of Data Lakes

Big Data  |  Intelligent Information Management (IIM)

When Systems Don't Talk to Each Other

In my organization, we have dozens of corporate systems, and all of them do really important things as part of the work of the organization. But if somebody wants to ask a question like, "How are we providing services into a particular community?" and "What are the types of services in a particular community we're offering over time?" - we have to go to multiple systems to answer these sorts of high-level questions.

The Data Lake Solution - With a Governance Twist

We're now setting up a data lake where we can actively extract data from our core systems. But here's where I come in with my information management hat - it's all about information governance. There's a great deal of excitement in the organization around "Cool! We've got all this data. Now we can ask all these questions we haven't asked before." But there is a whole mindful minefield of governance questions.

Making Privacy Work in Practice

We're working to ensure that every item of data is classified - whether it's people data, commercial data, or operational data. This means that downstream, when we're releasing that information to the end user in the organization, we'll have data masking. If they're looking at a data mart that has a hundred data fields from six different systems, they may not actually see all hundred data fields based on their access level.

The Power of Collaboration

This kind of project requires getting the technologists in the virtual room (because everything's a lot of Teams meetings now), along with the data people, the engineering people, and myself as the governance person. We all have to accept that we don't have a full understanding of the problem - the data people have to tell us about the data in detail, the systems engineers have to tell us what the technology is doing, and I'm there to talk about governance principles.

 

This blog post is based on an original AIIM OnAir podcast. When recording podcasts, AIIM uses AI-enabled transcription in Zoom. We then use that transcription as part of a prompt with Claude Pro, Anthropic’s AI assistant. AIIM staff (aka humans) then edit the output from Claude for accuracy, completeness, and tone. In this way, we use AI to increase the accessibility of our podcast and extend the value of great content.

About Chris Foley

Chris Foley is the Director of Foley Business Consulting Pty Ltd. He serves as the Data and IM Lead for The Salvation Army in Australia, a role which allows him to contribute his varied professional experience for a social purpose. His career highlights have included school teaching (including a period working in a Toy Museum), a small business owner and independent IM consultant, and as a presenter at IM industry conferences. He is an active member of AIIM and RIMPA Global, and holds university degrees in Information Management, Education, History and Social Sciences.