1Spatial Principal Consultant Mark Farrington talks about data integration and tackling the “Third V”.
I love being a consultant. Like many of my peers, I enjoy working with clients, understanding their issues and helping to solve their problems.
1Spatial is a spatial solutions provider and we see our role as partnering with clients to really understand what they need. We can then use a range of products and services to build the best possible solution for their specific situation.
One of the great privileges of being a consultant is working with forward-thinking customers from different organisations and sectors, seeing a range of challenges … and seeing common issues reoccur.
Often, these reflect emerging trends in the field. Those pathfinder clients are just the first to tackle the issue, the first to seek an innovative solution. It’s exciting to be a part of that: bringing your experience, expertise and a fresh perspective to help shape a solution.
Data integration and the Third V
One important trend that we see colouring many of our consulting engagements currently is that of data integration – joining data from different sources (with all the variation that might imply) into a single set to provide new insights.
The growing interest in, and availability of, geospatial or location data is a big driver of this. Organisations are seeing that location adds an important dimension to business decisions, whether those involve logistics, customer service, asset maintenance or infrastructure investment.
Speaking recently, Michael Stonebraker, winner of the 2014 Turing Award, noted that two of big data’s defining three Vs (Volume and Velocity) have basically been solved. The third however, Variety, remains a challenge:
“As near as I can tell, [data variety] is what is causing problems for nearly every major enterprise on the planet. I think what is going to kill everybody isn’t necessarily the number of connected devices, but the variety of independently-constructed data sources that enterprises are going to want to put together.”
Of course, data doesn’t have to be “big” for the variety of data to create problems.
At 1Spatial, we have developed significant expertise in the integration of spatial and non-spatial data from all across the enterprise. These data sources often exist in departmental silos. The data-sets are in different formats, of differing levels of quality and completeness, and collected at different times.
Part of our role is to find a cost-effective and time-efficient method to integrate data from those sources, in a consistent way, so that it is always ready for interrogation.
Regular readers won’t be surprised to learn that our solutions usually involve developing an automated, rules-based approach to squarely address this Third V of data variety.
Consultant as catalyst
One aspect of the consultant’s role is to take expertise and proven methodologies and apply those to a client’s situation. Another is to take experience gained from different clients and sectors and apply that by bringing a fresh perspective to the situation.
Our role is often to act as a catalyst, bringing together different view-points from within the organisation or suggesting fresh uses for existing data. For example, how could road-traffic data be used to predict the probability of leaks in water pipes?
Our collaborative, workshop approach helps enormously. Step One is always to listen and learn. Generally, we begin an engagement with a workshop which involves owners and users of an organisation’s data. When you speak to the people at the frontline, those who rely on the data in their everyday roles, you often find new connections.
As a catalyst, we push boundaries, bringing just the spark of an idea that enables the organisation to explore new approaches and new, internal partnerships. The problem of data integration is often one of breaking the owners – as well as their data – out of departmental silos.
Ultimately, organisations seek competitive advantage through lower costs, faster decisions or deeper insights.
I see our role as enabling that though improving data integration. Do you agree?
To learn more about the 1Spatial consulting approach, or to read about specific offerings, visit https://1spatial.com/services/consultancy/.