Barnsley Metropolitan Borough Council
Barnsley Council driving efficiency with 1Spatial and FME
When Barnsley Metropolitan Borough Council updated its document management system (DMS), it used FME to manage the migration. The old DMS was ten years old and had not been developed in recent years. It was also, as ICT Manager Riley Marsden explains, “a very niche product. We were the only UK customer and had just one member of staff supporting it. As a result, he was the only guy in the UK who could manage the software.” The council decided to move to the more widely supported Microsoft SharePoint as its new DMS, but the decision left Riley’s team with the challenge of migrating ten years’ worth of documents from the old system to the new. These covered every area of Barnsley’s service delivery and the migration would include critical file parameters such as access permissions and complex metadata. “The timescale was really challenging,” explains Riley. “The project to install the original solution took a team of seven people four years to complete, but we had over five million documents to migrate in just eight months.”
The licence for the existing DMS had to be renewed at the end of the year – at a cost of “hundreds of thousands of pounds” – if the migration was not complete. Barnsley needed a cost-effective solution to migrate its vast document store as quickly and efficiently as possible.
Finding the best tool for the job
The team evaluated various tools. “The main challenge was the many possible parameters for any document. We might have several hundred thousand documents with no permissions, but a few thousand – like social care records – that required complex and very strict levels of confidentiality. We had a lot of very niche scenarios. When we looked at what was available in the market, many products could do some of the things we wanted, but were constrained in other areas.
Other tools, we realised, would become very expensive once we started to look at our different requirements.” However, Riley realised that Barnsley might already have the tool it needed. In recent years, the team had become experts in Safe Software’s data integration application, FME.
"As well as making our day to day operations easier, FME has contributed to some of the biggest projects our department has undertaken in recent years," he explains. Originally used to manage geospatial data, FME was now being applied to a wide range of data tasks within the council. Recognising that the existing DMS was essentially a database of documents, and with support from 1Spatial, the team developed solution using FME and SharePoint web services to upload the documents, along with all required meta-data, into SharePoint.
Significant savings from in-house tools and skills
Barnsley’s innovative use of a familiar tool helped it deliver the project quickly without paying for costly consultancy or additional software.
"We’ve saved at least £50,000,” says Riley. “Had the project over-run, it could have cost £250,000. And, had we needed to use consultants to tackle some of the complex scenarios, we would easily have doubled the amount we spent."
He continues, “We think we’re using FME in a pretty unique way. It’s saved us money and yet provided more user-friendly, more customised solutions, which are easier for us to support going forwards.”
A flexible, versatile solution
The flexibility of FME’s workflows means that the Barnsley team could easily create new workflows for unique scenarios. “The value of FME is that the process was within our control. We could test it and build the workflows quickly ourselves. The main advantage was that we could tweak it many, many times. We could handle different parameters, add permissions, even build a folder hierarchy within SharePoint to sit above a migrated document. That’s where other tools on the market start to get very expensive,” Riley explains.
The team also used FME to embed additional functionality within SharePoint for council staff. Sharing information on Cautionary Contacts is one example.
The council maintains a register of cautionary contacts: residents with a history of violent or abusive behaviour towards council staff. This information must be shared with staff who are likely to be at risk, but the information is also highly confidential. The previous process – emailing a spreadsheet to nominated staff – was onerous both to maintain and manage. It was also prone to data errors.
The team developed a new solution where the data entry was controlled and secured in SharePoint, and then automatically extracted and plotted at postcode level by FME. The “at risk” postcodes were then shared within web map in the same SharePoint interface. Stripped of personal information, the map could be viewed by all staff. Anyone with business in a highlighted post-code could then seek more detailed, sensitive information to assess their risk. The new process is easier to manage and always up to date.
Savings on other projects
Riley and his team have used FME on other projects, too, saving the council money while delivering better solutions. The council used FME to combine data from 10 different sources to support its Troubled Families programme. Four years later, and despite evaluating alternative systems costing up to £100,000, Riley says, "The programme has generated over £2,000,000 in central government funding and we’ve still not found a better solution to our FME ‘quick fix’."
FME has proved to be a powerful and versatile tool for Barnsley and, despite having developed significant skills over recent years, Riley and the team still draw on 1Spatial for training and consultancy support. As a leading, global Platinum Partner and Authorised Partner for Safe Software, 1Spatial helps customers to release the full value of FME.
Riley books consulting days regularly and the team stores challenges to discuss on the day.
"What we know about FME, and the experience we have, comes from 1Spatial. They have given us the ability to think about problems in terms of data transformation and workflows, and they’ve given us the FME skills to resolve those problems,” he says.