Extracting information from multiple sources into a data lake is a potential solution for enabling cities to perform analytics work and share data across silos.
Data Governance Tools Quest, RSA and other companies can help organizations share data securely and seamlessly, and a number of cloud-based providers, according to O’Donnell, offer a way to centralize and analyze data.
“We’ve seen over the past few years the ability to leverage data across multiple clouds,” he says. “Because the Ministry of Transport may have something to do Google Cloudwhile another division does something with AWS; in the past, this made it impossible. Now, there are tools that some cloud providers have enabled for you to work or transmit data, or have an application that works on data that resides in multiple clouds.
RELATED: Cities can now measure how “smart” they are.
Technology solutions help cities connect their data
In Carlsbad, California, The creation of a data lake is one of the city’s planned data management efforts. The city council has earmarked significant funding to develop systems that will make it easier for departments to use, analyze, visualize and share data, according to chief innovation officer David Graham.
Carlsbad has also done an extensive data inventory of approximately 4.5 terabytes of structured data, in addition to other items, and added a data manager position in recent years.
“One of the challenges faced by governments of all sizes is the collection, governance, management and distribution of data,” says Graham. “How we process, share and distribute data is fundamental to keeping our municipal operations right.”
Carlsbad uses the Board platform as the main data visualization and analysis tool. In anticipation of the eventual use of a data lake, the city is trying to increase its amount of structured data and the number of systems that can access unstructured data, Graham says.
“The city has more than 100 software applications,” he says. “We know we can probably consolidate and reduce that number.” So, as the city upgrades or replaces systems, it looks for “multipurpose software applications that are easy to integrate with other systems, that are best-in-class for the area they are in.” focus and ultimately make it easier across the organization to access and use data,” says Graham.
The city’s data manager must now be involved in the acquisition of any new data storage or collection-related items to avoid having disparate systems that cannot be integrated.
TO EXPLORE: How does real-time data analysis benefit transit agencies?
“It doesn’t matter how many terabytes you have stored somewhere; if it can’t be turned into information for the organization, you might as well just throw it on a hard drive and put it in your filing cabinet,” says Graham. “It’s data usage and access that ultimately impacts the data you’ve collected.”
In Madison, Wis., agencies’ technology choices can vary, according to Ellen Anderson, the city’s chief data and innovation officer. For example, the public works team uses a comprehensive asset management system that incorporates reporting, while other agencies rely on Excel spreadsheets.
The city data team uses tools such as Microsoft’s Power BI platform. It can combine multiple sources of information into interactive graphs to process and visualize data for efforts, including a program that involves the coordination of paramedics, social workers, and other entities responding to 911 calls.
“We connect it to our raw data and use certain functions like Power Query to join and clean our data,” Anderson explains. “We actually use their data management tools as our primary data management, and we also use visualization to share with our stakeholders what’s going on.”
The solution’s capabilities align with Madison’s ongoing data governance and usage initiative, which includes fostering a collaborative and data-driven culture.
“It’s a great way to empower people to work with their own data without needing outside experts…to come and show them what they need to do, because it’s very user-friendly,” says Anderson. “But it’s also a pretty powerful way to see what’s going on with a lot of data.”
Click on the banner below to get a personalized smart city content experience by becoming an Insider.