The San Sebastian City Council is committed to sustainable development and increasing the quality of life of citizens. The challenge is to improve the quality of the data stored and the implementation of data analysis tools in the different departments of the council, to turn the city into a 'smart city'. New technologies and the combination of different areas of knowledge:implementation of data warehousing methodology, Big Data, tools for Business Intelligence for the visualisation of data in real time and the definition of value indicators have an enormous impact on relevant data-based decision making, an essential step towards turning San Sebastian into a Smart City.
San Sebastian City Council Challenge
Bosonit collaborated with the San Sebastian City Council to carry out the deployment and configuration of its centralised data storage platform, and the exploitation of its content. The objective was to pool the information held by the different departments that make it up, thus offering citizens a complete view of the city's indicators through its open data portal. The platform is developed with free ecosystem technologies. Big Data, It also includes specific connectors for proprietary visualisation tools, open data portals, CMS, APIs, etc.
Technologies used
Data lake: distribution, Hortonworks; storage system, HDFS; processing framework, spark with python, spark SQL, Spark Streaming and Hive; data import, Flume; securitisation framework, Ranger.
Visualisation and analysis: Qlik Sense.
Integration continues: Jenkins.
Targets achieved
Implementation of the methodology: the different departments have been able to adopt a new way of working that is data-oriented, quality-oriented and focused on the common benefit of the municipality.
Platform: the objective of having a DataHub based on the paradigm of the Big Data fully functional and in production, in which to store the valuable information of the city council and all the departments that make up the consistory.
Integration: it has been possible to define and implement a working methodology that allows, in a programmatic and automatic way, new data from each department of the city council to be sent periodically to the central DataHub, for its subsequent processing and exploitation, also automatically.
Exploitation: it has been possible to define private and public value indicators for each department and for the city council as a whole, on the basis of which relevant decisions can be taken to turn San Sebastian into a Smart City.
