N.World
bosonit-success-case-nastat

NASTAT project: Facilitating the processing and dissemination of statistical operations

Nastat is the name given to the Statistical Institute of Navarre, the public body responsible for the collection, processing and dissemination of statistical data in this autonomous community.

NASTAT

The challenge

The objective was to to cover several needs related to the processing and dissemination of different statistical operationsThe treatment has so far been carried out manually by the agency's technicians.

Scope and limitations

The target model consisted of the orchestration of different tools, mostly open source, covering the following proposed needs:

  •  Firstly, the need was raised to centralising all the information available to Nastat as well as the information that would be available to it in the future. by designing and implementing a Big Data architecture in a cloud environment..
  • Secondly, the automation of ETL processes that they carried out in order to thus reducing the manual work of data collection and processing..
  • Thirdly, the processes of dissemination of statistical operations to the public were improved through visualisations using BI tools.This replaced the dissemination via Excel files that used to be carried out. Finally, the back-end of the new website was created to integrate all these elements.

Execution

The project has been composed of the development and deployment of three main tools:

  • Big Data Architecture where to centralise information.
  • BI software for the creation of visualisations, with the aim of making them easily interpretable by the end user.
  • Web portal where to publish the information after processing.

Process carried out

During the development of the project, a perfect coordination between the different development teamsThe project was designed to provide functional support so that both the logic and the developments would help the teams to meet the stipulated objective.

Technologies used

  • Python for the integration of data into the HDFS system, both from files available in Nastat and from others extracted by web scraping techniques.
  • Spark for the development of ETL processes through the PySpark API.
  • Hive for data aggregation, consultation and analysis.
  • Jenkins for the automation of the entire data lifecycle, from its collection to its final form.
  • Power BI for the dissemination of data to end-users through interactive visualisations.
  • Liferay for the creation of the web portal to publish the information.

Conclusions

The project has made it possible, in addition to crear a necessary application to facilitate the work of Nastat members. y promote in this way the digital transformation of public institutionsforging a good relationship between Bosonit and the Government of Navarrawhich will lead to new collaborations.

Javier Antón

Javier Antón

Business Intelligence Tech Lead at Bosonit

You may be interested in

Take the leap
technological.

Contact us.