29 July 2020

Opinion column: “The value of open data in the midst of the pandemic”

By: Data Observatory Engineering Team

Data Observatory designed, implemented and currently operates the solution that was created in the GitHub repository of the Ministry of Science, a data platform on COVID-19 and its impact in Chile. The Data Observatory’s work has consisted of ingesting public information from its original source, transforming it into interoperable standard formats, and making it available on the platform.

For the Data Observatory (DO) it is a project aligned with the mission of the organization, turning the DO into a hub in which discussions, consensus and collaborations are generated that add value by allowing more people to dedicate their time to working with the data, focusing on their analysis.

We have added collaborations of people, academics, public and private organizations, among which are the University of Development, the Institute of Complex Engineering Systems of the University of Chile, the Civil Aeronautics Board, the ministries of Transport, Environment and Health, among others.

The repository currently has 46 data products. It is contributed by data providers, interested in participating in the open project and collaborating from different perspectives: extraction and pre-processing, processing and visualization, and data analysis. The data sources that feed the platform must be open, in the opensource sense; that is, when submitting a contribution, you must guarantee access to all files (data sources, source code, and processing output). Furthermore, by using open sources, transparency is guaranteed, generating trust and more collaborations with the community, in positive feedback. In this sense, we have added collaborations of people, academics, public and private organizations, among which are the University of Development, the Institute of Complex Engineering Systems of the University of Chile, the Civil Aeronautics Board, the ministries of Transport, Environment and Health, among others.

The biggest advantages of GitHub are related to the openness and agility that this platform enables. Its users are free to comment / suggest / contribute what they deem appropriate, and this triggers enriching discussions that allow us to reach consensus that in turn enable the co-creation of new solutions. In this sense, we have received collaborations from third parties regarding the development of solutions, data quality assurance, and suggestions / requests for new products, or improvements to existing ones.

Only between April 21 and June 17, the platform registered about 346 thousand visits and was downloaded more than 18 thousand times, giving rise to multiple and diverse modeling and visualization applications of the pandemic carried out by third parties.

The quality of openness is reflected in the traffic generated. As a reference, only between April 21 and June 17, the platform registered about 346 thousand visits and was downloaded more than 18 thousand times, giving rise to multiple and diverse modeling and visualization applications of the pandemic carried out by third parties. . At least 22 Chilean research groups have reported using the platform.

Regarding the hosting / development of the solution, the design and implementation have been carried out thinking about having all the necessary infrastructure in the cloud. Likewise, data collection and processing is done through github actions, a platform for continuous integration / deployment, which allowed integrating AWS services to develop an API that allows data to be consulted. Like knowledge, data flows dynamically, opening continuous spaces for learning and developing disruptive solutions to meet the needs and implications of this pandemic. The important thing for us is to contribute to this happening based on real and reliable data.

* Data Observatory is an initiative led by the Ministries of Economy and Science, together with Universidad Adolfo Ibáñez and Amazon Web Services; which seeks to contribute to the generation of solutions and capabilities in Data Science and related technologies, which are useful in various sectors of science, technology and the economy.

Related
19 August 2020
Entrevista EMOL TV, Open Data Cube
Read more
18 August 2020
Entrevista RADIO AGRICULTURA, Programa Faro Economía y Empresa
Read more
24 July 2020
Interview with Samuel Varas, Acting Executive Director DO
Read more