Dark data into the light of public control
Abstract
A considerable volume of data remains outside the knowledge of the governmental organizations, without curation, turning into dark data. In the public control area, where there are silos from several sources, with a growing volume, including citizens, dark data has been a topic not explored in the literature. This article brings the main concepts in dark data topic, listing its characteristics and risks, thus elaborating a conceptual map for the public control area. There is also the proposal of an approach that offers high abstraction for identification, classification, and monitoring of dark data, especially for the public control.
Keywords:
dark data, public control, electronic government, big data
References
Cafarella, M., et al. (2016). Dark Data: Are we solving the right problems?. In 2016 IEEE 32nd International Conference on Data Engineering (ICDE) (pp. 1444-1445). IEEE.
De Sa, C., et al, C. (2016). Deepdive: Declarative knowledge base construction. ACM SIGMOD Record, 45(1), 60-67.
Gallaher, D., et al. (2015). The process of bringing dark data to light: The rescue of the early Nimbus satellite data. GeoResJ, v. 6, p. 124-134, 2015.
Gimpel, G.; Alter, A. (2021). Benefit From the Internet of Things Right Now by Accessing Dark Data. IT Professional, v. 23, n. 2, p. 45-49, 2021.
Goetz, T. (2007). Freeing the dark data of failed scientific experiments. Wired Magazine, 15(10), 15-10.
Hawkins, B.; et al. (2020). Data dissemination: shortening the long tail of traumatic brain injury dark data. Journal of neurotrauma, 37(22), 2414-2423.
Heidorn, P. (2008). Shedding light on the dark data in the long tail of science. Library trends, 57(2), 280-299 .
Heidorn, P., et al. (2018). Astrolabe: curating, linking, and computing astronomy’s dark data. The Astrophysical Journal Supplement Series, 236(1), 3.
Henriques, A. (2021). Big data analytics para o desenvolvimento humano: um estudo no Governo Federal Brasileiro. Tese de Doutorado. Fundação Getúlio Vargas-FGV.
Hernández, D., et al. (2018). Bauspace: a scalable infrastructure for soft sensors development. In Proceedings of the 47th International Conference on Parallel Processing Companion (pp. 1-4).
Leonelli, S. (2013). Why the current insistence on open access to scientific data?. Big data, knowledge production, and the political economy of contemporary biology. Bulletin of Science, Technology & Society, 33(1-2), 6-11.
Liu, Y., et al. (2021). Deep Hash-based Relevance-aware Data Quality Assessment for Image Dark Data. ACM/IMS Transactions on Data Science, 2(2), 1-26.
Kim, G., et al. (2014). Big-data applications in the government sector. Communications of the ACM, v. 57, n. 3, p. 78-85.
Macleod, M., et al (2014). Biomedical research: increasing value, reducing waste. The Lancet, 383(9912), 101-104.
Manyika, J., et al (2015). The Internet of Things: Mapping the value beyond the hype. (Vol. 24). New York, NY, USA: McKinsey Global Institute.
Menegazzi, D. (2021). Um guia para alcançar a conformidade com a LGPD por meio de requisitos de negócio e requisitos de solução. Dissertação de Mestrado. Universidade Federal de Pernambuco.
Moumeni, L., et al. (2021). Dark data as a new challenge to improve business performances: review and perspectives. In 2021 International Conference on Digital Age & Technological Advances for Sustainable Development (ICDATA) (pp. 216-220). IEEE.
Munappy, A., et al. (2020). From ad-hoc data analytics to dataops. In Proceedings of the International Conference on Software and System Processes (pp. 165-174).
Munot, K., et al. (2019). Importance of Dark Data and its Applications. In 2019 IEEE International Conference on System, Computation, Automation and Networking (ICSCAN) (pp. 1-6). IEEE.
O’Donnell, G. (1998). Accountability horizontal e novas poliarquias. In Revista Lua Nova, Nº 44; São Paulo. CEDEC.
Schembera, B. (2021). Like a rainbow in the dark: metadata annotation for HPC applications in the age of dark data. The Journal of Supercomputing, 77(8), 8946-8966.
Schembera, B., e Durán, J. (2020). Dark data as the new challenge for big data science and the introduction of the scientific data officer. Philosophy & Technology, 33(1), 93-115.
Shukla, M., et al. (2015). POSTER: WinOver enterprise dark data. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (pp. 1674-1676).
Stahlman, G., Heidorn, P., e Steffen, J. (2018). The astrolabe project: identifying and curating astronomical ‘dark data’ through development of cyberinfrastructure resources. In EPJ Web of Conferences (Vol. 186, p. 03003). EDP Sciences.
Trajanov, D., et al (2018). Dark data in internet of things (IOT): challenges and opportunities. In 7th Small Systems Simulation Symposium (pp. 1-8).
Zhang, C., et al. (2016). Extracting databases from dark data with deepdive. In Proceedings of the 2016 International Conference on Management of Data (pp. 847-859).
De Sa, C., et al, C. (2016). Deepdive: Declarative knowledge base construction. ACM SIGMOD Record, 45(1), 60-67.
Gallaher, D., et al. (2015). The process of bringing dark data to light: The rescue of the early Nimbus satellite data. GeoResJ, v. 6, p. 124-134, 2015.
Gimpel, G.; Alter, A. (2021). Benefit From the Internet of Things Right Now by Accessing Dark Data. IT Professional, v. 23, n. 2, p. 45-49, 2021.
Goetz, T. (2007). Freeing the dark data of failed scientific experiments. Wired Magazine, 15(10), 15-10.
Hawkins, B.; et al. (2020). Data dissemination: shortening the long tail of traumatic brain injury dark data. Journal of neurotrauma, 37(22), 2414-2423.
Heidorn, P. (2008). Shedding light on the dark data in the long tail of science. Library trends, 57(2), 280-299 .
Heidorn, P., et al. (2018). Astrolabe: curating, linking, and computing astronomy’s dark data. The Astrophysical Journal Supplement Series, 236(1), 3.
Henriques, A. (2021). Big data analytics para o desenvolvimento humano: um estudo no Governo Federal Brasileiro. Tese de Doutorado. Fundação Getúlio Vargas-FGV.
Hernández, D., et al. (2018). Bauspace: a scalable infrastructure for soft sensors development. In Proceedings of the 47th International Conference on Parallel Processing Companion (pp. 1-4).
Leonelli, S. (2013). Why the current insistence on open access to scientific data?. Big data, knowledge production, and the political economy of contemporary biology. Bulletin of Science, Technology & Society, 33(1-2), 6-11.
Liu, Y., et al. (2021). Deep Hash-based Relevance-aware Data Quality Assessment for Image Dark Data. ACM/IMS Transactions on Data Science, 2(2), 1-26.
Kim, G., et al. (2014). Big-data applications in the government sector. Communications of the ACM, v. 57, n. 3, p. 78-85.
Macleod, M., et al (2014). Biomedical research: increasing value, reducing waste. The Lancet, 383(9912), 101-104.
Manyika, J., et al (2015). The Internet of Things: Mapping the value beyond the hype. (Vol. 24). New York, NY, USA: McKinsey Global Institute.
Menegazzi, D. (2021). Um guia para alcançar a conformidade com a LGPD por meio de requisitos de negócio e requisitos de solução. Dissertação de Mestrado. Universidade Federal de Pernambuco.
Moumeni, L., et al. (2021). Dark data as a new challenge to improve business performances: review and perspectives. In 2021 International Conference on Digital Age & Technological Advances for Sustainable Development (ICDATA) (pp. 216-220). IEEE.
Munappy, A., et al. (2020). From ad-hoc data analytics to dataops. In Proceedings of the International Conference on Software and System Processes (pp. 165-174).
Munot, K., et al. (2019). Importance of Dark Data and its Applications. In 2019 IEEE International Conference on System, Computation, Automation and Networking (ICSCAN) (pp. 1-6). IEEE.
O’Donnell, G. (1998). Accountability horizontal e novas poliarquias. In Revista Lua Nova, Nº 44; São Paulo. CEDEC.
Schembera, B. (2021). Like a rainbow in the dark: metadata annotation for HPC applications in the age of dark data. The Journal of Supercomputing, 77(8), 8946-8966.
Schembera, B., e Durán, J. (2020). Dark data as the new challenge for big data science and the introduction of the scientific data officer. Philosophy & Technology, 33(1), 93-115.
Shukla, M., et al. (2015). POSTER: WinOver enterprise dark data. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (pp. 1674-1676).
Stahlman, G., Heidorn, P., e Steffen, J. (2018). The astrolabe project: identifying and curating astronomical ‘dark data’ through development of cyberinfrastructure resources. In EPJ Web of Conferences (Vol. 186, p. 03003). EDP Sciences.
Trajanov, D., et al (2018). Dark data in internet of things (IOT): challenges and opportunities. In 7th Small Systems Simulation Symposium (pp. 1-8).
Zhang, C., et al. (2016). Extracting databases from dark data with deepdive. In Proceedings of the 2016 International Conference on Management of Data (pp. 847-859).
Published
2022-09-19
How to Cite
ALBUQUERQUE, Alessandro Marinho de; DORNELES, Carina F..
Dark data into the light of public control. In: BRAZILIAN SYMPOSIUM ON DATABASES (SBBD), 37. , 2022, Búzios.
Anais [...].
Porto Alegre: Sociedade Brasileira de Computação,
2022
.
p. 78-89.
ISSN 2763-8979.
DOI: https://doi.org/10.5753/sbbd.2022.224331.
