The D-NET software toolkit

Manghi, Paolo; Artini, Michele; Atzori, Claudio; Bardi, Alessia; Mannocci, Andrea; Bruzzo, Sandro La; Candela, Leonardo; Castelli, Donatella; Pagano, Pasquale

doi:10.1108/prog-08-2013-0045

Cited by 16 publications

(4 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is achieved by supporting standard protocols, e.g., OAI-PMH (Lagoze and Van de Sompel, 2001), as well as webbased proprietary APIs. Such facilities are at the core of aggregative infrastructures (Manghi et al, 2014), e.g., OpenAIRE counts on the Zenodo OAI-PMH service to collect datasets published by this repository. In the case of programmatic APIs, they can also support the deposition phase, thus favouring the integration of the repository in publishing workflows.…”

Section: Dataset Discovery and Accessmentioning

confidence: 99%

Are Scientific Data Repositories Coping with Research Data Publishing?

Assante

Candela

Castelli

et al. 2016

Data Science Journal

View full text Add to dashboard Cite

Research data publishing is intended as the release of research data to make it possible for practitioners to (re)use them according to "open science" dynamics. There are three main actors called to deal with research data publishing practices: researchers, publishers, and data repositories. This study analyses the solutions offered by generalist scientific data repositories, i.e., repositories supporting the deposition of any type of research data. These repositories cannot make any assumption on the application domain. They are actually called to face with the almost open ended typologies of data used in science. The current practices promoted by such repositories are analysed with respect to eight key aspects of data publishing, i.e., dataset formatting, documentation, licensing, publication costs, validation, availability, discovery and access, and citation. From this analysis it emerges that these repositories implement well consolidated practices and pragmatic solutions for literature repositories. These practices and solutions can not totally meet the needs of management and use of datasets resources, especially in a context where rapid technological changes continuously open new exploitation prospects.

show abstract

Section: Dataset Discovery and Accessmentioning

confidence: 99%

Are Scientific Data Repositories Coping with Research Data Publishing?

Assante

Candela

Castelli

et al. 2016

Data Science Journal

View full text Add to dashboard Cite

show abstract

“…The injection of a ground truth in the graph is performed before the deduplication process is fired. This can happen manually via shell scripts or via APIs (as in the case of OpenAIRE, where data processing activities are managed by D-NET orchestration workflows (Manghi et al , 2014)). The process is performed by Map jobs that scan the HBase table for the given ground truth and for each row create a representative row in the HBase graph table, virtually delete rows and columns deprecated by the merging process and adds the inverse relationships towards representative objects.…”

Section: Gdup Implementationmentioning

confidence: 99%

Entity deduplication in big data graphs for scholarly communication

Manghi

Atzori

Bonis

et al. 2020

DTA

Self Cite

View full text Add to dashboard Cite

PurposeSeveral online services offer functionalities to access information from “big research graphs” (e.g. Google Scholar, OpenAIRE, Microsoft Academic Graph), which correlate scholarly/scientific communication entities such as publications, authors, datasets, organizations, projects, funders, etc. Depending on the target users, access can vary from search and browse content to the consumption of statistics for monitoring and provision of feedback. Such graphs are populated over time as aggregations of multiple sources and therefore suffer from major entity-duplication problems. Although deduplication of graphs is a known and actual problem, existing solutions are dedicated to specific scenarios, operate on flat collections, local topology-drive challenges and cannot therefore be re-used in other contexts.Design/methodology/approachThis work presents GDup, an integrated, scalable, general-purpose system that can be customized to address deduplication over arbitrary large information graphs. The paper presents its high-level architecture, its implementation as a service used within the OpenAIRE infrastructure system and reports numbers of real-case experiments.FindingsGDup provides the functionalities required to deliver a fully-fledged entity deduplication workflow over a generic input graph. The system offers out-of-the-box Ground Truth management, acquisition of feedback from data curators and algorithms for identifying and merging duplicates, to obtain an output disambiguated graph.Originality/valueTo our knowledge GDup is the only system in the literature that offers an integrated and general-purpose solution for the deduplication graphs, while targeting big data scalability issues. GDup is today one of the key modules of the OpenAIRE infrastructure production system, which monitors Open Science trends on behalf of the European Commission, National funders and institutions.

show abstract

“…Oferece serviços de gerenciamento de dados capazes de fornecer acesso a diferentes tipos de fontes de dados externas, armazenar e processar objetos de informações de qualquer modelo de dados, convertê-los em formatos comuns e expor objetos de informações a aplicativos de terceiros por meio de vários acessos padrão (MANGHI et al, 2014).…”

Section: Instituto De Ciência E Tecnologia Da Informação -D-netunclassified

Recuperação de informação: descoberta e análise de workflows para agregação de dados do patrimônio cultural

Siqueira,

Lopes Martins

2020

Cionline

View full text Add to dashboard Cite

In recent years, different cultural institutions have been making efforts to spread culture through the construction of a unique search interface, which integrates digital objects and facilitates data retrieval for lay users. However, integrating cultural data is not a trivial task, as these are diverse and unique, requiring a variety of steps between collection and presentation. In order to identify these steps, this research aims to locate data aggregation workflows and discuss them. To this end, descriptive and bibliographic research, of a qualitative nature, was carried out in academic databases and in gray literature. As a result, eight projects are presented: American Art Collaborative, DigitalNZ, D-NET Software, Europeana, Mexicana, Parthenos Aggregator, TROVE and UNLV's Linked Data Project. The analysis of the set of workflows resulted in eight different steps to be performed: 1. Extract, 2. Structure, 3. Transform, 4. Reconcile, 5. Store, 6. Publish, 7. Expose and 8. Enable new applications. In addition, the need for more detailed stages is also visible, so that it is possible to replicate the workflow, and enjoy its benefits in other institutions.

show abstract

The D-NET software toolkit

Cited by 16 publications

References 35 publications

Are Scientific Data Repositories Coping with Research Data Publishing?

Are Scientific Data Repositories Coping with Research Data Publishing?

Entity deduplication in big data graphs for scholarly communication

Recuperação de informação: descoberta e análise de workflows para agregação de dados do patrimônio cultural

Contact Info

Product

Resources

About