Skip to main content

Connecting the Dots: EOSC Projects enhancing scholarly data interoperability

  • : "CONNECTING THE DOTS: CROSS PROJECT CONVERSATIONS" IS ABOUT EXPLORING COLLABORATIVE EFFORTS ACROSS MULTIPLE PROJECTS TO ADVANCE THE FIELDS OF OPEN SCIENCE AND RESPONSIBLE RESEARCH ASSESSMENT. IN THIS SERIES OF DISCUSSIONS, WE WISH TO GIVE A VOICE TO REPRESENTATIVES OF PROJECTS AND INITIATIVES WITH WHOM WE COLLABORATE IN GRASPOS TO SHED LIGHT ON OUR WORK TOGETHER.

The first edition of Connecting the Dots is a discussion between four EOSC projects representatives about advancing scholarly data interoperability.

Stefania A1 Stefania AmodeoScilake  Andrea M1 Andrea MannocciGraspOS 
Matt B1 Matt BuysFAIRCORE4EOSC Elli P1 Elli PapadopoulouOSTrails

In view of the joint workshop they will hold at the EOSC Symposium 2024, they share insights into some of the questions surrounding their work on the development and implementation of the Scientific Knowledge Graph - Interoperability Framework.  

Can you tell us more about what we mean by interoperability when it comes to sharing data among infrastructures? 

Elli P1 Elli: Well, interoperability has different levels and all are needed depending on the context and what we want to achieve; it may refer to technical or semantic interoperability for example targeting metadata to ensure that systems can understand and use metadata from different sources or it may refer to data formats that enable systems that present technical differences with each other to exchange data. There is also legal interoperability to ensure data can be shared across different jurisdictions for example by using the same open licenses.

Matt B1 Matt: Interoperability refers to the use of common frameworks and standards that enable metadata to flow seamlessly between systems via open APIs, ensuring efficient and consistent data exchange.

Andrea M1 Andrea: Exactly, for example, within the context of open research information, interoperability can be seen also as a mechanism to feed back the results of local enhancement and curation, and overcome the "silo effect" across different infrastructures and converge to more uniform corpora.

Stefania A1 Stefania: In SciLake, using common specifications for sharing scientific knowledge  enables the creation of an ecosystem, that we call “Scientific Lake”, where open data, tools, and services from various infrastructures can be effectively combined and utilized in favor of knowledge discovery and research reproducibility.

What is the Scientific Knowledge Graph - Interoperability Framework (SKG-IF)? and how is it being used or tested in EOSC projects like GraspOS, SciLake, OSTrails, and FAIRCORE4EOSC?

Stefania A1 Stefania: In SciLake, we aim to extend the SKG-IF model to account for domain-specific entities and facilitate the integration of value-added services for research communities. Our goal is to extend the model in a structured way so it can be registered as an official variation of the SKG-IF. We're actively involved in developing this extension through our members' engagement in the SKG-IF working group of the Research Data Alliance (RDA).

Andrea M1 Andrea: Within the GraspOS project, we intend to use the SKG-IF as a means to exchange information in a uniform way across the federated data and service providers. Because of the peculiarities of the treated domain, which involves OS-aware responsible research assessment, we intend to extend the SKG-IF in order to tackle the specific needs of the project.

Elli P1 Elli: In OSTrails we bring services of both national and thematic research ecosystems to co-design IFs for DMPs, SKGs and FAIR assessments. For the purpose of our project, an SKG is defined as any database/repository with information that matches the data model of SKGs; this information can be research products, processes, actors, agents. We are working together in order to expose these databases/repositories in an SKG-IF compliant manner and to enhance the SKG-IF with extensions according to rich metadata supporting research data creation and analysis, for example information about instrument and facility usage. 

Matt B1 Matt: The SKG-IF establishes a standard designed to ensure interoperability and promote the reuse of metadata across the scholarly communication ecosystem. We are working to adopt this standard in EOSC projects to enhance data sharing and collaboration.

How does interoperability within the EOSC align with the broader vision of open science and collaborative research across Europe?

Andrea M1 Andrea: Interoperability is all about collaboration: reaching as much convergence as possible while maintaining diversity. It is a way to preserve local specificity of community data and services, and at the same time enable exchange of data to achieve multidisciplinary science.

Stefania A1 Stefania: I would add that interoperability allows for more efficient use of research resources by reducing duplication of efforts and enabling the integration of existing tools and services. This efficiency is key to advancing collaborative research on a European scale. Moreover, by enabling interoperable infrastructures, EOSC contributes to the long-term preservation and accessibility of research outcomes.

Elli P1 Elli: One of the major challenges in research, admittedly, is the silos that researchers are faced with during the research process which prevents them from accessing and reusing data. This is a serious shortcoming that, among others, Open Science aims to eliminate and EOSC facilitates this effort by providing solutions to connect research infrastructures across national and institutional boundaries. That way researchers from different fields can access and build upon each other’s work, thereby strengthening collaborations, as my colleagues just highlighted as well.

Matt B1 Matt: Interoperability across open infrastructures enables more effective reuse of community knowledge, reducing the burden on stakeholders. This alignment supports open science by creating richer metadata that better represents the entire spectrum of research activities, ultimately enhancing the scholarly record.

Are there specific outcomes you hope to achieve with your joint session aT the eosc symposium 2024?

Matt B1 Matt: We aim to expand the dialogue within the EOSC community and gather valuable insights from participants that will guide the direction of our development efforts.

Andrea M1 Andrea: I believe the workshop can be important to bring new perspectives and factors into the discussions happening in relevant working groups and interest groups under the Research Data Alliance.

Stefania A1 Stefania: I agree with my colleagues, it’s important to familiarise more members of the EOSC community with the SKG-IF, which is a robust international framework, and gather feedback on our development plans to reflect community needs. By bringing diverse stakeholders together, we aim to generate concrete ideas for advancing scholarly data interoperability in EOSC. The workshop also offers an excellent opportunity for our EOSC projects to collaborate and work on exploiting project results. I'm looking forward to a dynamic mix of learning, sharing, and planning that will enhance our collective efforts.

Elli P1 Elli: This is a good opportunity to learn what each one of us is involved in and how we support the EOSC-IF, which goes beyond SKGs. For example, in OSTrails we are also looking at interoperability between DMP platforms and between FAIR assessment tools, but also interoperability across SKGs, DMPs, FAIR assessments in support of the planning, tracking and assessing phases of research. Opening this discussion with the EOSC community is important first to get input in line with its members' practices and the EOSC vision, and then to achieve further collaborations for alignment between our projects.

Could you share some of the challenges you faced in your projects in the testing and implementation process?

Matt B1 Matt: One challenge we've faced is the existence of numerous standards across our community. Developing new standards sometimes leads to further fragmentation and silos. To address this, we need to focus on continuous review and building on existing efforts through strong collaboration.

Stefania A1 Stefania: Moreover, integrating resources from our project into the EOSC ecosystem presents significant challenges. The EOSC architecture is quite complex, particularly after recent major changes. On a positive note, several EOSC events have been helpful, and this workshop is especially crucial for fostering collaboration and developing collective solutions.

What are the next steps in the development and implementation of the SKG-IF within EOSC, and who are the specific stakeholders in the broader research community who should be involved in the process?

Andrea M1 Andrea: we are now transitioning to full semantic interoperability for our core model by leveraging and reusing standard ontologies and dictionaries, such as SPAR ontologies. We are engaging with relevant stakeholders, such as diverse SKG providers, and the projects that are intended to build on top of the framework.

How do you envision the SKG-IF evolving to meet the future needs of research data interoperability and discovery?

Elli P1 Elli: I would prefer to call it EOSC-IF in a few years time! That means to be a core component of the emerging EOSC nodes and to enhance SKGs with more information that EOSC covers in support of the whole research lifecycle and activities, e.g. for planning knowledge production. 

Andrea M1 Andrea: I hope the extension mechanism we are putting in place will spark the discussion around the SKG-IF and that it will be soon used to create multiple extensions of the core entities and relationships targeting different nuances and requirements that did not find an adequate representation in the first iteration of the framework.

 

Thank you very much Stefania, Elli, Andrea and Matt!  

To learn more about the SKG-IF and scholarly data interoperability in EOSC, don't miss out on their joint Unconference session at the EOSC Symposium 2024: EOSC collaborative frontiers to achieve interoperability and enhance scholarly data, 22 October, 11:30-12:30

Watch the trailer of the session below. 

 

Connecting the Dots: Monitoring Open Science at university level for research assessment

  • : "CONNECTING THE DOTS: CROSS PROJECT CONVERSATIONS" IS ABOUT EXPLORING COLLABORATIVE EFFORTS ACROSS MULTIPLE PROJECTS TO ADVANCE THE FIELDS OF OPEN SCIENCE AND RESPONSIBLE RESEARCH ASSESSMENT. IN THIS SERIES OF DISCUSSIONS, WE WISH TO GIVE A VOICE TO REPRESENTATIVES OF PROJECTS AND INITIATIVES WITH WHOM WE COLLABORATE IN GRASPOS TO SHED LIGHT ON OUR WORK TOGETHER.

The third edition of Connecting the Dots brings together representatives from the European University Association (EUA), the Université de Lorraine, and the University of Belgrade - Faculty of Chemistry in a discussion on Open Science monitoring for research assessment purposes. 

Ana Đorđević unibe

Ana Đorđević- University of Belgrade - Faculty of Chemistry, Coordinator

Rita Morais official 1 square

Rita Morais - European University Association, Adviser for Research & Innovation   

biljana kosanovic

Biljana Kosanović - University of Belgrade, Information specialist 

Nicolas Fressengeas.final red 2018

Nicolas Fressengeas - Université de Lorraine, Vice President for the digital, data and Open Science policies

Following the GraspOS Community of Practice meeting held on 22 January 2025, Ana Đorđević, Rita Morais, Biljana Kosanović, and Nicolas Fressengeas have accepted to answer our questions and share their knowledge and insights on monitoring of Open Science in their respective organisations.

Can you tell us more about your institution and how it is engaging with current efforts to reform research assessment?

Nicolas Fressengeas.final red 2018 Nicolas: Université de Lorraine is a large research intensive university of more than 4000 researchers and research support staff, with an important disciplinary scope, ranging from Humanities and Social Sciences to Science, Technology and Medicine. It has been strongly engaged in advancing research assessment ever since the beginning of 2022, when it contributed to the Paris Open Science European Conference, OSEC2022, that launched the Paris Call on Research Assessment. This call contributed to the creation of CoARA, the Coalition for Advancing Research Assessment, of which Université de Lorraine naturally became a member. At the time and following the signature of the DORA, the Declaration of San Francisco on Research Assessment, it had already engaged in producing internal advice and procedures to emphasise the qualitative nature of research assessment over a purely quantitative one. As of today, it has continued the process by involving the whole university research community starting in June 2024, with the aim to publish its CoARA action plan in the first half of 2025. 

biljana kosanovic Biljana: I’m working at the University of Belgrade Computer Centre and we developed and maintained sustainable infrastructure for 48 repositories from scientific and academic institutions in Serbia. We also developed the national CRIS eNauka (eScience). We introduce as much data on openness as possible into the national CRIS, as well as data that can be obtained by using open tools (e.g. OpenCitations). In this way, we present the existing data to the decision makers on evaluation, with the hope that they will include them in the construction of new indicators.

Ana Đorđević unibe Ana: I work as a librarian and manage the Cherry institutional repository at the University of Belgrade - Faculty of Chemistry. Cherry, developed by the University of Belgrade Computer Centre in 2018, was the first faculty-specific repository of its kind. In addition to managing Cherry, I also coordinate the GraspOS project at the Faculty, a project which focuses on Responsible Research Assessment - a cause which I am passionate about! As part of this initiative, we’ve integrated Cherry with infrastructures such as OpenCitations and BIP! Ranker to promote Open Science practices. We’re currently developing a new reward system to recognise and encourage researchers for good practices in Open Science and research transparency.

Rita Morais official 1 square Rita: The European University Association has been actively involved in research assessment reform for several years. Initially, the Association focused more closely on research assessment in the transition to Open Science. Over time, the reform of academic careers, including assessment, has become a strategic priority, as reflected in EUA’s strategic plan and its vision for universities in 2030. EUA has also been closely involved in the drafting of the Agreement on Reforming Research Assessment and in establishing CoARA. Currently, the Association coordinates the CoARA Working Group on Reforming Academic Career Assessment. At the same time, EUA continues to inform and raise awareness among its members about the reform of research assessment.

Does your institution have an established Open Science policy? If so, what are its main components? 

Ana Đorđević unibe Ana: Yes, we have an institutional Rulebook on Open Science at the Faculty. It is currently being revised to align with the national Open Science Platform 2.0 and our ongoing development of a new researcher reward system. The updated regulations will reflect these changes. The text of our current regulations is available here.

Nicolas Fressengeas.final red 2018 Nicolas: Yes it has! It gets its inspiration from the French national plan for Open Science. As such, its main components aim to open the production of its researchers as widely as possible. The term production encompasses the researcher’s writings — papers and books — as well as the research data and software they produce. However, Université de Lorraine strongly suggests that article publication charges are not the way to go towards open access, as it is not sustainable in the long run and inherently induces conflict of interest at the publisher level. In the same spirit, the University has a cautious approach towards so-called transformative agreements and recommends the use of community-owned or controlled infrastructures for the opening of all research productions.

Rita Morais official 1 square Rita: The work of EUA on Open Science is framed by the Association’s Open Science Agenda, which defines three main priority areas for EUA’s work on Open Science: Open Access to scholarly outputs in a just scholarly publishing ecosystem, FAIR research data, and research assessment. On Open Access, EUA advocates for academic ownership of scholarly communication and publishing, as well as a just scholarly communication ecosystem (i.e. transparent, diverse, economically affordable and sustainable, technically interoperable, and steered by the research community). EUA calls for FAIR research data as the norm in producing and sharing scientific knowledge, for new professional profiles for data-intensive careers, and for an active engagement in EOSC. On research assessment, the Association advocates for a responsible, transparent, and sustainable research assessment system, for the inclusion of Open Science as part of research assessment practices, and assessment approaches that balance qualitative and quantitative metrics.

biljana kosanovic Biljana: I am more involved at the national level. The Open Science Platform 2.0, the new national policy in Serbia, was approved in December 2024 and emphasises five key goals: (1) Open Access to scholarly publications; (2) availability of research data; (3) open and transparent access to research infrastructures; (4) transparency of scholarly communication and methodology, including the availability of software source code, design documentation of research hardware and other digital objects used in the analysis of research data, and (5) the development of the digital infrastructure and competencies that make it possible to achieve the goals of the policy. Its main elements are described here

Does your institution monitor Open Science practices for research assessment purposes? Are there specific indicators or frameworks used to measure Open Science contributions?

biljana kosanovic Biljana: At the institutional level, the OpenAIRE Monitor Dashboard is used but only for an overall picture. At the national level, we have incorporated a similar measure of openness within eScience. Unfortunately, during the discussion on the national policy (just adopted, and mentioned above), no consensus was reached, but it was written: "The Ministry will define mechanisms for rewarding researchers who consistently follow the principles of Open Science through publishing research data, software and hardware in open access, pre-registration of research, pre-prints, open peer review, citizen science projects, development of free and open-source software and open hardware, open materials, etc. Institutions should encourage and reward Open Science practices as far as their resources allow.” It certainly shows a clear desire, but not the intention to put it into practice right now.

Ana Đorđević unibe Ana: We are proud to be part of the OpenAIRE Monitor Dashboard, where the main insights we gain include key metrics on our research outputs, Open Access compliance, and collaboration with international research funders. In partnership with the University of Belgrade Computer Centre and supported by the Horizon Europe GraspOS project, we are developing the first institutional researcher reward system in Serbia. This system is based on three key principles:

1. Awarding badges to researchers who share their scientific results through green open access via the institutional repository Cherry. This enhances the visibility of scientific works, especially those whose published versions are restricted due to publisher rights;

2. Awarding badges to departments that facilitate green Open Access by encouraging researchers to share their results through Cherry;

3. Awarding badges to researchers who actively participate in seminars on Open Science and Responsible Research Assessment, organised by librarians.

Through these initiatives, we aim to incentivise and recognise contributions to Open Science practices and Responsible Research Assessment.

Rita Morais official 1 square Rita: Since 2016, EUA has conducted several large-scale surveys among its members, gathering information on the establishment of Open Science policies and practices. The 2019 and 2020-2021 editions specifically examined Open Science in research assessment in European universities, showing that the inclusion of Open Science elements in academic assessments remained limited. More recently, findings from the CoARA Working Group on Reforming Academic Career Assessment indicate that Open Science activities are considered low or moderately important in researchers’ career evaluation, but this is expected to improve as reforms on research and academic career assessment are implemented.

Nicolas Fressengeas.final red 2018 Nicolas: The answer is no. Université de Lorraine is nonetheless highly engaged in Open Science monitoring as it heads the French national project that allows to releasethe French Open Science monitor. It is also deeply involved in theOpen Science monitoring initiative which aims, withUNESCO and theFrench Ministry for Higher Education and Research, to bring together Open Science monitoring initiatives worldwide. As such, it providesa detailed monitor on its own Open Science achievements, providing many indicators, among which are the Open Access rate and the proportions of papers that share their data and software.

However, the purpose of this monitoring is not directly linked to research assessment. It rather sheds light on the road to crafting and adapting evidence-based Open Science policies. It also allows showcasing the opening of science by a given laboratory or a larger research community. If these communities choose to do so, they can however ask to add this monitoring in the assessment of their research.

Initiatives like CoARA have highlighted the need to base research assessment primarily on qualitative evaluation for which peer review is central, supported by responsible use of quantitative indicators. Are there ways in which your institution promotes the responsible use of metrics in research assessment?            

Rita Morais official 1 square Rita: EUA is a member of CoARA and, as outlined in our Action Plan, we will continue to support our members in reflecting on institutional approaches to research assessment. We will also continue to advocate for responsible, transparent and sustainable assessment practices, raise awareness, build capacity and create support for university initiatives among the main actors (including [early-career] researchers, other research performing organisations, research funders and policy makers).

Nicolas Fressengeas.final red 2018 Nicolas: This is all the purpose of our commitment to CoARA. As suggested above, we had already modified our message to research evaluators to engage them in a more qualitative process. This process will be asserted in 2025 by our CoARA action plan, the aim of which is exactly this goal.

biljana kosanovic Biljana: There is clear dissatisfaction in the research community with the current indicators, but no clear proposals for how this could be changed. From our side (primarily  IT implementation though eNauka), we show that such measures exist, and that there is space for discussion about them.

Ana Đorđević unibe Ana: While we are not currently participating in the CoARA initiative, we are committed to promoting Responsible Research Assessment. We are actively exploring ways to integrate qualitative evaluation with responsible use of quantitative indicators to ensure a balanced and fair approach to research assessment within our institution.

What are the main challenges your institution faces in reforming research assessment? What specific services or infrastructures are needed to further support Open Science-aware research assessment? 

biljana kosanovic Biljana: Serbia is a small scientific community (max. 25,000 researchers at all levels, including PhD students), and therefore all evaluation decisions are made at the national level. The institutional policies are grounded in national-level legislation. Through the funding of eNauka, the Ministry enables comprehensive and public access to all outputs produced by the scientific community. This initiative provides hope that the Ministry will be prepared to amend normative acts and legislative regulations concerning evaluation practices.

Ana Đorđević unibe Ana: In my perspective, a significant challenge we face is the lack of librarians at the faculties. This lack of specialised staff makes it difficult to identify and engage the right experts to organise and conduct Open Science seminars and workshops - efforts that are crucial for advancing Open Science practices within the research community. To better support Open Science-aware research assessment, it is essential that we invest in the training and engagement of more librarians and Open Science specialists, in order to build the infrastructure needed for these initiatives.

Nicolas Fressengeas.final red 2018 Nicolas: Recognising the efforts that it takes to implement Open Science is only one driver, among many, of the reform. Indeed, the focus on publish or perish also hinders other research activities such as its links with society. We feel that the main challenge is the need for a coordinated global culture change. As Toma Susi puts it, Open Science needs no martyrs: any local only change is doomed to fail. This worldwide coordination is CoARA’s goal.

We feel that Open Science-aware research assessment needs to solely rely on open data: on data that can be accessed equally by assessors and assessees. This is the reason Unversité de Lorraine has redirected its data curation workforce from closed databases to the open OpenAlex. Consequently, and until OpenAlex open data can be fully trusted for this type of activity, Université de Lorraine has taken its distances with international rankings, most of which are closed-source based. This last step is crucial to unlock the possibilities of real change.

Rita Morais official 1 square Rita: As Nicolas pointed out, reforming research assessment requires a change in academic culture – a process that is both long and challenging. Many institutions are eager to implement reforms aligned with DORA and CoARA, and are taking important steps in that direction. There is a clear momentum for change. At the same time, there is a lot of uncertainty on how to implement reforms in practice, how these will unfold and which consequences they may have, particularly for early-career researchers. The success of reform efforts depends on multiple factors, including stakeholder engagement, available resources, academic cultures, and regulatory and policy frameworks – each of which can either facilitate or hinder reform efforts. But what we have seen, and gathered data on, is that the motivation to change, one step at a time, remains fundamental to reform progress. Peer learning and exchanges between universities, participation in platforms like CoARA and other initiatives on reforming research assessment, national and international coordination, are all needed to sustain the momentum for change in research assessment.

 

Thank you very much for your time Ana, Rita, Biljana, and Nicolas!  

To learn more about the role of context in Responsible Research Assessment, we invite you to take a look at the slides from the Community of Practice meeting which have been made available here.

Connecting the Dots: The Barcelona Declaration on Open Research Information – Openness of research information as a prerequisite for research assessment reform

  • : "CONNECTING THE DOTS: CROSS PROJECT CONVERSATIONS" IS ABOUT EXPLORING COLLABORATIVE EFFORTS ACROSS MULTIPLE PROJECTS TO ADVANCE THE FIELDS OF OPEN SCIENCE AND RESPONSIBLE RESEARCH ASSESSMENT. IN THIS SERIES OF DISCUSSIONS, WE WISH TO GIVE A VOICE TO REPRESENTATIVES OF PROJECTS AND INITIATIVES WITH WHOM WE COLLABORATE IN GRASPOS TO SHED LIGHT ON OUR WORK TOGETHER.

The second edition of Connecting the Dots brings together representatives from CoARA, the Barcelona Declaration on Open Research Information and GraspOS to discuss ways in which open research information is needed for research assessment reform. 

Stefania A1 Bianca Kramer Barcelona Declaration Organising Team  
Andrea M1 Thanasis VergoulisGraspOS Lead Technical Manager  
Elli P1 Janne Pölönen CoARA Steering Board member  

Following the GraspOS Community of Practice meeting held on 13 November 2024, Bianca Kramer, Thanasis Vergoulis and Janne Pölönen have accepted to answer our questions on ways in which open research information can support and strengthen reform in research assessment practices. 

Why IS the transition from closed to open research information important, and who will benefit from it?

Elli P1 Janne: There are various benefits of transitioning to open research information. Closed commercial databases and tools are expensive and exclusive, so open research information is more equitable if organisations and researchers around the world have free access to research information needed for research assessment and for determining research impacts. The most frequently used commercial databases, Web of Science (WoS) and Scopus, are also very exclusive about literature from which publications and citations are indexed. Therefore, open research information may also promote inclusiveness, if it covers a broader range of contributions irrespective of communication language, format or context, than WoS and Scopus, which prioritise peer-reviewed articles in international English language journals.

Andrea M1 Thanasis: All the previous points are accurate and to the point. Furthermore, it is clear that applications, analyses, and processes relying on proprietary data sources locked behind paywalls suffer from a lack of transparency. This restricts the ability of the research community to independently validate the findings or provide alternative interpretations of the data.  

Stefania A1 Bianca: Indeed, for researchers, these biases and lack of transparency affect them when they are being evaluated or assessed, and also when they want to find scientific information using bibliographic databases and search engines. Open research information will thus benefit researchers and research performing and funding organisations directly.   

What does the signing of the Barcelona Declaration mean? In concrete terms, what will it imply for signatories?

Elli P1 Janne: Signing the Barcelona Declaration signals recognition of the importance of open research information, and a commitment to making strategic choices that in the long run contribute to quality, development and use of open research information in research assessment.

Andrea M1 Thanasis: Yes, I agree. By signing the Barcelona Declaration research performing organisations make it clear that they recognise the challenges with the current status quo and acknowledge that change is needed. In addition, each organisation that signed the Declaration commits to actively supporting open research information providers in various ways and contributing to the broader dialogue on how to effectively realise the shared vision of the Declaration.  

Stefania A1 Bianca: It is also important to emphasise that the commitments in the Declaration are intended to be aspirational - we do not expect signatories to already meet all commitments before signing the Declaration, or to meet them in a specific time period. There is, however, the expectation for signatories to actively work towards the commitments within their organisation. One thing we want to facilitate is for organisations (both signatories and supporting providers of data, services and infrastructure) to work together to make this transition easier and quicker. 

In what ways can open infrastructures support an Open Science-aware research assessment system?

Stefania A1 Bianca: Open research information can make research assessment more transparent and fair. It allows for all information used to be openly accessible, and it also allows research assessment to be performed in a more equitable way - not being dependent on selective, closed databases, but with more agency over the data being used (from a number of different sources). Open infrastructures can support this by making sure that the research information they collect and provide access to is both open and FAIR - for instance, by using persistent identifiers, allowing full access (and reuse rights) to underlying data, and making the data interoperable. 

Elli P1 Janne: One of the main goals of the responsible research assessment movement is to recognise and reward Open Science practices. This includes transparency – following the principle “As open as possible, as closed as necessary” – of the research processes, and by extension, assessment processes. Another aspect, already highlighted in the first question, is that open research information may potentially support recognition of the broader range and diversity of Open Science practices and contributions. 

Andrea M1 Thanasis: Yes, exactly. Implementing research assessment practices based on open infrastructures and guided by Open Science principles, is increasingly recognised as a prerequisite for achieving the goal of making research assessment practices more responsible and inclusive. At the same time, it is important to emphasise the reciprocal relationship: an Open-Science-aware research assessment framework can, in turn, benefit Open Infrastructures and the broader Open Science ecosystem by encouraging and rewarding the adoption of Open Science practices.  

What are the next steps in your organisation regarding the joint roadmap*?

*A joint roadmap for open research information was recently developed at the Paris Conference on Open Research Information (23-24 September 2024)

Andrea M1 Thanasis: From the GraspOS perspective, the vision of the project is well-aligned with the Declaration’s vision for a transition from closed to open research information. Furthermore, various project partners have already signed or supported the Barcelona Declaration, with their representatives willing to actively participate in the discussions within the newly forming working groups. These partners are also working to develop internal action plans that align with the key commitments outlined in the Declaration. 

Elli P1 Janne: CoARA has welcomed but has not yet formally endorsed or signed the Barcelona Declaration. However, it is one of the principles of the Agreement on Reforming Research Assessment to “Ensure independence and transparency of the data, infrastructure and criteria necessary for research assessment and for determining research impacts”. To facilitate implementation of this principle, CoARA has a thematic Working Group “Towards Open Infrastructure for Responsible Research Assessment” (OIRRA). One further step for the CoARA community could be to support the Barcelona Declaration, which indicates a clear pathway to open research information.

Stefania A1 Bianca: For the Barcelona Declaration itself, the most important next step is that we are currently setting up working groups for signatories and supporters to work together on a number of topics on the joint roadmap. The roadmap contains a set of actions that were formulated by signatories and supporters at the Paris Conference on Open Research Information in September 2024. These are mostly actions where organisations together can do more than each individual organisation (like sharing information and expertise, formulating minimal standards, and acting together rather than individually). We will provide working groups with organisational support, and also want to work closely together with other related initiatives, to reinforce each other and prevent duplicate efforts. 

In your opinion, what are some of the challenges that infrastructures for open research information could face in the near future?

Elli P1 Janne: There are several challenges. Information on research outputs, activities and impacts produced by researchers, institutions and infrastructures is scattered across various platforms and difficult to use and reuse systematically in assessments. Therefore, we need to improve interoperability. Another important issue is how can we achieve sufficient quality, structure and robustness of open research information needed for research assessment purposes. In addition, one important barrier for relying on open research information is that access and analytics require relatively high technical competence compared to the ready-made commercial solutions. It is important to dedicate effort to develop analysis tools and dashboards to support use of open research information.

Andrea M1 Thanasis: I believe that Janne provided an excellent overview of this topic. If I were to add anything, it would be that open infrastructures should find proper ways to implement curation mechanisms to achieve improved coverage and quality for certain types of metadata, like affiliations and topics, potentially leveraging the networks of volunteer experts, such as those involved in initiatives like the Barcelona Declaration. 

Stefania A1 Bianca: I would add that with all these requirements and expectations, infrastructures for open research information need to be actively used as well as financially supported to be able to continue to exist and further develop. This also puts a responsibility on research performing and funding organisations. By engaging with infrastructures, they can actively contribute to the continued improvement of data quality and functionality, and by reinvesting money currently spent on closed, proprietary systems, they can contribute to their sustainability. 

 

Thank you very much for your time Bianca, Janne and Thanasis!  

To learn more about the Barcelona Declaration and openness of research information for research assessment reform, we invite you to take a look at the slides from the Community of Practice meeting which have been made available here.