Digital resources in the Social Sciences and Humanities OpenEdition Our platforms OpenEdition Books OpenEdition Journals Hypotheses Calenda Libraries OpenEdition Freemium Follow us

Tag: BAD

The Book Analytics Dashboard project: reflections on coordination, collaboration, and community consultation

This blog post was initially published by Kathryn Napier on the the Curtin Open Knowledge Initiative (COKI) website on May 30, 2023.

The Book Analytics Dashboard (BAD) project (2022-2025) is a 3-year, Mellon Foundation funded project that is creating a sustainable analytics service to support diverse Open Access (OA) book publishers. Affectionately referred to as the BAD project, our goal is to provide publishers with user-friendly tools to navigate complex data about how their books are being used. The project is grounded in the premise that efficient, user-friendly usage analytics services are needed to safeguard and support diversity in the voices, perspectives, geographies, topics, and languages made visible through OA books. 

The BAD project is building on the earlier (2020-2022) Mellon-funded Developing a Pilot Data Trust for Open Access Ebook Usage project, affectionately referred to as the OAeBU project. The BAD project is scaling workflows, infrastructure and customer support processes originally developed during the earlier project. In addition to technical refinement and scaling, BAD is developing a long-term plan for housing, maintenance, and funding of the analytics service as a sustainable community infrastructure.

The team working on BAD is truly international. The Principal Investigator (PI) team comprises: Lucy Montgomery from Curtin University (Australia); Cameron Neylon (Curtin University); Niels Stern and Ronald Snijder (OAPEN Foundation, the Netherlands), as well as community cultivation expert Katherine Skinner (Research Lead at IOI, based in the United States). 

I lead the project’s technical team, which is based at Curtin University and has close links to the Curtin Institute for Computation. The Curtin team is working closely with OAPEN to support the development of the governance and sustainability model needed to ensure the long-term stability of a dashboarding service. The whole project team meets weekly, coordinating a series of teleconference meetings and coworking sessions across hemispheres and timezones.

Cameron Neylon (Curtin University), Rebecca Handcock (Curtin University), Kathryn Napier (Curtin University), Sînziana Păltineanu (OAPEN Foundation), Laura J. Wilkinson (OAPEN Foundation), Ronald Snijder (OAPEN Foundation), Lucy Montgomery (Curtin University), Katherine Skinner (Formerly Educopia Institute, now IOI), Niels Stern (OAPEN Foundation) meeting online in March 2023.

Consult, listen, iterate

Engaging with community feedback productively has demanded willingness, not just to consult stakeholders, but also to listen and iterate technical directions. The decision to move away from Elasticsearch Kibana in favour of Looker Studio during the first year of the project is an example of how this process of consulting, listening, and iterating has worked in practice. 

In 2022, when the BAD project kicked off, an important first priority was community consultation and the development of the first annual Technical Roadmap. A draft technical roadmap was made available for community comment and feedback in June-July 2022. The draft technical roadmap had a dual purpose: making technical systems and plans visible to stakeholders; and inviting those stakeholders into a conversation about where effort and resources should be focussed. 

The original project plan placed a heavy focus on standardising technical workflows developed during the previous OAeBU project, in order to support scaling of a dashboard service. This included the use of ElasticSearch Kibana, a system chosen because of its ability to allow for fine-grained user access roles and permissions, and well suited to keeping the information contained in dashboards private. However, ElasticSearch Kibana is less well-suited to a publisher-focussed dashboard service that engages with the value of dashboards to support advocacy and public communication. Being able to easily make dashboards public, embedding dashboards within publisher websites, and simple, visually appealing user interfaces required a different solution.

An example of pilot dashboards produced using ElasticSearch/Kibana during the OAeBU project.

Community consultation and feedback in the early stages of the BAD project highlighted important differences between the primary use cases imagined during the OAeBU project, which focussed on keeping data private; and the needs of users focussed on external messaging and public sharing of visualisations. Improving user experience needed to become a higher priority. Rather than simply standardising ElasticSearch Kibana workflows, the BAD dashboards needed both a visual and technical revamp. Technical work shifted focus to producing dashboards that can be used both publicly for marketing and communication and internally for reporting by publishers.

An example of the BAD dashboards produced using Looker Studio during the first year of the BAD project (see the template dashboard here, and the University of Michigan Press’ dashboard embedded in their website here).

We therefore shifted our focus on technical priorities in the updated technical roadmap (released in January 2023) to investigating alternative dashboarding solutions, and produced a template dashboard using Looker Studio, a dashboarding solution offered by Google. This template dashboard formed the basis of focus group community consultations that took place in the first quarter of 2023, and the dashboard design will continue to iterate and incorporate feedback through the life of the BAD project.

Virtual, in-person, hybrid: working as a truly global team

The first year of the BAD project presented both challenges and exciting opportunities. Coordinating project management and deliverables across a multinational and multidisciplinary team is not a simple exercise, and we have discovered that a combination of detailed in-person meetings and workshops with regular teleconference meetings has resulted in the best outcomes for the project. The lightbulb moment that occurred in late 2022 where we shifted our technical priorities and dashboarding system would not have occurred without a series of in-person meetings.

Laura J. Wilkinson (OAPEN Foundation), Ronald Snijder (OAPEN Foundation), Lucy Montgomery (Curtin University) and Cameron Neylon (Curtin University) meeting in person for a series of project workshops in The Hague, October 2022.

This shift in technical focus came at an ideal time within the first year of the project. We are now well placed to further refine the dashboard design through the feedback received from the focus groups (detailed in the draft technical roadmap for year 2), which will also inform the continued development of the dashboarding services sustainability, governance and financial models. We will continue to endeavour to build the best service we can with the combined talents from our multidisciplinary team.

Kathryn Napier

Learn more about the BAD project

Sînziana Păltineanu

More Posts

Top findings from the BAD focus groups

The Book Analytics Dashboard (BAD) Project (2022-2025) is focused on creating a sustainable open access (OA) book focused analytics service. During this project, we are scaling up from our prototype dashboard to a fully functioning service. New to BAD? Learn more about the people, background, and work packages in our project overview.

In the first quarter of 2023, we carried out a series of focus groups with publishers to test some ideas and get their feedback to help inform our future decisions.

We really enjoyed meeting the participants and we thank them warmly for their participation. So, what did we learn from our interactions with them? You can jump to the full findings on Zenodo (which includes our methodology and instruments), explore the BAD template dashboard (currently featuring University of Michigan data – thank you U of M!), or continue reading here for a summary.

How will the Book Analytics Dashboard help publishers?

Our participants told us that this data would provide them with a better understanding of the usage of their OA books across different countries, including which titles are most popular, how language and translation affects circulation, and comparing users’ preferences for full books vs chapters. Participants largely agreed that one of the most important aspects of the service is its ability to combine, normalise, deduplicate, and visualise OA data so that it can be quickly used and trusted. 

“Our current articulation of OA is narrative-, not data-driven, so it’s very labour intensive”

Some participants also hoped the data might help inform their publishing decisions; others hoped it would inspire acquiring editors and other decision makers to become more engaged with usage data. We also heard from participants that having tangible dashboards and reports to show people would be very helpful in getting editorial boards, authors, and funders to understand OA impact. Publishers acknowledge that this needs to be done with appropriate context and explanation, but also note that this type of data is increasingly requested by authors and funders.

A key value proposition of BAD is that it allows easy synthesis and consolidation of disparate data sources, which is otherwise a very manual process that is often conducted by publishing staff members.

Features of the Book Analytics Dashboard

Publishers appreciate being able to download visual elements of the dashboard and export data from it. They value the accuracy of the data sources used, allowing them to make comparisons. Participants encouraged us to maximise this by making provenance information available so that users know that we’re providing standardised and good-quality data. 

The focus group participants identified a number of features we can improve or add, and these will be reviewed by the technical team for inclusion in the technical roadmap for years two and three of the project (see the technical roadmap for year one).

Publishers helped us explore which features would be present in different levels of the service, for example a standard dashboard with additional features as a premium offer (such as an API, integrations, or more detailed reports). It would also be possible to include additional ONIX streams (for example, from non-OA titles) as an à la carte option.

Financial sustainability

Though few participants are immediately ready to commit financially to the dashboard service, they widely acknowledged that such data and dashboards will be essential for their operations in the not-too-distant future. We discussed a number of options for pricing, and there was agreement that there should not be a flat fee for all publishers.

Community governance

We asked publishers what they wanted and expected in terms of community governance qualities or characteristics for BAD. To our surprise, many participants were reluctant to commit to being involved due to bandwidth constraints. There was a strong preference for consultative rather than democratic engagement with BAD governance. This is quite a contrast with other open scholarly infrastructures where community participation in governance of the organisation is considered a key value. Perhaps this is because people realise how much time and effort it requires to stay abreast of developments and think deeply about these aspects of work; it also might stem from cultural differences in scholarly publishers (as compared with librarians). It could also be because of the participants’ high levels of trust in OAPEN, the suggested long-term home for BAD.

Publishers welcomed being part of a community around OA books usage data as long as that did not demand they undertake governance responsibilities. They appreciate periodic stakeholder reports and annual meetings, as well as opportunities to contribute to a technical roadmap – though participants do not think that they should make decisions about technology.

“We can make suggestions, but should not fly the plane.”

As we expected, participants told us that it was important that the OA books data is provided in trust, not sold on, and is only used appropriately. Transparency is key; as is the not-for-profit status of this service.

Many publishers were happy with the idea of BAD becoming an OAPEN service under OAPEN’s existing governance. 

Next steps

The focus groups findings have given us plenty to think about! Our technical team are currently considering all the feature requests and tweaks that we discovered, and our findings will also directly feed into our work around sustainability and modelling different pricing possibilities.

Laura J. Wilkinson & Katherine Skinner

Learn more about the BAD project

Laura J. Wilkinson

ORCID iD: https://orcid.org/0000-0002-8922-7839

More Posts