It’s a new year and at OurResearch we’re starting off 2023 full steam ahead! We’ve revamped the OpenAlex documentation so that it’s easier to get started, and easier to find the fields and filters that are available in the OpenAlex API. It should take less “clicks” to find what you need.

Poised for growth

The major change we made was to highlight the core entities (works, authors, etc) in OpenAlex, giving them their own up-front space. OpenAlex grew considerably in 2022, not only in number records, but also by the number of ways that you can filter, group, and search scholarly data. This new approach provides more room to add and document filters. We can better describe the unique search capabilities available in each entity. Overall, it sets us up to grow again in 2023.

Our goal is to maintain friendly and approachable documentation, so hopefully we’ve kept that up as well. If you find something broken, or have some suggested improvements, let us know!

We’ve improved the author search feature within OpenAlex, so you get more results when searching for author names that may or may not include diacritics. For example, a search for the name “David Tarragó” will return the same number of results as the the version that is converted via Lucene’s ASCII folding filter, which in this case is “David Tarrago”.

When searching with diacritics, results with the queried diacritics are more likely to be ranked towards the top. So the two searches may have slightly different rankings. You can see the results of these two searches in the API:

• Search for David Tarragó: https://api.openalex.org/authors?search=david%20Tarrag%C3%B3
• Search for David Tarrago: https://api.openalex.org/authors?search=david%20Tarrago

These queries return the same number of results, with diacritic and non-diacritic names included. Keep in mind that results are weighted by the author’s works count, so that has an impact on relevance as well.

Why make this change?

When creating the OpenAlex author search capability, it was important for us to honor author’s names by respecting diacritics. So searching with a diacritic returned results with diacritics. However, this strict approach makes it harder to find some authors. We’re comfortable with the compromise of searching with and without diacritics at the same time, while giving priority to the intended search query. Hopefully this improved feature is helpful!

Did you know that you can request up to 50 DOIs in a single API call? That’s possible due to the OR query in the OpenAlex API and looks like this:

https://api.openalex.org/works?filter=doi:10.3322/caac.21660|https://doi.org/10.1136/bmj.n71|10.3322/caac.21654&mailto=support@openalex.org

We simply separate our DOIs with the pipe symbol ‘|’. That query will return three works associated with the three DOIs we entered. As you can see in the query, a short form DOI or long form DOI (as a URL) are both supported.

This will save time and resources when requesting many DOIs. This technique works with all IDs in OpenAlex, to include OpenAlex IDs and PubMed Central IDs (PMID).

Example with python requests

Let’s write an example python script to show how we can get DOIs in batches of 50 using requests:

import requests

dois = ["10.3322/caac.21660", "https://doi.org/10.1136/bmj.n71", "10.3322/caac.21654"]
pipe_separated_dois = "|".join(dois)
r = requests.get(f"https://api.openalex.org/works?filter=doi:{pipe_separated_dois}&per-page=50&mailto=support@openalex.org")
works = r.json()["results"]

for work in works:
  print(work["doi"], work["display_name"])

# results
https://doi.org/10.3322/caac.21660 Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries
https://doi.org/10.1136/bmj.n71 The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
https://doi.org/10.3322/caac.21654 Cancer Statistics, 2021

Hope this is helpful!

Hi I’m Casey. I am excited to announce that I am now full time with OurResearch as a software engineer working on OpenAlex and Unpaywall!

My Journey

I freelanced for OurResearch prior to joining full time this summer. With Jason and Heather’s help I maintained Paperbuzz, Cite-As, and also built out a project to catalog academic journal pricing. With freelancing I was able to improve my python and data management skills in order to tackle bigger projects.

Prior to freelancing I enjoyed a career in the US Air Force, which I am proud of. I’m fortunate to have hundreds of hours as aircrew on multiple aircraft, as well as a variety of technical and leadership assignments. So if you ever want to talk airplanes be ready because I might talk your ear off!

My academic experience comes from my time in university pursuing advanced education.

My Vision with OurResearch

In December I helped build the API and set up Elasticsearch for a project called OpenAlex. That project has continued to grow and I love to see how many people are using it. My core job with OpenAlex is to provide front-line customer support, as well as maintain and improve the API and search infrastructure. I’m also working on several parts of UnPaywall.

It’s incredible that OurResearch tools are freely open and available. I find OurResearch has similar core values as my time in the Air Force: small teams empowered to make decisions, humble and accepting of feedback in order to make things better. That’s why we believe our community of users are invaluable and important in keeping those tools free, open, and easy to use.

So we will listen to your feedback, fix bugs and implement features quickly, and continue to maintain our documentation so the dataset and APIs are as frictionless as they can be. We welcome and need your help with this mission! So do not hesitate to contact me or the team.

I look forward to improving OpenAlex and Unpaywall, and to meeting those of you using OurResearch products!

– Casey

Unsub is a dashboard that helps you reevaluate your big deal’s value and understand your cancellation options.

For the last few years we’ve supported a small set of very large publishers.

One of the most requested features has been support for more publishers.

As of today – right now – we support all publishers.

We heard you, and we’re super excited to get this in your hands. Here’s some important details:

• All publishers are supported. We no longer support specific publishers, but rather we support any publisher.
• A mix of publishers is supported. This was another oft requested feature, mostly related to aggregators, and actually naturally arose out of our change to support all publishers. Unsub dashboards no longer have logic filtering what titles are in your dashboard by publisher – so it’s just as easy for a dashboard to have titles from one publisher or 20 publishers.
• Title prices are now required. Supporting all publishers, it’s not feasible for us to collect and update titles prices for all of their titles. For existing Unsub packages created before today, we’ve incorporated the public prices we had (for the big 5 we supported: Elsevier, Springer Nature, Wiley, Taylor & Francis, SAGE) into your packages. For new packages moving forward, you’ll have to upload your own title prices. We’ve updated the documentation accordingly.
• APC report has moved from package to institution level. We have APC data for the big 5 publishers, but now that we’re moving to any publisher, we can no longer provide publisher specific APC reports. However, you can now get an APC report for your institution that includes an estimate of your APC spend for the big 5 publishers (Elsevier, Springer Nature, Wiley, Taylor & Francis, SAGE). See the APC Report documentation page for more.

But, we didn’t stop there. Here’s some additional features you can use today that we think you’ll enjoy:

• Packages now have Descriptions. When you login to Unsub you’ll see evidence of this change straight away. You can use this package attribute to include a lot of detail about your package to remind your future self and others of important details about your package. See the docs for more information.
• Package views now have an Edit Details tab. In this tab you can change the package name and description. See the docs for more information.
• Packages have an optional filter setup step. This could be used for a variety of use cases, but first and foremost can be used to get back to the state of your package before today’s changes. That is, we no longer filter by publisher. If you had a Wiley package before today you should have only seen titles published by Wiley in your dashboard. However, moving forward, we do not filter by publisher, so that same Wiley package may include some titles from other publishers that were in your COUNTER reports. You can use this new feature to limit the set of titles that appear in your dashboard. See the Upload journal filter documentation page to learn more.

Notes:

• During testing, we heard that aggregators may not provide a COUNTER 5 TR_J2 file. As we require a TR_J2 file if you choose COUNTER 5 in Unsub, we provide a fake TR_J2 file. Let us know if you run into any issues with this! See the docs page for more info.
• As we support more publishers, we’ll run into more edge cases. We’ve heard that some publishers only provide a COUNTER 5 TR_J1 file – and do not provide TR_J2, TR_J3, and TR_J4 files. We don’t currently support the COUNTER 5 TR_J1 file. Get in touch if this is something you need.
• There may be “growing pains” moving from support for 5 publishers to all publishers. For example, journal metadata that’s crucial to Unsub may not be complete for some journals. Please do get in touch if you run into any issues. We’ll be keeping an eye on things and will

If you are not a current Unsub subscriber and you’re interested to learn more schedule a demo or go ahead and purchase.

If you are a current Unsub subscriber, log in, kick the tires, and let us know what you think.

To learn more about all the new features head over to our documentation.

In an upcoming webinar (date to be announced soon) I’ll dive into all the new features and answer any questions.

We’re starting an Unsub (https://unsub.org/) webinar series next week!

Why would you want to attend? These webinars should help you get better value from Unsub regardless of whether you want to just understand your options, get a better deal on your big deal, or cancel your big deal. 

Every two weeks we’ll cover a new topic, with two time slots for each topic to serve a wider array of time zones: morning and afternoon PST (Pacific Standard) time.

If our webinar times don’t work for you, we are planning to record webinars and upload them for anyone to watch on Vimeo (https://vimeo.com/unsub).

Here are the first three topics we’ll cover:

• Feb 8 & 10: Unsub demo – an overview of the product
• Feb 22 & 24: Eric Schares demoing Unsub Extender
• Mar 8 & 10: Deep dive on Unsub scenarios

Other topics are in the works – we’ll announce them soon. Let us know here, elsewhere, or email me (scott@ourresearch.org) if there’s any topics you’d like covered in our webinar series.

The webinar series is free. However, we will require registration so we know how many people are coming and to make it easier for you to remember to attend (i.e., Zoom email confirmation, add to your calendar, etc). 

Our first webinar is titled Unsub Demo – An Overview of the Product – Feb 8 and 10:

• Date: Tuesday, February 8, 2022
• Time: 8:30 – 9:30 am PST (Time converter tool)
• Registration: https://us02web.zoom.us/meeting/register/tZIvduqvrT0iGtwrPdj0PQiTBcFhPEK7SQzx 

• Date: Thursday, February 10, 2022
• Time: 3:30 – 4:30 pm PST (Time converter tool)
• Registration: https://us02web.zoom.us/meeting/register/tZYocOmsqTkqGtyljdyLuTzm9vGCd-lhFn81

We’ll put out registration links soon for subsequent webinar topics.

Why OurResearch?

I’m thrilled to have landed a job with OurResearch working full-time on Unsub. When I was looking for a job this summer I wanted a new experience; I wanted to be challenged and to learn new skills – Unsub was the perfect match. With respect to coding, I moved from 100% R programming to 100% Python. In addition, the domain (tools for librarians) is very different from my previous job (open source software for researchers) – just the big change I wanted. 

Academic Libraries

Despite coming into this job without experience working as a librarian, I’ve always deeply appreciated libraries and the work librarians do. During my time in academia (bachelors through post-doc) I benefited a lot from various university libraries (Rice University and Simon Fraser University, to name a few), and experienced the technological change from print to electronic as ILL requests first came in hardbound and printed form, then transitioned to electronic forms. I’m excited to be able to help librarians after benefiting from their work for so many years.

What I’ll work on

As the Unsub product owner I’ll make decisions about features, implement those features, fix bugs, do demos for librarians, and of course do lots of support. I’m excited to make Unsub the best tool for librarians to reevaluate big deals and understand their cancellation options. 

Challenges and opportunities

The biggest challenge I see in maintaining Unsub is making sure our forecasts are as accurate as possible. I’ve learned already that it can be difficult to keep track of what publishers are doing with respect to big deals, title by title prices, etc. 

There’s a big, neh huge, opportunity here to push scholarly literature much further towards open access – while at the same time freeing up library budgets to support more collaborative players in the scholarly publishing community.

Last month, we announced that we’re launching a replacement for Microsoft Academic Graph (MAG) this December–just before MAG itself will be discontinued.  We’ve heard from a lot of current MAG users since then. All of them have offered their support and encouragement (which we really appreciate), and all have also all been curious to learn more. So: here’s more! It’s a snapshot of what we know right now.  As the project progresses, we’ll have more details to share, keeping everyone as up-to-date as we can.

Name

We’ve now got a name for this project: OpenAlex. We like that it (a) emphasizes Open, and (b) is inspired by the ancient Library of Alexandria — like that fabled institution, OpenAlex will strive to create a comprehensive map of the global scholarly conversation. We’ll start with MAG data, and we’ll expand over time. Along with the name, we’ve got the beginnings of a webpage at openalex.org, and a Twitter account at @OpenAlex_org.

Mailing list

We’ve now got a mailing list where you can sign up for more announcements as they happen. You can sign up for the mailing list on the new OpenAlex homepage.

Funding

Our nonprofit OurResearch recently received a $4.5 million grant from the Arcadia Fund, a charitable fund of Lisbet Rausing and Peter Baldwin. This grant has been in the works for some time, and is a big part of why we felt confident in announcing OpenAlex when we did. In the proposal, there was already a plan for a project similar to OpenAlex, so we were able to quickly pivot the grant details to direct about a million dollars to the development of OpenAlex. 

It’s a three year grant, which will give us plenty of time to develop and launch OpenAlex, as well as test and launch a long-term revenue. This model will not be built on selling data (see Openness below), but rather based on selling value-added services and service level agreements. We’ve got experience with this approach: we’re funding Unpaywall this way, and it’s been both open and fully self-sustaining for several years now.

Openness

We’re passionate about openness. It’s the “Our” in our name–we think research should belong to all of us, as humans.  Openness is the first of core values, and it’s a big piece of our recent public commitment to the Principles of Open Scholarly Infrastructure (POSI). A lot of our excitement about OpenAlex comes from the chance to make this rich dataset unprecedentedly open.  Specifically:

• The code will be open source under an MIT license, hosted on our GitHub account and backed up by Software Heritage.
• The data will be as openly licensed as possible. Some of the data consists of facts, which have no copyright (see this Crossref post for more about that idea). Where copyright is applicable, and where we have the option, we’ll apply the CC0 waiver. Where other rightsholders are involved, we will encourage them to allow a similarly open license.
• The data will be free, as in no cost. It will be available via a free API (more details below) with generous limits, as well as periodic data dumps available at no charge (we may require the downloader to cover the 3rd party data transfer fees if these get heavy).

Data we are losing (at least to start)

As mentioned in our initial announcement, OpenAlex will be missing some data that MAG currently has–particularly at our launch in 2021, due to the very tight timeline. More accurately, we’ll have this data, but won’t be keeping it up to date. Specifically we won’t have: 

• Conference Series and Conference Instances. Importantly, we’ll continue to bring in the vast majority of conference papers. But won’t be keeping track of  new conferences themselves (eg, The 34th Annual Conference of Foo), and with that the ability to link conference papers to those conferences.
• Citation Contexts (the full text of the paragraph where each citation originally appeared)
• Most abstracts. We will however probably have those (minority of) abstracts that publishers send Crossref or PubMed for redistribution.
• Full coverage of DOI-unassigned works:. MAG is particularly good at finding scholarly papers without a DOI. We’ll be less good, especially at first. We will include many DOI-unassigned works…just not as many as MAG.

There is some other data that may or may not make it into OpenAlex by December 2021. We are still testing these for feasibility:

• Patents
• Paper recommendations

Data we are adding

Although we’ll be missing some data, we’ll also be bringing some new data to the party — stuff MAG doesn’t have right now. Specifically will include:

• The Open Access status of papers (via the Unpaywall dataset, which has become the industry standard). We’ll be able to tell you whether a given paper is OA or not, its copyright license, and where to find it. 
• A more comprehensive list of ISSNs associated with each journal, including the ISSN-L, which is helpful for deduplicating journals.
• ORCID for author clusters. To start with, this will just be in unambiguous cases, when assignment is clear via the Crossref and ORCID datasets. Over time we may apply fancier, more inferential assignments.
• ROR IDs for institutions, in addition to GRIDs

Over the long term, our goal with OpenAlex is to create a truly comprehensive map of the global scholarly conversation, so we’ll be continually looking to  expand and enhance the data it includes.

Data dumps

There will be (at least) two ways to get at the data: data dumps, and the API (below). 

Data dumps will be in the same table/column format as the MAG data, so that the downloads can be a drop-in replacement. There may be some additional tables and additional columns for new data we’re adding, and some data values will be missing (both of these are described above), but if you’re running code to ingest MAG dumps right now, you’ll be able to run pretty much the same code to ingest OpenAlex dumps in December. That’s a really important part of this project for us, because we know it will save a lot of folks a lot of time.

We will release new data dumps every 2 weeks, as either a full dump or an incremental update or both (we’re still looking into that). The data will likely be hosted on AWS S3 rather than Microsoft Azure.

API 

The other way main to get at the data will be via the API. Here we will be doing it pretty differently than Microsoft. We will not be supporting the Microsoft Academic Knowledge API or Microsoft Academic Knowledge Exploration Service (MAKES). Instead, we will host a central, open REST API that anyone can query. This API will have two kinds of endpoints: entity endpoints, and slice-and-dice endpoints. Both will be read-only (GET), deliver data in JSON format, and be rate-limited but with high rate-limits. 

• Entity endpoints will let you quickly retrieve a specific scholarly entity (eg paper, person, journal, etc) by its ID. Signatures will look like  /doi/:doi and /journal/:issn.
• Slice-and-dice endpoints will let you query the data with filters to return either item lists, or aggregate group counts. An example call might look something like /query?filter=issn:2167-8359,license:cc-by&groupby=year (that would give you the annual counts of CC-BY-licensed articles from the journal PeerJ). You could also use the slice-and-dice endpoints to do things like build a faceted scholarly search engine, or an evaluation tool.

Timeline

We appreciate that having a scheduled beta (or alpha!) release of the API and data dump would be very helpful. And we further realize that the sooner we can let you know that schedule, the better. Unfortunately, we don’t know the timeline for these releases yet. Our current best guess is the early fall. We’ll certainly be doing our best to get something pushed out there as soon as possible. We encourage you to  join the mailing list so we can keep you up to date. 

Your comments

Finally, we welcome your comments and questions! We’ve gotten oodles of helpful feedback already, and we really appreciate that. We’re especially interested in getting your current use-case for MAG…we’re working to prioritize supporting those cases, first and foremost.  You can do that via our community survey here, or drop us a line at team@ourresearch.org.

OurResearch, a nonprofit seeking to speed the global adoption of Open Science, announced today that it had been awarded a new 3-year, $4.5M (USD) grant from the UK-based Arcadia, a charitable fund of Lisbet Rausing and Peter Baldwin.

The grant, which follows an 2018 award for $850,000, will help expand two existing open-source software projects, as well as support the launch of two new ones:

• Unpaywall, launched in 2017, has become the world’s most-used index of Open Access (OA) scholarly papers. The free Unpaywall extension has 400,000 active users, and its underlying database powers OA-related features in dozens of other tools including Web of Science, Scopus, and the European Open Science Monitor. All Unpaywall data is free and open.
• Unsub is an analytics dashboard that helps academic libraries cancel their large journal subscriptions, freeing up money for OA publishing. Launched in late 2019, Unsub is now used by over 500 major libraries in the US and worldwide, including the national library consortia of Canada, Australia, Greece, Hong Kong, and the UK. 
• JournalsDB will be a free and open database of scholarly journals. This resource will gather a wide range of data on tens of thousands of journals, emphasizing coverage of emerging open venues. 
• OpenAlex will be a free and open bibliographic database, cataloging papers, authors, affiliations, citations, and journals. Inspired by the ancient Library of Alexandria, OpenAlex will strive to create a comprehensive map of the global scholarly conversation.  In a recent blog post, the team announced that OpenAlex will be released in time to serve as a replacement for Microsoft Academic Graph, whose discontinuation was also recently announced.

OurResearch’s ongoing operations costs (about $1M annually) are currently covered by earned revenue from service-level agreements. The new funding will go toward accelerating development of new features and tools.

The new tools and features will be developed in keeping with OurResearch’s longstanding commitment to openness. OurResearch recently became one of the first to commit to the Principles of Open Scholarly Infrastructure (POSI), a set of guidelines encouraging openness, sustainability, and responsive governance. OurResearch has always fully shared its source code and datasets, and maintains a transparency webpage publishing salaries, tax filings, and other information. The proposal for this grant is itself shared on Open Grants.

“We are very grateful to the Arcadia Foundation for this grant, which will help us innovate more quickly than ever before. There is an urgent need for open scholarly infrastructure,” said Heather Piwowar, one of OurResearch’s two cofounders. 

“Since our beginning at a hackathon ten years ago, we’ve been working to build sustainable, open, community-oriented software tools to make research more open,” added her cofounder Jason Priem. “We’re so excited about the ways this grant will help us further that vision.” 

Work on the grant is expected to begin at once, with early versions of both JournalsDB and the OpenAlex launching later this year.

———————————-

OurResearch is a nonprofit that builds tools to help accelerate the transition to universal Open Science. Started at a hackathon in 2011, they remain committed to creating open, sustainable research infrastructure that solves real-world problems.

Arcadia is a charitable fund of Lisbet Rausing and Peter Baldwin. It supports charities and scholarly institutions that preserve cultural heritage and the environment. Arcadia also supports projects that promote open access and all of its awards are granted on the condition that any materials produced are made available for free online. Since 2002, Arcadia has awarded more than $777 million to projects around the world.

We love our name, but in the last few years we’ve found that it’s a bit confusing. In a lot of contexts, it’s not totally clear whether you’re talking about “Our Research” (the enthusiastic Open Science nonprofit) or “our research” (some research that belongs to some people, some of whom are us). That’s bad.

So, we’re changing the name. Or more accurately, we’re changing the spelling, by getting rid of the space. Our Research is now OurResearch! We’ve updated the spelling in all the places we could think of; this includes modifying our logo (hi-res version here):

If you find outdated usages we missed, please let us know. Also, if you’re using the old spelling or logo anywhere, we’d be thankful if you could change it to the new one, when it’s convenient. 

Thanks for reading, and thanks for your support! We’ll let Drake handle the outro: