London, UK (Scicasts) — How do we define impact? How can one say whether the knowledge uncovered in a research study is important enough to change the course of history?
The ESRC divides impact into Academic and Economic & Societal parts. The Academic impact “is the demonstrable contribution that excellent social and economic research makes to scientific advances, across and within disciplines, including significant advances in understanding, method, theory and application”.
Economic and societal impact, on the other hand, is the demonstrable contribution that excellent social and economic research makes to society and the economy, of benefit to individuals, organizations and nations.
Yet both are often difficult to define and the discoveries that break the news seem to be making an immediate and significant contribution to our lives - be it a cure for cancer or an insight into the workings of human brain.
In addition, the philosophy of individualism sparks and promotes ‘the celebrity approach’ to science research, creating an illusion of one scientist (or a fairly small team) being behind each breakthrough. We celebrate significant leaps forward, encourage groups who do outstanding, ground-breaking research with grants and research facilities, all the while as if forgetting the whole body of preceding research without which no great discovery would have been possible.
Setting off on a research topic, a scientist may only have a vague idea of what they will discover, but they never ‘know’ it from the start. Rarely are there any clear ‘milestone goals’ to achieve or a schedule to follow or a ‘project delivery deadline’.
A researcher has to use their creativity, experience and imagination to decide upon which path to follow, and once on that path, they have to keep going forward, sometimes without even a map – or drawing one as they move along.
As the best hypotheses in science are the most disruptive ones, so the best projects tend to be those carrying the highest risks. Risk-taking and dealing with potential failure meet young researchers at the very beginning of their career, from the time they enter their PhD and throughout the time they progress up the academic ladder.
It’s true, the higher academic ranks serve as safety cushions, somewhat securing the researchers running several projects at a time from a complete career disaster. Yet even at the very top, choosing the wrong direction of research can prove fatal and result in a dissolution of a whole group or a laboratory.
The salaries in academia leave much to be desired, too, and unlike the high investment risks made in the financial world, rarely pay off or serve as important motivation. In which case, why do academics venture into the world of the unknown, virtually without any safety net to protect them, only to what seems like make it or break it in the end?
Some may view scientists as antisocial, unshaved adrenaline junkies, whose whole purpose in life seems to be conquering heights for the sake of it and taking risks with the sole purpose of experiencing that “Eureka!” moment when all your theories all of a sudden begin to make sense.
Another way to look at it, I would argue, is to view scientists as the Servants of Knowledge. The one with a capital K.
The Economy of Knowledge vs The Knowledge-Driven Economy
Ever since the times of Adam Smith, who pioneered the idea of perpetual economic progress, the scientific ventures and capitalism have been walking (or should I rather say, leaping) forward, hand in hand. New knowledge acquired through research allowed the invention of the steam engine, and the new knowledge brought forth by nuclear researchers helped define the course of World War II.
Over the past three centuries, less and less people have been questioning the role of scientific research in moving the world forward and improving the quality of our lives. Yet the knowledge itself have been mostly considered the means to achieve the humanity’s ambitious vision of the future. Knowledge is the fuel of progress, we often hear today, and mining for knowledge is a challenging yet noble task.
Knowledge as fuel, just as oil or gas or coal, is an attractive metaphor, yet it is not a complete one. If all that academics do all day is digging (or mining) for knowledge – what does this make an honourable university Professor? That’s right, a miner. The payroll slips seem to confirm that, too.
That would hold true if we viewed Knowledge as commodity. But whereas coal is rarely exchanged for oil or gas, the Knowledge can successfully be exchanged for more Knowledge, and in return lead to ever more Knowledge generated and so on, the cycle repeating itself. Just as the money properly invested or exchanged back and forth at the right times generates ever more money and drives forward the financial world.
“Admit it, you find me fiscally attractive.” Knowledge exchange is what drives interactions between scientists. The prospect of obtaining more knowledge of better value prompts collaborations between science groups.
This way, the academic world becomes a ‘united kingdom’ within the global empire, with its own economical (and even political) structure. If money is the currency, the ultimate goal and the God of the financial world, so Knowledge is the currency, the ultimate goal and the God of the world of science.
Conversion rates and the investor’s dilemma
The notion of knowledge as the primary goal and the driver of academic research does not mean, of course, that it cannot be converted into money.
The conversion happens ‘at the surface’, in the realm of the so-called applied science, and more rarely, in cases when an important discovery in fundamental science opens immediate opportunities for novel products or medicines. Otherwise, most of translational research itself is fuelled by the whole body of Knowledge generated by researchers in various fields all over the world.
However, in the real world where research funding is often scarce and has to be aligned with the country’s strategic priorities, the question arises over what studies to fund. No matter how tempting it would seem to leave the ‘free market’ of knowledge to take flight on its own, the funding bodies have their obligations to fulfill before the government and taxpayers whose money they invest in research.
Of all grant applications submitted to the NIH, only around 20% on average receive the funding. Similarly, total success rates for grant applications at MRC are within 20-25%. The strategic choices over funding reflect investment into areas where the fiscal value of the knowledge currency is the strongest – in other words, where the currency conversion rates are the most beneficial.
This brings us back to the notion of research impact and definition of the knowledge currency unit – academic publication.
The debate over what should be considered as the impact of a single publication has been raging on for over a decade. The innovative systems such as Altmetric now calculate ‘the online popularity’ of each publication, counting the number of times an article has been mentioned in a discussion on social media. The more traditional approach equates impact with the number of direct citations of an article in other academic publications.
Meanwhile, the members of the most orthodox community (surprisingly, including many young researchers) seem to almost give up the idea of individual publication’s impact and rather focus on the impact factor of the journal where the study has been published.
Or should I, perhaps, say ‘deposited’? For, upon the arrival of banks, rarely anyone keeps cash in their closet anymore.
The wealthy bankers of the science world
For centuries, the traditional way of preserving scientific knowledge was to keep the original manuscript and its copies in libraries. As the mass publishing had become widespread, academic journals gradually took over the role of intermediaries who carefully select, edit and publish research reports in a convenient magazine form.
I would argue that plentiful publishing houses nowadays function as “banks" of the Knowledge economy. Similar to their financial counterparts, the journals accept the researchers’ deposits and lend the Knowledge away through subscriptions to anyone wishing to use it.
Except in the world of publishing ‘knowledge deposits’ generated by scientists become a property of the publishing house and the account owner has no way of retrieving his deposits – unless they retract the paper. This tiny detail makes the system virtually bulletproof, ensuring that there never could be a ‘financial crisis’ in the publishing world. Unless, of course, the ‘green’ open access movement succeeds in its mission and the idea of publishing research results in open repositories prevails over the notion of the journal impact factor.
It has been over two decades since the time when Stevan Harnad first suggested using public repositories to hold scientific results. However, his proposal is yet to become mainstream - partly due to the conservative mindset and partly due to the prestige of the impact factor.
Publishing research in a high impact factor journal is akin to keeping all your money in Deutsche Bank – you can almost wear your debit card as a badge of honour. Yet I kept the whole of my minimum PhD student wage in Deutsche Bank. I was lucky to be part of an important institution that signed a contract agreement with Deutsche for its research employees.
The price for access to the Knowledge bank resources could be as high as several millions for universities and prominent research organizations. For instance, the UCL spent a total £3,052,170 on journal subscriptions in 2015. If all its 6,000-7,000 academic staff held credit cards from Deutsche Bank, paying 80€ card usage fee plus 60€ current account fee per annum (rates in Germany), the institution would have spent £770,000 (approximately four times less).
Similarly, the 30-40% profit margins achieved by some publishing houses are akin to paying a 30% interest rate on your credit card. You will feel it.
Moreover, if Deutsche Bank were to find itself in the midst of a public scandal and all the bank’s wealthy clients decided to withdraw their deposits, the bank would probably collapse. Should the same happen to a publishing house, it is much less likely that all academic users would withdraw their journal subscriptions. Sorry, Deutsche – academic publishers are better protected against potential downfall.
Some recent analysis suggests that moving away from subscription fees to paying publication charges for open access could lower the expenses (MPI white paper). But we are yet to see the full transition happen and the estimates for publication costs at the moment seem to vary across different publishing houses.
The Impact Score
As the bottom line under the two thousand words I have written on the subject, let me try and finally define Impact as I see it.
For many studies, the true impact could accurately be estimated only years (or decades) later. Yet more often than not we would like to know the status of a publication soon after it has been released, which implies that any kind of ‘an immediate’ impact factor would be a predictive value.
Moreover, the impact of a piece of cancer research may be very high within the field, yet would carry close to no value to scientists in the field of energy research. And vice versa, a small technological advance in physics that has little impact within its own field (and is therefore published in an unknown journal) may inspire researchers developing biopolymers – and result in a high impact value for the society overall.
In other words, the Impact of each study is quite ‘personal’.
Touching upon the above mentioned points, the impact could also be short-term, immediately opening up new research opportunities or leading to development of new products or medicines, or long-term, when a discovery takes years to be picked up.
The discovery of penicillin is, perhaps, a good example of the latter – it took over a decade and a world war for the antibiotic to become the mainstream cure against bacterial infections.
Perhaps, we should all try and use the ‘predictive approach’ to the impact of each study? It would save a lot of hassle around publishing, allow the studies to be publicly released within research repositories and take opinion of the whole research community into consideration.
At Scicasts, we decided to name this predictive value ‘Impact Score' – that little scale which appears underneath every news report. It consists of 'Personal significance' for each reader, predictive 'Short-term implications' and 'Long-term implications', as well as the 'Overall Score', which is an average of the above three values.
Every member of our community is invited to contribute their ratings and we would like to see how they compare with the other factors available today. As well as how they will change over time.
What does the Impact of a research study mean to you? Share your views by clicking on the Add Response button below.
This is an opinion article. Therefore, the views expressed by the author do not necessarily reflect the public stance of Scicasts Ltd.