A general trend in health economic modelling over the past few years has been the introduction of more ‘advanced’ methodologies. A particularly fundamental example has been the suggestion of a move away from Microsoft Excel© and made-for-purpose proprietary health economic modelling software such as TreeAge©, towards models developed with programming languages such as R. This is motivated by the flexibility, efficiency and power of code-based platforms. Another methodology for advancing health economics methods that has recently been in the spotlight is a move towards open-source models, which would allow for more collaborative and repeatable economic assessments to be made for new treatments.
These issues have been explored in some depth over recent years; many health economics and outcomes research (HEOR) professionals (especially in academic circles) believe that adoption of ‘advanced’ methodologies would be positive for HEOR.1-7 However, there is also considerable resistance towards the use of ‘advanced’ methodologies, which generally have not been widely adopted in industry, especially in the context of health technology assessments (HTAs).
A key theme at ISPOR Europe 2022 was where we go next with these ‘advanced’ methodologies. In particular, there was much consideration of the practical issues inherent in adopting these methodologies more widely, for example:
- Are advanced methodologies sufficiently transparent?
- Is there enough expertise across health economics professionals to ensure successful implementation of advanced methodologies?
- How should conflicts between advanced methodologies and commercial interests be resolved?
Adopting R in HTA
The question of how the adoption of R can be facilitated, especially in the context of HTA submissions, was the topic of much general discussion at ISPOR Europe 2022. In particular, two short courses were dedicated to teaching R-based modelling methods, and R-based modelling was explored at a workshop presenting perspectives from academia, industry and HTA.8-10
Currently, the advantages and disadvantages of R as a platform for health economic modelling are well-established; some of the key considerations are given below:
Figure 1: Opportunities and challenges in adopting R in HTA
Despite the limitations of R, there seems to be consensus that, at the very least, it could be a useful alternative to Microsoft Excel© or other proprietary software in some contexts. Without a doubt, R is a much more powerful and flexible programming tool than the currently-available alternatives. There is also potential for creating user-friendly and visually-appealing user interfaces through the use of R Shiny apps, which are accessible to users with no programming skills.
However, to date, there has been little use of R in industry, for a number of reasons.
Lack of relevant skills
A key issue in the introduction of R models in industry settings is that R is associated with a steep learning curve and therefore requires more time and investment training-wise compared to other, proprietary software. Widespread upskilling across the field would therefore be necessary in order to ensure that R models are a viable option. It is also worth noting that, for entrants to the field, requirements for familiarity with R would lead to higher training overheads compared to Microsoft Excel© or some proprietary software.
However, this problem is by no means insurmountable. R is widely used across multiple fields, and therefore, there are numerous freely available online training resources for people of all skill levels, as well as an increasing number of HTA-specific training resources (for example, courses and seminars offered by the R for HTA Consortium). The only remaining requirement would be time and motivation for HEOR professionals to make use of these resources.
Lack of precedent
Another key limitation is the lack of existing examples of models built in R submitted to HTA bodies; this has led to considerable uncertainty, both around the acceptability of models built in R to HTA bodies in general, as well as uncertainty around best practice for model building in R. This lack of precedence, and resulting uncertainty, means that industry has no incentive to pioneer new modelling methods. Submitting economic assessments to HTA bodies requires significant investment of both time and capital; therefore, why use a code-based platform such as R when there is a larger precedent for using a model built in Microsoft Excel©?
Once again, however, this may be changing. A particularly interesting development has been the recent publication of guidelines for the submission of R models to the Zorginstituut Nederland (ZIN), the HTA agency in the Netherlands.11 This is a first in HTA and establishes a precedent which other HTA agencies may follow (it is worth noting that several major HTA bodies, including the National Institute for Health and Care Excellence [NICE] and Canada’s Drug and Health Technology Agency [CADTH], were represented in a working group involved in the development of the ZIN guidelines). The guidelines also send a clear message that HTA submissions with models built in R are welcomed by ZIN and give useful guidance on preferred methodologies which were previously ambiguous (for example, which existing R packages could be incorporated into the code).
So, what are the next steps in introducing R models in industry? There is clearly a circular issue of how to overcome the lack of precedent for this new methodology – until someone submits a model programmed in R to a HTA body, there will be no precedent for submitting R models, and limited motivation for committing to training – but while the lack of precedent persists, there will still be reluctance to break new ground by submitting models built in R. There is no clear solution for how this cycle could be broken, but possibilities include well-publicised ‘pilot’ HTA submissions, possibly including models which are double-programmed in both R and Microsoft Excel©, as well as initiatives to raise awareness and facilitate training. However, while there are still clearly a few roadblocks before widespread adoption of R for health economic modelling, this seems to be moving towards becoming more a question of ‘when’ than ‘if’!
Open Source Models
Another development which comes hand-in-hand with a shift towards coded models is increasing interest in open source models. In general, the term ‘open source’ refers to anything that is freely available, which users may modify, distribute and use as they wish. This concept is specifically associated with software where the source code is made available for all users to view, copy, alter or share (some well-known examples of this include the Linux operating system and the Mozilla Firefox internet browser; R itself is also open source). This generally results in decentralised development, with an emphasis on collaboration.
Open source models in the context of health economics would involve making available, and maintaining, health economic models (including underlying code and written documentation describing the methodology and assumptions used in building the model) for all to use. There is increasing recognition that open source models could have a role to play in health economics; at ISPOR Europe 2022, this was the subject of a workshop and a discussion group and there is also an ongoing open source models special interest group.12,13
A key motivation for the introduction of open source models is that they facilitate a ‘scientific’ approach – the use of open source models may increase transparency, credibility and reproducibility of health economic models, as well as promoting collaboration. The open source approach is also efficient, since this reduces the overheads for building multiple individual models from scratch. The inherent flexibility and openness of this approach are already recognised by many fields adjacent to health economics (for example, data science), and HEOR professionals are increasingly coming to the same conclusions.1 The potential advantages of open source models are explored further in Figure 2.
Figure 2: Potential advantages of open source models
On the other hand, the principal of open source modelling seems at odds with health economics, a field in which models are often custom-built for particular interventions in specific contexts. Open source models would likely be more focused on diseases than individual interventions, which differs to the current HTA process. Furthermore, the open source model poses challenges for both confidentiality interests (for example, if individual patient data inform model inputs) and commercial interests. It is also unclear who should have ownership of open source models, and who would be responsible for maintenance of these models.
These are important questions which have yet to be resolved. There is clearly some uncertainty around whether open source models are the right way forward for health economics (in a recent study of ISPOR members, three quarters of respondents considered legal concerns and issues with transfer of data to be barriers to the use of open source models).1
However, health economics is increasingly being applied in new contexts. In particular, there is increasing interest in moves towards ‘living’ approaches to HTA. ‘Living’ HTA would require flexible, reusable models which can be updated efficiently to facilitate consistent assessment and re-assessment of available therapeutic interventions; open source models would be an ideal fit for these requirements.
Where do we go from here? There is evidently a need for further research into determining appropriate guidance and safeguarding methodology, especially to ascertain whether a solution can be found for the legal and commercial issues surrounding data sharing. There is also a need to establish ‘use-cases’ to determine situations in which the use of open source models is practical or beneficial for health economics.
In conclusion, there is a broad spectrum of new technical approaches in health economics which have emerged in academic spheres but are moving towards acceptance in industry settings. There is now general acceptance that more powerful, but accessible, approaches offer substantial advantages for health economic modelling. However, some approaches appear closer to adoption than others. In particular, the introduction of R as a well-established tool for health economic modelling looks to be on the horizon, whereas a full open source modelling ecosystem is yet to find its footing. Since there is growing acceptance that advanced methodology in health economics is a positive change, everyone involved in health economics and industry can play a role in driving this process:
Figure 3: How can methodological advances and innovation in health economic modelling become reality?
If you would like any further information on the themes presented above, please do not hesitate to contact Sophie Ip, Health Economist or Thomas Kloska, Health Economist (LinkedIn), Deputy Head of Health Economics and Statistics (LinkedIn). Sophie and Thomas are employees at Costello Medical. The views/opinions expressed are their own and do not necessarily reflect those of Costello Medical’s clients/affiliated partners.