Wale J, Scott AM, Hofmann B, Garner S, Low E, Sansom L. Why Patients should be Involved in Health Technology Assessment. Int.J.Technol.Assess.Health Care. Epub 2017 May 22. PMID: 28528585.

OBJECTIVES:
Some countries make considerable effort to involve patients and patient groups in their health technology assessment (HTA) processes; others are only just considering or are yet to consider patient involvement in HTA.
METHODS:
This commentary offers four arguments why patient involvement should be prioritized by those HTA agencies that do not yet involve patients: (1) from a patients' rights perspective, (2) based on patient and community values, (3) centering on evidentiary contributions, and (4) from a methodological perspective.
RESULTS:
The first argument builds on the Alma-Ata Declaration, which holds that patients have a right and duty to have a say in the planning and delivery of their health care, individually and collectively. Where HTA is used to determine access to technologies and services, we argue that patients have a right to be heard. The second argues that decisions about treatments and services need to be aligned with the core values and morals of the patients whom the health system serves. The third argues that patients have unique knowledge and insights about living with a health condition and their needs for services and treatments regarding that condition, which can add to the knowledge base and value of the HTA process. The fourth argues that involvement of patients can facilitate methodological advancement of HTA, in areas such as early scientific advice and managed entry with evidence development.
CONCLUSIONS:
An HTA process that includes patient perspectives can, therefore, provide added value to patients, policy makers and healthcare professionals alike.

DOI: https://doi.org/10.1017/S0266462317000241.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28528585.

Campbell B, Dobson L, Higgins J, Dillon B, Marlow M, Pomfrett C. A New Health Technology Assessment System for Devices: the First Five Years. Int.J.Technol.Assess.Health Care. Epub 2017 May 15. PMID: 28502278.

OBJECTIVES:
The aim of this study was to review 5 years of activity from a new system devised by the National Institute for Health and Care Excellence (NICE), for assessing medical devices and diagnostics aimed at identifying and speeding adoption of technologies with clinical and cost advantages, compared with current practice in the United Kingdom healthcare system.
METHODS:
All eligible notified technologies were classified using the Food and Drug Administration and Global Medical Device Nomenclature nomenclatures. Decisions about selecting technologies for full assessment to produce NICE recommendations were reviewed, along with the reasons given to companies for not selecting products.
RESULTS:
Between 2009 and 2014, 186 technologies were notified (46 percent therapeutic and 54 percent diagnostic). Thirty-nine were judged ineligible (no regulatory approval), and 147 were considered by an independent committee. Of these, eighty (54 percent) were not selected for full assessment, most commonly because of insufficient evidence (86 percent): there were uncertainties specifically about benefits to the health service (54 percent), to patients (39 percent), and about cost (24 percent). The remaining 67 were selected and assessed for Medical Technology guidance (52 percent) (noninferior and/or lower cost consequences than current practice), for Diagnostics guidance (43 percent) or other NICE recommendations about adoption and use. Classifying technologies by two different systems showed no selection bias for any technology type or disease area.
CONCLUSIONS:
Identifying new or under-used devices and diagnostics with potential benefits and promoting their adoption is important to health services in the United Kingdom and worldwide. This new system offers a means of fostering both uptake and further research. Lack of research data on new products is a major obstacle to evaluation.

DOI: https://doi.org/10.1017/S0266462317000253.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28502278.

Foglia E, Lettieri E, Ferrario L, Porazzi E, Garagiola E, Pagani R, Bonfanti M, Lazzarotti V, Manzini R, Masella C, et al. Technology Assessment in Hospitals: Lessons Learned from an Empirical Experiment. Int.J.Technol.Assess.Health Care. Epub 2017 Jun 05. PMID: 28578752.

OBJECTIVES:
Hospital Based Health Technology Assessment (HBHTA) practices, to inform decision making at the hospital level, emerged as urgent priority for policy makers, hospital managers, and professionals. The present study crystallized the results achieved by the testing of an original framework for HBHTA, developed within Lombardy Region: the IMPlementation of A Quick hospital-based HTA (IMPAQHTA). The study tested: (i) the HBHTA framework efficiency, (ii) feasibility, (iii) the tool utility and completeness, considering dimensions and sub-dimensions.
METHODS:
The IMPAQHTA framework deployed the Regional HTA program, activated in 2008 in Lombardy, at the hospital level. The relevance and feasibility of the framework were tested over a 3-year period through a large-scale empirical experiment, involving seventy-four healthcare professionals organized in different HBHTA teams for assessing thirty-two different technologies within twenty-two different hospitals. Semi-structured interviews and self-reported questionnaires were used to collect data regarding the relevance and feasibility of the IMPAQHTA framework.
RESULTS:
The proposed HBHTA framework proved to be suitable for application at the hospital level, in the Italian context, permitting a quick assessment (11 working days) and providing hospital decision makers with relevant and quantitative information. Performances in terms of feasibility, utility, completeness, and easiness proved to be satisfactory.
CONCLUSIONS:
The IMPAQHTA was considered to be a complete and feasible HBHTA framework, as well as being replicable to different technologies within any hospital settings, thus demonstrating the capability of a hospital to develop a complete HTA, if supported by adequate and well defined tools and quantitative metrics.

DOI: http://dx.doi.org/10.1017/S0266462317000356.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28578752.

Mathes T, Antoine SL, Prengel P, Bühn S, Polus S, Pieper D. Health Technology Assessment of Public Health Interventions: a Synthesis of Methodological Guidance. Int.J.Technol.Assess.Health Care. Epub 2017 Apr 24. PMID: 28434414.

OBJECTIVES:
The evaluation of public health interventions poses some challenges. As a consequence, health technology assessment (HTA) methods for public health interventions (PHI) have to be adapted. This study aimed to summarize the available guidance on methods for HTA of PHI.
METHODS:
We systematically searched for methodological guidance on HTA of PHIs. Our focus was on research synthesis methods to evaluate effectiveness. Relevant information was synthesized narratively in a standardized way.
RESULTS:
Only four guidance documents were identified specifically for HTAs of PHI. The approaches used for HTAs of PHIs are broader and more flexible than those for medical interventions. For this reason, there is a tendency to identify the intervention components and context factors that influence the effectiveness and transferability of an intervention rather than to assess its effectiveness in general. The details in the guidance vary without justification. Unjustified heterogeneity between the different guidance approaches is most pronounced for quality assessment, assessment of applicability, and methods to integrate qualitative and quantitative evidence. Descriptions for the assessment of integrity, heterogeneity, sustainability, context factors, and applicability are often vague.
CONCLUSIONS:
The heterogeneity in approaches indicates that there is currently no consensus on methods to deal with the challenges of the PHI evaluations. A possible explanation for this may be that the methods are not sufficiently developed, and advantages and disadvantages of a certain method in relation to the research question (e.g., broad/focused) have not yet been sufficiently evaluated.

DOI: https://doi.org/10.1017/S0266462317000228.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28434414.

Convergence, Divergence and Hybridity: A Regulatory Governance Perspective on Health Technology Assessment in England and Germany. Global Policy. 2017 Mar;8(S2):69-75

Countries adopt different methods and processes to evaluate the benefits and costs of health technologies. It is important to identify and analyse the factors that influence the uptake and use of these methods and processes across countries. In this article, we introduce a regulatory governance approach to the analysis of convergence, divergence and hybridity in Health Technology Assessment (HTA) methods, discussing and critically analysing national processes for HTA in two major EU member states: England and Germany. We argue that any reasonably sophisticated account of national approaches to HTA must recognise that globalisation and the emergence of advanced industrial society involves the potential for widely varying processes, methods and evidential requirements. We suggest that this potential also confronts health policy analysts with the challenge of constructing analytical frameworks capable of identifying the diverse institutional, domestic and other factors that shape national approaches to HTA.

Special Issue: Health Care Policy, Resource Allocation and Financial Sustainability

DOI: http://dx.doi.org/10.1111/1758-5899.12215.

Benoit C, Gorry P. Health Technology Assessment: the Scientific Career of a Policy Concept. Int.J.Technol.Assess.Health Care. 2017 Jan;33(1):128-34. PMID: 28566099.

OBJECTIVES:
The aim of this work was to provide a comprehensive overview of the evolution of the health technology assessment (HTA) concept in the scientific literature through a scientometric approach.
METHODS:
A literature search was conducted, by selecting publications, as well as news from the media, containing "health technologyassessment" in their title, abstracts, or keywords. We then undertook a bibliometric and network analysis on the corpus of 2,865 publications thus obtained.
RESULTS:
Since a first publication in 1978, interest in HTA remained marginal until a turning point in the late 1980s, when growth of the number of publications took off alongside the creation of the U.K.'s NICE agency. Since then, publications have spread across several journals. The ranking of the organizations that publish such articles does not reflect any hegemonic position. However, HTA-related scientificproduction is strongly concentrated in Commonwealth and Nordic countries. Despite its transnational aspects, research on HTA has been framed within a small number of scientific networks and by a few opinion leaders.
CONCLUSIONS:
The "career" of the HTA concept may be seen as a scientific-knowledge based institutionalization of a public policy. To succeed in a country, HTA first needs scientific prerequisites, such as an organized scientific community working on the health sector and health services. Then, it appears that the recognition of this research by decision makers plays a key role in the development of the field.

DOI: http://dx.doi.org/10.1017/S0266462317000186.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28566099.

Carroll C, Kaltenthaler E, Hill-McManus D, Scope A, Holmes M, Rice S, Rose M, Tappenden P, Woolacott N. The Type and Impact of Evidence Review Group Exploratory Analyses in the NICE Single Technology Appraisal Process. Value Health. 2017 Jun;20(6):785-91. PMID: 28577696.

BACKGROUND:
As part of the UK National Institute for Health and Care Excellence (NICE) single technology appraisal process, independent evidence review groups (ERGs) critically appraise a company's submission relating to a specific technology and indication.
OBJECTIVES:
To explore the type of additional exploratory analyses conducted by ERGs and their impact on the recommendations made by NICE.
METHODS:
The 100 most recently completed single technology appraisals with published guidance were selected for inclusion. A content analysis of relevant documents was undertaken to identify and extract relevant data, and narrative synthesis was used to rationalize and present these data.
RESULTS:
The types of exploratory analysis conducted in relation to companies' models were fixing errors, addressing violations, addressing matters of judgment, and the provision of a new, ERG-preferred base case. Ninety-three of the 100 ERG reports contained at least one of these analyses. The most frequently reported type of analysis in these 93 ERG reports related to the category "Matters of judgment," which was reported in 83 reports (89%). At least one of the exploratory analyses conducted and reported by an ERG is mentioned in 97% of NICEappraisal consultation documents and 94% of NICE final appraisal determinations, and had a clear influence on recommendations in 72% of appraisal consultation documents and 47% of final appraisal determinations.
CONCLUSIONS:
These results suggest that the additional analyses undertaken by ERGs in the appraisal of company submissions are highly influential in the policy-making and decision-making process.
Copyright © 2017 International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. All rights reserved.

DOI: http://dx.doi.org/10.1016/j.jval.2016.08.729.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28577696.

Cookson R, Mirelman AJ. Equity in HTA: what doesn't get measured, gets marginalised. Isr.J.Health.Policy.Res. 2017 Jul 10;6:38. PMID: 28694961.

When making recommendations about the public funding of new health technologies, policy makers typically pay close attention to quantitative evidence about the comparative effectiveness, cost effectiveness and budget impact of those technologies - what we might call "efficiency" criteria. Less attention is paid, however, to quantitative evidence about who gains and who loses from these public expenditure decisions, and whether those who gain are better or worse off than the rest of the population in terms of their health - what we might call "equity" criteria. Two studies recently published in this journal by Shmueli and colleagues suggest that this efficiency-oriented imbalance in the use of quantitative evidence may have unfortunate consequences - as the old adage goes: "what gets measured, gets done". The first study, by Shmueli, Golan, Paolucci and Mentzakis, found that health policy makers in Israel think equity considerations are just as important as efficiency considerations - at least when it comes to making hypothetical technology funding decisions in a survey. By contrast, the second study - by Shmueli alone - found that efficiency rules the roost when it comes to making real decisions about health technology funding in Israel. Both studies have limitations and potential biases, and more research is needed using qualitative methods and more nuanced survey designs to determine precisely which kinds of equity consideration decision makers think are most important and why these considerations do not appear to be given much weight in decision making. However, the basic overall finding from the two studies seems plausible and important. It suggests that health technology funding bodies need to pay closer attention to equity considerations, and to start making equity a quantitative endpoint of health technology assessment using the methods of equity-informative economic evaluation that are now available.

FREE FULL TEXT: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502411/pdf/13584_2017_Article_162.pdf
DOI: http://dx.doi.org/10.1186/s13584-017-0162-3.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28694961.
PubMed Central: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5502411/.

Hall PS. Real-world data for efficient health technology assessment. Eur.J.Cancer. 2017 Jul;79:235-7. PMID: 28522211.
DOI: http://dx.doi.org/0.1016/j.ejca.2017.04.003.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28522211.

IJzerman MJ, Koffijberg H, Fenwick E, Krahn M. Emerging Use of Early Health Technology Assessment in Medical Product Development: A Scoping Review of the Literature. Pharmacoeconomics. 2017 Jul;35(7):727-40. PMID: 28432642.

Early health technology assessment is increasingly being used to support health economic evidence development during early stages of clinical research. Such early models can be used to inform research and development about the design and management of new medical technologies to mitigate the risks, perceived by industry and the public sector, associated with market access and reimbursement. Over the past 25 years it has been suggested that health economic evaluation in the early stages may benefit the development and diffusion of medical products. Early health technology assessment has been suggested in the context of iterative economic evaluation alongside phase I and II clinical research to inform clinical trial design, market access, and pricing. In addition, performing early health technology assessment was also proposed at an even earlier stage for managing technology portfolios. This scoping review suggests a generally accepted definition of early health technology assessment to be "all methods used to inform industry and other stakeholders about the potential value of new medical products in development, including methods to quantify and manage uncertainty". The present review also aimed to identify recent published empirical studies employing an early-stage assessment of a medical product. With most included studies carried out to support a market launch, the dominant methodology was early health economic modeling. Further methodological development is required, in particular, by combining systems engineering and health economics to manage uncertainty in medical product portfolios.

FREE FULL TEXT: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5488152/pdf/40273_2017_Article_509.pdf
DOI: http://dx.doi.org/10.1007/s40273-017-0509-1.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28432642.
PubMed Central: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5488152/.

MacGillivray BH. Characterising bias in regulatory risk and decision analysis: An analysis of heuristics applied in health technology appraisal, chemicals regulation, and climate change governance. Environ.Int. 2017 May 9;105:20-33. PMID: 28499120.

In many environmental and public health domains, heuristic methods of risk and decision analysis must be relied upon, either because problem structures are ambiguous, reliable data is lacking, or decisions are urgent. This introduces an additional source of uncertainty beyond model and measurement error - uncertainty stemming from relying on inexact inference rules. Here we identify and analyse heuristics used to prioritise risk objects, to discriminate between signal and noise, to weight evidence, to construct models, to extrapolate beyond datasets, and to make policy. Some of these heuristics are based on causal generalisations, yet can misfire when these relationships are presumed rather than tested (e.g. surrogates in clinical trials). Others are conventions designed to confer stability to decision analysis, yet which may introduce serious error when applied ritualistically (e.g. significance testing). Some heuristics can be traced back to formal justifications, but only subject to strong assumptions that are often violated in practical applications. Heuristic decision rules (e.g. feasibility rules) in principle act as surrogates for utility maximisation or distributional concerns, yet in practice may neglect costs and benefits, be based on arbitrary thresholds, and be prone to gaming. We the problem of rule-entrenchment, where analytical choices that are in principle contestable are arbitrarily fixed in practice, masking uncertainty and potentially introducing bias. Strategies for making risk and decision analysis more rigorous include: formalising the assumptions and scope conditions under which heuristics should be applied; testing rather than presuming their underlying empirical or theoretical justifications; using sensitivity analysis, simulations, multiple bias analysis, and deductive systems of inference (e.g. directed acyclic graphs) to characterise rule uncertainty and refine heuristics; adopting "recovery schemes" to correct for known biases; and basing decision rules on clearly articulated values and evidence, rather than convention.

FREE FULL TEXT: http://www.sciencedirect.com/science/article/pii/S0160412016303877?via%3Dihub
DOI: http://dx.doi.org/10.1016/j.envint.2017.05.002.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28499120.

Maignen F, Osipenko L, Pinilla-Dominguez P, Crowe E. Regulatory watch: Outcomes of early health technology assessment dialogues in medicinal product development. Nat.Rev.Drug Discov. 2017 Feb 2;16(2):79. PMID: 28148947.

[First paragraph]

The development of medicinal products has been shifting towards a value-driven model. However, although companies need to engage with health technology assessment (HTA) bodies to inform effective strategies to gain market access for their new medicinal products, HTA scientific advice (SA) procedures are not yet routine steps in the clinical...

DOI: http://dx.doi.org/0.1038/nrd.2016.286.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28148947.

Payne K, Eden M, Davison N, Bakker E. Toward health technology assessment of whole-genome sequencing diagnostic tests: challenges and solutions. Personalized Medicine. 2017 May;14(3):235-47

Whole-genome sequencing (WGS) is being applied within research settings across Europe to develop genomic WGS-based diagnostic tests. The focus of this perspective paper is to describe if, and how, current approaches of health technology assessment could be applied to WGS-based diagnostic tests. This perspective draws on the collective view from a trans-European multidisciplinary consortium of methodologists, clinicians and scientists. Specific challenges can be described by using the PICO (population, intervention, comparator, outcome) framework to inform health technology assessment. Practical solutions are suggested which require joined-up, multidisciplinary working across healthcare systems using existing expert networks so that emergent issues for the health technology assessment of WGS can be met in a timely fashion.

DOI: https://dx.doi.org/10.2217/pme-2016-0089.

Rehfuess EA, Gerhardus A, INTEGRATE-HTA. INTEGRATE-HTA: adopting and implementing an integrated perspective on complex interventions. J.Public.Health. 2017 Mar 1;39(1):209-12. PMID: 27881575.

Most public health interventions can be considered complex. According to the [first paragraph ; html links removed] UK Medical Research Council (MRC), complex interventions are characterized by several interacting components, the number and difficulty of behaviours required by those delivering or receiving the intervention, multiple groups or organizational levels targeted, many and variable outcomes, and explicitly permitted flexibility or tailoring of the intervention.1

DOI: http://dx.doi.org/10.1093/pubmed/fdw119.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=27881575.

Scott AM, Hofmann B, Gutiérrez-Ibarluzea I, Bakke Lysdahl K, Sandman L, Bombard Y. Q-SEA - a tool for quality assessment of ethics analyses conducted as part of health technology assessments. GMS Health.Technol.Assess. 2017 Mar 15;13:Doc 02. PMID: 28326147.

Introduction:
Assessment of ethics issues is an important part of health technology assessments (HTA). However, in terms of existence of quality assessment tools, ethics for HTA is methodologically underdeveloped in comparison to other areas of HTA, such as clinical or cost effectiveness.
Objective:
To methodologically advance ethics for HTA by: (1) proposing and elaborating Q-SEA, the first instrument for quality assessment of ethics analyses, and (2) applying Q-SEA to a sample systematic review of ethics for HTA, in order to illustrate and facilitate its use.
Methods:
To develop a list of items for the Q-SEA instrument, we systematically reviewed the literature on methodology in ethics for HTA, reviewed HTA organizations' websites, and solicited views from 32 experts in the field of ethics for HTA at two 2-day workshops. We subsequently refined Q-SEA through its application to an ethics analysis conducted for HTA.
Results:
Q-SEA instrument consists of two domains - the process domain and the output domain. The process domain consists of 5 elements: research question, literature search, inclusion/exclusion criteria, perspective, and ethics framework. The output domain consists of 5 elements: completeness, bias, implications, conceptual clarification, and conflicting values.
Conclusion:
Q-SEA is the first instrument for quality assessment of ethicsanalyses in HTA. Further refinements to the instrument to enhance its usability continue.

FREE FULL TEXT: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5352988/pdf/HTA-13-02.pdf
DOI: http://dx.doi.org/10.3205/hta000128.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28326147.
PubMed Central: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5352988.

Soril LJ, MacKean G, Noseworthy TW, Leggett LE, Clement FM. Achieving optimal technology use: A proposed model for health technology reassessment. SAGE Open Med. 2017 Apr 19;5:2050312117704861. PMID: 28491310.

BACKGROUND:
Healthcare providers, managers and policy-makers in many jurisdictions are focused on a common goal: optimizing value and quality of care provided to their citizens within a resource envelope. Health technology reassessment is a structured, evidence-based assessment of the clinical, social, ethical and economic effects of a technology currently used in the healthcare system to inform optimal use of that technology in comparison with its alternatives. There are, however, few practical experiences with health technology reassessmentand, as such, a nascent theoretical and methodological base. Health technology reassessment is a key strategy to achieve optimalhealthcare resource utilization, and establishing a model for health technology reassessment is a required methodological step.
METHODS AND RESULTS:
The purpose of this article is to answer three formative questions: (1) What is health technology reassessment? (2) When should a health technology reassessment be implemented? (3) What is the role of health technology reassessment in evidence-informed health policy? Finally, we propose a conceptual framework for health technology reassessment, which others can modify, adapt, or adopt in their own context. The model consists of three broad phases and six iterative stages: (1) identification, (2) prioritization, (3) evidence synthesis, (4) determine policy/practice recommendation, (5) policy/practice implementation and (6) monitoring and evaluation. Two foundational components (meaningful stakeholder engagement and ongoing knowledge exchange and utilization) are represented across all stages.
CONCLUSION:
This description of health technology reassessment and the proposed model can be used by healthcare policy-makers and researchers to advance the field of technology management, with the goal of achieving optimal use throughout a technology's lifecycle.

FREE FULL TEXT: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5406119/pdf/10.1177_2050312117704861.pdf
DOI: http://dx.doi.org/10.1177/2050312117704861.
PubMed: https://www.ncbi.nlm.nih.gov/pubmed/?term=28491310.
PubMed Central: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5406119.

Booth A, Noyes J, Flemming K, Gerhardus A, Wahlster P, Jan van der Wilt G, Mozygemba K, Refolo P, Sacchini D, Tummers M, et al. Guidance on choosing qualitative evidence synthesis methods for use in health technology assessments of complex interventions. . [s.l.]: INTEGRATE-HTA. February 1 2016.

EXECUTIVE SUMMARY

Challenges in assessments of health technologies
In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies which

are complex, i.e. consist of several interacting components, target different groups or organizational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring (Craig et al., 2008),

are context-dependent - current HTA usually focusses on the technology, not on the system within which it is used,

perform differently depending on the way they are implemented,

have different effects on different individuals.

Furthermore, HTA usually assesses and appraises aspects side-by-side, while decision-making needs an integrated perspective on the value of a technology. In the EU-funded INTEGRATE-HTA project, we developed concepts and methods to deal with these challenges, which are described in six guidance documents. An integrated assessment requires that a variety of effectiveness, cost effectiveness, socio-cultural and ethical questions are simultaneously addressed. Many questions will require reference to qualitative research data. Qualitative evidence syntheses (qualitative systematic reviews) offer one possible way in which findings from qualitative research might be systematically integrated within an HTA. They attempt to identify transferable findings from a body of evidence with a view to addressing a specific contextual problem. Multiple methods of qualitative evidence synthesis currently exist. Even though increasing numbers of available published examples are facilitating the consolidation of lessons learnt very little guidance exists on how to select an appropriate method of synthesis.

Purpose and scope of the guidance
The aim of the INTEGRATE-HTA project is to provide concepts and methods that enable a patient-centred, comprehensive, and integrated assessment of complex health technologies. This guidance on choosing appropriate methods of qualitative evidence synthesis (QES) is to be used when a review team has genuine uncertainty about which type of QES to undertake to meet the needs of a particular question or purpose. It may also be used when a review team seeks to make an informed judgement between two or more competing methods or methodologies. This guidance is not intended to be used prescriptively; additional considerations may inform the final selection of an appropriate synthesis method. It simply seeks to help a review team to navigate an otherwise bewildering array of methodological choices. Pointers to detailed specification of the characteristics of each methodology, together with published examples, are provided for further clarification and exemplification. Development of the guidance This guidance represents further development of principles first explored at a variety of HTA and systematic review conferences, workshops and forums. For the first time we have systematically explored the published literature in relation to the choice or selection of qualitative synthesis methodologies. We analysed 26 separate items published between 2001 and 2014 and identified those elements thought to be important when selecting a synthesis method. We used an embryonic five item framework to organise these considerations. Subsequently we expanded the framework to seven items covered by the RETREAT mnemonic (Review question – Epistemology – Time/Timescale – Resources – Expertise – Audience and purpose – Type of Data). We then compiled a list of specific considerations when selecting a synthesis method according to the published literature and the expertise of members of the Cochrane Qualitative and Implementation Methods Group. The final guidance, revised after internal and external peer-review includes approaches suitable for stand-alone methodologies and those used when integrating quantitative and qualitative data.

Application of this guidance
The INTEGRATE-HTA process seeks to facilitate production of a comprehensive integrated assessment of a complex technology to inform a complex decision problem. Within this context it is important to be able to select appropriate review methodologies – selection of a quantitative review methodology is comparatively straightforward with a limited number of available alternatives, usually determined by the nature of the data and its heterogeneity. In contrast selection of a method for qualitative synthesis is more complex, not least when it is required to integrate with the quantitative elements of a larger review.

Step 1: To use this guidance you first take stock of what is already known about the topic; the nature of the Review question, the characteristics of the evidence base, the quantity and quality of included study designs, the Type of Data required to satisfactorily address the overall review question and all its associated sub-questions.

Step 2: Next you consider the available Resources for the review; the Time, and the requisite Expertise.

Step 3: Then you examine the intended Audience and Purpose and any implications that these might have for the Epistemology behind the review. Having broadly characterised these factors you will have limited the number of available options in terms of synthesis methodology.

Step 4: Finally you turn your consideration to more specific features of the methodology to inform the final selection of the method.

Conclusions
In current HTA, different aspects are usually assessed and presented independent of each other. Context, implementation issues and patient characteristics are rarely considered. The INTEGRATE-HTA Model enables a coordinated assessment of all these aspects and addresses their interdependencies. The perspective of stakeholders such as patients and professionals with their values and preferences is integrated in the INTEGRATE-HTA Model to obtain HTA results that are meaningful for all relevant stakeholders. Finally, health policy makers obtain an integrated perspective of the assessment results to achieve fair and legitimate conclusions at the end of the HTA process. This guidance document is directed at specific challenges relating to the assessment of complex technologies, such as the need to integrate qualitative research studies within a qualitative evidence synthesis and then with the results from a review of the effectiveness and cost-effectiveness studies. It offers a way to navigate challenges posed by a variety of methodological choices that determine the nature of the final HTA in incorporating data to inform effectiveness, economic, ethical, socio-cultural, and legal aspects of HTA. The application of the model will usually require more time and resources than traditional HTA. An initial assessment of the degree and the character of complexity of a technology might be helpful to decide whether or not the whole process or only specific elements will be applied.

Integrated Health Technology Assessment for Evaluating Complex Technologies (INTEGRATE-HTA). This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)

FREE FULL TEXT: http://www.integrate-hta.eu/wp-content/uploads/2016/02/Guidance-on-choosing-qualitative-evidence-synthesis-methods-for-use-in-HTA-of-complex-interventions.pdf.

Gerhardus A and INTEGRATE-HTA Project Team. Integrated health technology assessment for evaluating complex technologies (INTEGRATE-HTA): An introduction to the guidances. . [s.l.]: INTEGRATE-HTA. February 1 2016.

[From the introduction]

INTEGRATE-HTA was a research project involving interdisciplinary scientists from seven European countries. It ran from 1.1.2013 to 31.12.2015 and has been co-funded by the European Commission within the FP7 Cooperation Work Programme. INTEGRATE-HTA has developed six guidances to support an integrated assessment of complex health technologies. The guidances have been tested in a case study on palliative care. The main audiences for the guidances are producers and users of health technology assessments.

Integrated Health Technology Assessment for Evaluating Complex Technologies (INTEGRATE-HTA). This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)

FREE FULL TEXT: http://www.integrate-hta.eu/wp-content/uploads/2016/02/INTEGRATE-HTA-An-introduction-to-the-guidances.pdf.

Pfadenhauer L, Rohwer A, Burns J, Booth A, Bakke Lysdahl K, Hofmann B, Gerhardus A, Mozygemba K, Tummers M, Wahlster P, et al. Guidance for the Assessment of Context and Implementation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions: The Context and Implementation of Complex Interventions (CICI) Framework. . [s.l.]: INTEGRATE-HTA. February 1 2016.

EXECUTIVE SUMMARY

Challenges in assessments of health technologies In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies which

are complex, i.e. consist of several interacting components, target different groups or organizational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring,

are context-dependent - current HTA usually focusses on the technology, not on the system within which it is used,

perform differently depending on the way they are implemented,

have different effects on different individuals.

Furthermore, HTA usually assesses and appraises aspects side-by-side, while decision-making needs an integrated perspective on the value of a technology. In the EU-funded INTEGRATE-HTA project, we developed concepts and methods to deal with these challenges, which are described in six guidances.

Where context-dependency and implementation of a technology matter, it is critical to make this explicit, to document available information as fully as possible and to assess if and how this may affect intervention reach and effectiveness.

Purpose and scope of the guidance
The purpose of this guidance is to provide a framework to commissioners, producers and users of systematic reviews and health technology assessments (HTA) that allows for the systematic conceptualisation, assessment and documentation of the setting, context and implementation of a complex intervention. It presents an overarching framework comprising eight domains of context (i.e. setting, geographical, epidemiological, socio-cultural, socio-economic, ethical, legal and political issues) and four domains of implementation (i.e. provider, organisation and structure, funding and policy), including definitions and descriptions of each of these domains.

Development of the guidance
The Context and Implementation of Complex Interventions (CICI) framework was developed in an iterative fashion. Based on a scoping review of existing theories, models and frameworks concerned with “context”, “setting” and “implementation”, an initial framework for conceptualising, assessing and documenting the interacting dimensions of context (including setting) and implementation was developed. Due to the lack of conceptual clarity identified in the scoping review, and in order to provide a sound scientific basis for this framework,systematic literature searches were performed for the concepts “setting”, “context” and “implementation”. The three concepts were appraised as partially mature. Pragmatic utility concept analysis, as developed by Morse and colleagues, was selected as the most suitable method to evaluate concept use by comparing and contrasting applications across the health field. During data extraction, relevant constructs informing the selected theories, models and frameworks were also extracted and integrated with the initial CICI framework. This revised initial framework was applied in four systematic reviews (three quantitative and one qualitative review), as well as in one HTA. The findings from the applications were used to create the revised CICI framework presented in this guidance.

Application of the guidance
The CICI framework can be applied in HTA and systematic reviews of effectiveness, as well as in qualitative systematic reviews. The guidance provides definitions and descriptions of domains of context and implementation and provides examples which may be of relevance for each domain. In addition, it proposes a list of questions to assess each domain: a) to retrieve quantitative information about the domain (which characteristics influence …?) and b) to generate a more in-depth understanding of the domain’s influence (how do the characteristics influence …?). Moreover, the list encourages the researcher to assess relevant interactions between domains (e.g. ethical and socio-cultural domain). Additionally, we suggest a graphical representation of the domains contained in the CICI framework that supports researchers in systematically assessing domains of context and implementation.

The CICI framework can moreover be used to assess the applicability of a technology to a specific context. The domains serve as the basis for a semi-structured questionnaire that can be used with experts when exploring potential contextual barriers and facilitators to the implementation of a specific technology.

Conclusion
The CICI framework can be applied in quantitative, qualitative or mixed-method systematic reviews and in HTAs. In quantitative systematic reviews, it supports the documentation of relevant context and implementation aspects through data extraction, examination of heterogeneity and subgroup analysis. In qualitative or mixed-method systematic reviews, it serves as a starting point for identifying and formulating specific research questions to understand how context and/or implementation influence the intervention. In HTAs, an appraisal of the respective domains can be conducted by asking stakeholder panels about their importance and relevance. Overall, the framework helps to present context and implementation issues in an integrated fashion, and supports the assessments of applicability and generalizability of HTAs and systematic reviews.

Integrated Health Technology Assessment for Evaluating Complex Technologies (INTEGRATE-HTA). This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)

FREE FULL TEXT: http://www.integrate-hta.eu/wp-content/uploads/2016/02/Guidance-for-the-Assessment-of-Context-and-Implementation-in-HTA-and-Systematic-Reviews-of-Complex-Interventions-The-Co.pdf.

Wahlster P, Brereton L, Burns J, Hofmann B, Mozygemba K, Oortwijn W, Pfadenhauer L, Polus S, Rehfuess E, Schilling I, et al. Guidance on the integrated assessment of complex health technologies - The INTEGRATE-HTA Model. February 1 2016.

EXECUTIVE SUMMARY

Challenges in assessments of health technologies In recent years there have been major advances in the development of health technology assessment (HTA). However, HTA still has certain limitations when assessing technologies which

are complex, i.e. consist of several interacting components, target different groups or organizational levels, have multiple and variable outcomes, and/or permit a certain degree of flexibility or tailoring (Craig et al., 2008),

are context-dependent - current HTA usually focusses on the technology, not on the system within which it is used, fi perform differently depending on the way they are implemented,

have different effects on different individuals.

Furthermore, HTA usually assesses and appraises aspects side-by-side, while decision-making needs an integrated perspective on the value of a technology. In the EU-funded INTEGRATE-HTA project, we developed concepts and methods to deal with these challenges, which are described in six guidances. Because of the interactions, an integrated assessment needs to start from the beginning of the assessment. This guidance provides a systematic five-step-process for an integrated assessment of complex technologies (the INTEGRATE-HTA Model).

Purpose and scope of the guidance
The aim of the INTEGRATE-HTA project is to provide concepts and methods that enable a patient-centred, comprehensive, and integrated assessment of complex health technologies. The purpose of this guidance is to structure the overall HTA-process. The INTEGRATE-HTA Model outlines an integrated scoping process, a coordinated application of assessment methods for different aspects and an integrated and structured decision-making process. It is intended for HTA agencies, HTA researchers and those engaged in the evaluation of complex health technologies. As it links the assessment to the decision-making process, it also addresses HTA commissioners and other stakeholders using or planning HTAs.

While all technologies are arguably complex, some are more complex than others. Applying this guidance might lead to a more thorough and therefore more time-consuming process. Depending on the degree of complexity, one might choose to follow the whole process as described in this guidance, or only focus on certain steps. The guidance provides an operational definition to assess the complexity of technologies which can be used to identify specific aspects that will need more attention than others. What the guidance does not provide is a post-hoc solution for assessments that have already been completed.

Development of the guidance
The INTEGRATE-HTA Model presented in this guidance was developed based on a systematic literature search on approaches for integration, on the experiences of traditional HTAs, as well as on the other methodological guidances developed in the INTEGRATE-HTA project. It was tested in a case study on palliative care and iteratively revised during the practical application. The guidance was again revised after internal and external peer-review.

Application of this guidance
For a comprehensive integrated assessment of a complex technology, we developed a five-step process, the INTEGRATE-HTA model. In Step 1, the HTA objective and the technology are defined with the support from a panel of stakeholders. An initial logic model is developed in Step 2. The initial logic model provides a structured overview of the technology, the context, implementation issues, and relevant patient groups. It then frames the assessment of the effectiveness, as well as economic, ethical, legal, and socio-cultural aspects in Step 3. In Step 4, a graphical overview of the assessment results, structured by the logic model, is provided. Step 5 is a structured decision-making process informed by the HTA (and is thus not formally part of the HTA, but follows it).

Step 1:
In step 1, the technology under assessment and the objective of the HTA are defined. Especially for complex technologies, such as palliative care, the definition of the technology alone is a challenge that must not be underestimated. It is recommended to do this based on a tentative literature review and with the support of stakeholder advisory panels (SAPs) which should comprise clinical experts, academics, patients, possibly their relatives and/or other caretakers, and the public. The setting of an objective considering all relevant aspects of complexity and structured by assessment criteria is important. The assessment criteria will usually reflect values of the stakeholders as well as the input from the theoretical, methodological and empirical literature.

Step 2:
In step 2, an initial logic model is developed (see Guidance on the use of logic models in health technology assessments of complex interventions). The model provides a structured overview on participants, interventions, comparators, and outcomes. Parallel to this, groups of patients that are distinguished by different preferences and treatment moderators (see Guidance for the assessment of treatment moderation and patients’ preferences) are identified. Specific context and implementation issues are also identified as part of the initial logic model (see Guidance for the Assessment of Context and Implementation in Health Technology Assessments (HTA) and Systematic Reviews of Complex Interventions). The product of this step is the logic model as a graphical representation of all aspects and their interactions that are relevant for the assessment of the complex technology.

Step 3:
In step 3, the logic model serves as a conceptual framework that guides the evidence assessment. Depending on the specific aspect (e.g. effectiveness, economic, ethical, socio-cultural, or legal aspects) different methods are available for the assessment (see Guidance for assessing effectiveness, economic aspects, ethical aspects, socio-cultural aspects and legal aspects in complex technologies). The outputs of step 3 are evidence reports and standardized evidence summaries for each assessment aspect (e.g. report on economics, report on ethical aspects, etc.).

Step 4:
In step 4, the assessment results of step 3 are structured using the logic model developed in step 2. Whereas the initial logic model in step 2 specifies what evidence is relevant, the extended logic model to assist decision-making in step 4 visualizes the assessment results as well as the interaction with respect to the HTA objectives. It also allows for the consideration of different scenarios depending on the variation in context, implementation and patient characteristics.

Step 5: Step 5 involves a structured decision-making process and is not an integral part of the HTA in the narrow sense. Decision-making can be supported by applying quantitative e.g. MCDA- (Multi-criteria decision analysis) or qualitative decision support tools. Flexibility in the application of these tools by the decision committee is crucial, taking different decision settings and evidence needs into consideration.

Conclusions
In current HTA, different aspects are usually assessed and presented independent of each other. Context, implementation issues and patient characteristics are rarely considered. The INTEGRATE-HTA Model enables a coordinated assessment of all these aspects and addresses their interdependencies. The perspective of stakeholders such as patients and professionals with their values and preferences is integrated in the INTEGRATE-HTA Model to obtain HTA results that are meaningful for all relevant stakeholders. Finally, health policy makers obtain an integrated perspective of the assessment results to achieve fair and legitimate conclusions at the end of the HTA process. The application of the model will usually require more time and resources than traditional HTA. An initial assessment of the degree and the character of complexity of a technology might be helpful to decide whether or not the whole process or only specific elements will be applied.

Integrated Health Technology Assessment for Evaluating Complex Technologies (INTEGRATE-HTA). This project is co-funded by the European Union under the Seventh Framework Programme (Grant Agreement No. 306141)
.