- (2015) Volume 2, Issue 2
University of Southern Denmark, Institute of Public Health, Unit for Health Promotion Research, Denmark
National Research Council, Rome, Italy
Received date: November 25, 2015 Accepted date: December 11, 2015 Published date:December 14, 2015
Aim: Research evidence is rarely the primary driver of contextually salient and feasible policy making; instead, contextual knowledge and factors play a significant role. This paper argues why we need to develop indicators, which can be used to assess if policy making is done in an evidence-informed way.
Methods/results: Based on up-to-date theories and literature, this paper suggests a set of initial indicators using the example of physical activity policy making in a school setting.
Conclusion: These initial indicators need to be further tested and validated in empirical studies in different contexts.
Keywords
Policy making; Evidence-informed; Indicator; Research evidence; Context; Public health; Health promotion
Introduction
In the last 10 years the concept of evidence-informed policy making (EIPM) process has been accepted especially in public health and health promotion [1]. For policy making process in these areas evidence-informed is more relevant than the evidence-based approach, since research evidence is seldom the primary driver of policies. Decisions are also influenced by characteristics of settings or populations; contextual issues such as traditions, political priorities, values, and available resources such as money, time and competences [2-4]. Hence, the engagement and participation of stakeholders such as politicians, administrators, professional groups, citizens and nongovernmental organizations shape the policy making process in which input from research evidence and other types of knowledge needs to be balanced to make the process evidenceinformed [3-5]. Furthermore, the quality of evidence relevant for policy processes within public health and health promotion can hardly be assessed by traditional evidence hierarchies used in evidence based medicine. Instead, qualities as relevance and transferability must also be assessed [1,2].
As pointed out by Morgan-Trimmer (2014), since “policy is political; our ideas about knowledge translation must be too” [6]. Thus to look at knowledge translation as a passive process where researchers only disseminate research evidence to policy makers and expect it to play a role in the policy making process, might be too naive. A number of models have been proposed to describe the interaction between research and policy making, including different actors and perspectives and investigating the different roles of science and society as possible drivers of the knowledge utilization [7-10]. Hence, instruments for evaluation of both the process and effect of the use of evidence in EIPM are needed. Of all these instruments, indicators would be the most appropriate to assess if the policy making process is evidenceinformed. Among the many indicators proposed and developed in several fields of public health/health promotion, there is a lack of specific EIPM indicators; however, e.g. in the transport sector, attempts on indicator development have been made [11]. The fact that public health and health promotion work deal with population-level decision making in a highly contextual settings, means that indicators from clinical medicine do not fit. These indicators are based on a more individual doctor-patient approaches and isolation of interventions from their context [2]. Instead, the instruments needed for EIPM should cover different dimensions and also the process aspects of policy making. Thus, there is a need for development of indicators to support the process and/or assessment of whether policy making can be said to be informed by evidence in the field of public health and health promotion. To follow the principles of scientific approach, this development should happen along the lines of indicator development frameworks if possible.
The aim of this paper is to suggest which frameworks can lead to the future development of set of indicators to assess the level of or the effectiveness of EIPM in the field of public health and health promotion.
Indicators as EIPM instrument
It should be possible to find a way to assess if, how (well) and with what effect policy making process is informed by evidence. An indicator may aim to express how far in the process to reach an objective one is and what is the effect if we reach the objective. However, in spite of papers published on facilitators and barriers of EIPM [1,5], and recently also summarizing empirical research on interventions to enhance use of research in policy making [12], it still remains an open question when policy making can be said to be informed by evidence, to have been, or to have good chances to be informed by evidence through the policy trajectory. Beyond using evidence levels to guide and/or judge EIPM process, another option is to use indicators as a support tool to assess if and how relevant evidence is considered; to what extent stakeholders relevant for the policy in question participate in policy making process and how policy making process respects contextual priorities, values and resources.
How to develop set of theory-based indicators for EIPM process?
The multiplicity of dimensions involved in outlining indicators for EIPM would require a complex set of frameworks, each of them dealing with specific actions, processes and contexts. As EIPM may be regarded as an innovation inside an organisation, frameworks representing innovation process and related categories of determinants should be considered [13]. By another point of view, as the knowledge conversion process between tacit and explicit knowledge is deeply involved in policy making, also the related frameworks should be considered [14-16]. In addition, a further important dimension for real-world policy making to take into account is the systems approach. It helps to understand complexities of policy making as systems with non-linear interdependencies between different levels of policy making triggered by e.g. media events, negotiated interests and communication [17].
In this paper, our discourse is focused only on those framework specifically aimed at describing EIPM processes and the recursive phases of policy making inside an organisation.
Based on the recent literature, one such framework could be the trans-disciplinary model by Satterfield et al (2009) which describes how
a. evidence-informed decision making process happens in the economic, social and cultural context;
b. the best available research evidence is taken into account; and
c. the local priorities and values as well as
d. resources and competences from sectors and professionals are respected and taken into account.
The latter can be understood to include also existing infrastructures, networks etc. [3].
For indicator identification this means: The (most) EIPM process would happen when the points from a) to d) are taken into account and indicators are identified for each of the elements. Satterfield et al framework can be used for different phases of policy making process; e.g. in policy prioritisation, development, implementation and evaluation of whole policy making process.
Another framework to assess and guide EIPM – would be Knowledge to Action (KTA) Cycle [18], building on research evidence synthesis, providing locally relevant (research) knowledge, assessing facilitators and barriers of knowledge integration and working towards tailored interventions, from which we learn again and feed back to the knowledgebase. The KTA framework has been further developed by the Canadian Health Research Institute to include scalability of knowledge, inter-sectorial collaboration, implementation systems, economic analyses, and equity and ethics considerations [19].
KTA framework helps to see policy making as dynamic activity of a process of applying knowledge and learning from it. Furthermore, Daugbjerg et al (2009) in their analysis of physical activity policies in Europe also emphasized the need for being transparent for the accountability of policy decisions and implementation [17-19].
For indicator identification this means: Policy making process is evidence-informed when different stakeholders, including researchers, as active participants create their salient and feasible policies, pay attention to equity and ethics as well as evaluation and accountability of policy processes to learn from them and potentially scale them up or transfer to other contexts.
One more relevant framework is that related to the stages of knowledge utilization describing six stages of knowledge use seen as a process, which include: reception, cognition, discussion, reference, effort and influence [20,21].
For indicator identification this means: Policy making process is evidence-informed when knowledge is cumulatively transferred through the different stages of knowledge utilization. Figure 1 describes a scenario from physical activity policy in the school setting, given as an example of EIPM process in which the issues relevant for indicators described above are integrated.
Figure 1 Example of integrating the identified indicator areas.
Application of the set of indicators
Use of indicators in this context means application of these indicators, in different stages of policy making process and/ or with already developed policies, for assessment of whether policy making process can be said to be informed by evidence. The suggested frameworks could in principle be used by researchers and policy makers but also by other stakeholders and even citizens to define own indicators to assess the policy making and its process. This is possible especially when these suggested frameworks are used in a rough descriptive format. For scientific research purposes more refined, tested, SMART (specific, measurable, achievable, relevant, time bound) indicators are necessary. However, their development and testing can be built upon the suggested frameworks and indicator areas.
Naturally, when using these indicators, either to analyse existing policies or those to be developed, also measurement and potential quantification, as well as benchmarks, need to be thought of. Later empirical data should be used to test and validate the suggested indicators in different contexts. The central idea is that these indicators per se, on the generic level, would be independent of context; however, context-specific indicators have to be developed for different purposes.
Conclusion
EIPM process is a realistic way of integrating evidence with community priorities, values and resources. To enhance EIPM process, the suggested theory-based frameworks can be used to assess if and how policy making really is informed by evidence and at the same time respecting community’s influence on the policies concerning them.
Competing Interests
The research leading to these results within the REsearch into POlicy in Physcial Activity (REPOPA); Oct 2011-Sept 2016; has received funding by the European Union Seventh Framework Programme (FP7/2007-2013); grant agreement no 281532. This document reflects only the authors’ views and neither the European Commission nor any person on its behalf is liable for any use that may be made of the information contained herein.
7824