administrations and sections. The sections are the most elementary decision making unit.
Research function Is the section involved in preparation or evaluation?
policy To what extent does manager of the Is performance information of the section section use performance information of used in the accompanying documents to the
section in the annual evaluation of the annual budget? the professional staff?
Rescaling: 1 if there is involvement in Rescaling: the scale from 0 to 4 is Rescaling: 1 for yes, 0 for no preparation and/or evaluation. 0 if there divided by 4. is
only involvement in policy execution. Is performance information of the section To what extent does the manager use Is performance information of the section used in
the minister’s policy plans? performance information of the section in used in annual report of the section? the annual evaluation of the other personnel of the
section? Rescaling: 1 for yes, 0 for no Rescaling: the scale from 0 to 4 is Rescaling: 1 for yes, 0 for no or no divided by 4. Self Assessments? Rescaling: 1 for yes,
0 for no used for allocation of resources? annual report used in monitoring policy contracts with agencies? Rescaling: the scores on the scale from Rescaling: 1 for
yes, 0 for no 0 to 5 are divided by 5. Is performance information used for Is performance information of the section To adjusting policies? used for the steering of
teams? what extent is performance information of the section used in the annual evaluation of the manager of the section? Rescaling: the scores on the scale from
Rescaling: the scores on the scale from Rescaling: the scores on the scale from 0 to 5 are divided by 5. S (scores) /4 0 to 5 are divided by 5. S (scores) /4 0 to 4
are divided by 4. S (scores) /4
Is the performance information used in Is performance information of the section Is performance information of the section
Table 37: Indications from survey data for the initial user profiles
The combination of the two selection criteria (sectoral and departmental spread and concentration, and the initial user profiles) led to the following selection.
Policy Sector Education
Department Department of education
Section EDU_1 EDU_2 EDU_3 EDU_4 EDU_5
Research 0,75 0,05 0,65 0,92 0,65 0,42 0,4 0,25 0,7 0,92 0,7 0,55
Management 0,49 0,75 0,2 0,39 0,1 0,28 0,28 0,25 1 0,36 0,83 0,2
Profile (++ = 0.75; 25 -75; 0 < 25)
0,44 Research (++), Management (+) Accountability (+) 0 Research (0), Management (++) Accountability (0) 0,25 Research (+), Management (0) Accountability (+) 0,75
Research (++), Management (+) Accountability (++) 0,5 Research (+), Management (0) Accountability (+) 0,25
Spatial planning and Department of Environment and SPH_1 housing Infrastructure SPH_2 Waterways marine affairs and Department of Environment and WATER_1 Infrastructure
WATER_2 Culture Public Health Media Department of Welfare, Public Health and Culture Department of Welfare, Public Health and Culture Department of Science, Innovation
and Media CUL HEALTH MEDIA
Research (+), Management (+) Accountability (+)
0,31 Research (+), Management (+) Accountability (+) 0
Research (+), Management (+) Accountability (0)
0,63 Research (+), Management (++) Accountability (+) 0,75 Research (++), Management (+) Accountability (++) 0,69 Research (+), Management (++) Accountability (+) 0,5
Research (+), Management (0) Accountability (+)
Table 38: Selection of the cases based on policy sector, parent department and initial user profile
The survey data on the one hand and the semi structured interviews on the other should allow for both a triangulated user profile and an assessment of the effects of
performance information. The triangulation of the user profile will be based on a set of items that refers to the uses of performance information formulated by Behn
(2003) (Table 5). The effects are assessed from a list of effects are described on page 195. Note that we only studied the effects that have an impact on the output.
The interviews were not taped. Yet, a report of each interview was drafted and sent to the managers for feedback in order to increase construct validity (Yin 1994).
Most of the managers suggested improvements to the reports. The analysis is based on the final reports. Finally, the link between the profiles should provide insight
about the central hypothesis. Does use determine effect?
Eight purposes that public managers have for measuring performance Purpose Evaluate Control Budget For which questions can performance information be useful? How well
is my organization performing? How can I ensure that my subordinates are doing the right things? Category Research Management
On what programs, people, or projects should my organization spend Management money? How can I motivate people to align oneself with the objects of the section?
How can I convince superiors, politicians, interest groups and the media Accountability that my organization is doing a good job? How to show good performance of the
section to the employees? Why is what working or not working? What exactly should we do differently to improve performance? Accountability Research Research
Celebrate Learn Improve
Table 39: Eight purposes that public managers have for measuring performance (Behn 2003) – items in the semi-structured interviews
Finally, we stress that this study is qualitative research. Qualitative research goes beyond how much there is of something to tell us about its qualities (Miles and
Huberman 1994). Apart from the case selection, the study is based on approx. 10 page interview reports. This is rich data. Yet, in order to make sense of the data, we
will be counting. This is not in contradiction with the qualitative nature of the study. Even in qualitative research, a lot of counting goes on in the background.
Miles and Huberman (1994: p.253) give three reasons for using numbers in qualitative research. First, numbers allow to better see what data you have. Numbers are more
economical. Second, counting may be done to verify a hypothesis. It allows identifying the cases that confirm or reject the hypothesis. Thirdly,
they assert that counting protects from bias. It is more difficult to mould research findings – intentionally or unintentionally83.
9.6. Results 9.6.1. triangulation of the user profiles
The profiles that we derived form the survey and that were used in order to select the cases will now be examined. Triangulation in social research is the combination
of different methods, methodological perspectives or theoretical viewpoints (Miller and Brewer 2003). The metaphor of the stability of a tripod is used to assert that
the result of using varied approaches is a net gain. Four methods of triangulation can be identified (Denzin 1978). First, between-method triangulation uses different
research techniques – usually a qualitative and a quantitative one. Second, within method triangulation uses different variations within a technique (for instance
different scales in a survey) to measure the same thing. Thirdly, investigator triangulation implies that two or more researchers independently study the same
phenomenon. Fourthly, data triangulation is about using different data blocks but the same methodology. We are applying between method triangulation. The main idea is
to compensate the strengths and the weaknesses of both quantitative and qualitative methods. The triangulation is based on the semi-structured face-to-face interviews.
Table 40 represents the triangulation schematically. The first column is an acronym of the sections. The code refers to the policy sector: education (EDU), spatial
planning and housing (SPH), waterways and marine affairs (WATER), culture (CUL), health (HEALTH) and media (MEDIA). The second column repeats the initial profile that
was used for selecting the cases. Each case may have a high, moderate or low profile on the three categories of use – research, management and accountability. The
third column contains the updated profile based on the interviews. The fourth column provides the degree of convergence between the initial and updated profile. The
measure gives the number of actual corrections over the number of potential corrections. For instance a change from + to ++ is counted as one correction. A change from
++ to 0 is counted as two corrections. The latter correction is more drastic. EDU_1 potentially has four corrections. RES (++) can have a maximum of two corrections to
RES (0). MGMT (+) can have maximum one correction to MGMT (0) or MGMT (++). The same goes for ACC. The score 1/4 means that the initial profile is 25% different from
the updated profile. Finally, we indicated the direction of the correction. A positive correction indicates that the
Latest completed orders:
|#||Title||Academic Level||Subject Area||# of Pages||Paper Urgency|