Measurement and evaluation

This section includes entries about measurement.

Expand Sport-in Development: A monitoring and evaluation manual

Authors

Coalter, F

Date

2006

Keywords

Measurements, sport, monitoring, evaluation, logic model.

Country of research

United Kingdom

Summary of findings

This manual outlines a broad framework and methodology for undertaking monitoring and evaluations of sport-in-development projects.

It is based on the principle that such evaluations should not simply be concerned with outcomes and accountability, but should be developmental, formative and adopt a process-monitoring approach.

It proposes a participatory approach based on the development of logic models informed by theories of change.

The Manual contains five main sections:

1. What is Monitoring and Evaluation?
   This explains the principles of process-led evaluations and the nature of logic models.
2. Aims, objectives and Indicators.
   This explores the nature of aims and objectives and the relationship between them, explores the nature of indicators and explores the concepts of effectiveness, economy and efficiency.
3. Developing a Logic Model.
   This emphasises the importance of understanding programme assumptions about sport and develops an illustrative and detailed step-by-step logic model as the basis for programme evaluation.
4. Collecting information.
   This systematically illustrates the varying methods available for collecting relevant process and outcome information at each stage of the logic model.
5. Reporting Information
   This section provides a standard reporting format and advice on presentation of information. The Maul also includes a comprehensive illustrative questionnaire.

Methodology

Monitoring and evaluation tool

Source of reference

Coalter, F (2006) Sport-in Development: A monitoring and evaluation manual, UK Sport

Web reference

http://www.uksport.gov.uk/assets/File/News/monitoring_and_evaluation_140906.pdf

Expand Policy interventions implemented through sporting organisations for promoting health behaviour change (Review)

Authors

Priest, N; Armstrong, R; Doyle, J and Waters, E

Date

2008

Keywords

Measurements, sporting organisations, health promotion

Country of research

United Kingdom

Summary of findings

This is a systematic review of research on the impact of sporting organisations’ attempts to promote healthy behaviour change (related to smoking, alcohol, healthy eating, sun protection, discrimination, social and emotional health, physical safety and access for disadvantaged groups).  Its intention was to update a previous review in 2004 by reviewing all controlled studies which evaluated policy interventions organised through sporting sessions.  Extensive searches of a range of on-line databases were undertaken.  No studies were found which met the inclusion criteria: randomised control trials; quasi-randomises trials; controlled before and after studies.  The studies indentified and considered are included in a long list of references.  Consequently the authors outline a range of implications for future research in the area of policy interventions organised in sports settings:

•           There is a need for adequate control sporting organisations which do not receive the intervention.

 

•           Baseline data, post-intervention data and longer term follow-up are required.

 

•           There should be sufficient number of clusters (sporting organisations) in each comparison group to allow for generalisable results and the detection of significant differences.

 

•           If no control group is used, then studies should ensure that there are repeated measurements before and after the intervention to control for secular changes in the outcome.

 

•           Where possible, tools validated for population groups should be used to measure outcomes.

 

•           Studies must include both a process evaluation (to measure the integrity of the implementation and contribution of each component) and outcome-evaluation of behaviour change.

 

•           The intervention should have a sound theoretical base which is explicitly reported in the publication.

 

•           Studies must report on information relating to context.

 

•           Where policy is included as one component of the intervention, a factorial design should be used to determine the relative effectiveness of each component.

Methodology

Systematic review

Source of reference

The Cochrane Collaboration

Web reference

http://www.thecochranelibrary.com

Expand Investigating indicators for measuring the health and social impact of sport and recreation programs in Australia

Authors

Cunningham, J and Beneforti, M

Date

2005

Keywords

Measurement; performance indicator; outcomes.

Country of research

Australia

Summary of findings

The aim of this pilot study was to identify potential indicators for health and social outcomes from sport and recreation programmes in indigenous communities in Australia.  The project consisted of three stages:  (i) a literature review to identify specific indicators of potential relevance in the indigenous Australian context;  (ii) discussions with key members of three indigenous communities in the Northern territory to identify community expectations and experiences of sport and recreation programmes; and  (iii) consultation with relevant stakeholders to determine potential usefulness and appropriateness of the indicators identified in the first two stages.  The authors identified three groups of indicators:  (i) Programme viability and sustainability.  These include such aspects as turnover of sport and recreation officers, levels/stability of funding, community consultation and support, involvement, employment and training of local people, succession planning and adequacy of facilities and equipment.  (ii) Participation indicators. definition of target groups and 'sufficient’ participation are determined by programme’s aims;  (iii) Outcome indictators. The authors identify two types of such indicators - those which do/do not have strong evidence of an association with sport/recreation programmes; those which are/are not valued by the community; outcomes do/do not lend themselves to practical and culturally relevant measurement.  The authors conclude by suggesting that priority should be given to programme viability and sustainability indicators and that further research needs to be conducted on the relationships between sport and recreation programmes and health and social outcomes in order to determine the validity of these indicators and enable them to be used in routine monitoring and evaluation.

Methodology

Literature review; interviews

Source of reference

International Review for the Sociology of Sport, 40(1), 89-98

Web reference

www.sagepublications.com

Expand The perceived belonging in sport scale: Examining validity

Authors

Allen, JB

Date

2006

Keywords

Measurement; sport; perceived belonging in sport scale (PBS); validity; reliability.

Country of research

New Zealand

Summary of findings

This article explores the issue of measurement of the social meaning of sport for participants, especially perceptions of belonging (which the author regards as a major motivation for participation).  The article reports on research to examine various aspects of validity (i.e. does the measure assess the hypothetical construct that it is intended to reflect). The author reports on construct (including factorial validity and inter-gender variance), convergent and discriminate validity of the 18 item perceived belonging in sport scale (PBS) (which is adapted from an existing educational measure).  A brief review of theories of the need for belonging, social acceptance, sense of connection and security is provided and the author concludes that this approach has the potential to contribute to our understanding of participants’ perceptions of sports contexts.  It is proposed that a need for belonging can be viewed both as an antecedent to sports participation and outcome of certain contextual factors, such as cohesion. The importance of this is that it is assumed that individuals who feel that their basic psychological needs are being met will be more likely to participate for self-determined, intrinsic, reasons.  The article reports on research undertaken with 257 students from a medium sized English university (age range: 18-49 with varied sporting histories; males, n: 140, females, n: 117).  All completed  questions addressing four aspects of participation – the PBS, a satisfaction with social outcomes scale, the degree of perceived group/team cohesion and the nature of motivation (intrinsic, extrinsic, amotivation).  The results of the descriptive and factor analyses confirmed the construct validity of an amended 11 item  PBS scale, which includes statements about feeling part of a team; having your opinions taken seriously; being treated with respect; friendly team members; being oneself in a team.  The scale was also found to apply to both males and females and had convergent and discriminant validity. Positive relationships were found between perceived belonging and satisfaction with social outcomes.  Thee author suggests that belonging is more than a source of satisfaction, but mediates the relationship between contextual factors and satisfaction.  A positive relationship was also found between perceived belonging and perceived social cohesion of the group. Positive relationships were also found between perceived belonging and intrinsic motivation.  The author concludes that the PBS could be used to test predictions relating to perceived belonging as a mediator/moderator of the relationship between contextual/social environmental factors (e.g. cohesion/motivational climate, coaching behaviours) and desirable consequence of  sports participation ( e.g. self-determined motivation, commitment, enjoyment and long term participation.

Methodology

Questionnaire survey

Source of reference

Psychology of Sport and Exercise

Web reference

http://www.elsevier.com/wps/find/journaldescription.cws_home/620792/description#description