BMA Foundation for AIDS
Health Education Authority
Sex Education Forum

Published in 1997 by the BMA Foundation for AIDS, BMA House, Tavistock Square, London WC1H 9JP

Key messages

Research has identified some common characteristics of effective sex education programmes despite methodological concerns with some studies. Furthermore it has shown that some sex education programmes do encourage young people to delay first intercourse and use a condom or other contraception.

It is important that the effectiveness of sex education is evaluated and monitored in the following areas:

• how well sex education programmes are implemented
• the success of the programme in developing understanding, knowledge and attitudes
• the success of the programme in developing practical skills
• the success of the programme in affecting behaviour and health.

Effectiveness should not be defined solely in terms of behaviour change or health outcomes. Except in very large studies, it is unlikely that research will be able to detect direct effects of sex education on sexual behaviour and health.

A range of different research methods is needed in order to understand the effectiveness of different approaches to sex education. Large scale experimental designs (randomised controlled trials) may be necessary to identify behavioural change. However, smaller qualitative studies are needed to make sense of these results and to support the development of practice.

Research funders should seek to ensure that research is realistic, and that it will have a positive impact on the development of educational practice, and that it complements and builds upon existing research findings. Funders should also seek to promote collaboration between different research projects and to facilitate the dissemination of research findings to practitioners.

In summary, research methodology and outcomes must be appropriate and relevant to sex education practice, if research findings are to lead to more effective education. Researchers and practitioners need to cooperate more closely and strategically to ensure the development of good practice and research.

Introduction

There is growing interest in measuring the effectiveness of different programmes and models of teaching sex education. Such research can guide government, education and health authorities, advisers, schools, teachers, and trainers in improving sex education and ensuring it meets the needs of young people. The interest in effectiveness reflects the trends towards evidenced-based purchasing in the NHS and demonstrable outcomes in education. However, measuring effectiveness is not always straightforward.

This briefing paper is intended to help those who:

• commission/purchase interventions
• or commission/fund research
• or conduct research
• or advise and support schools on the development of good practice

to understand the role of research in the area of sex education.

This paper is based on a workshop held by the BMA Foundation for AIDS, the Health Education Authority and the Sex Education Forum, at which leading researchers, practitioners and advisors shared their experience and considered ways of developing outcome measures for research into the effectiveness of sex education which are realistic in practice and which reflect its underlying educational objectives. The paper reflects the discussion at the workshop but the views it expresses are not necessarily shared by all the participants.

What does effectiveness mean?

Before investigating whether a sex education programme is effective, one needs to consider what it could realistically be expected to achieve, and how to measure this. The aims of sex education are ambitious, and include:

• Enabling people to form and maintain relationships
• Promoting self-esteem
• Enabling control of fertility
• Enabling prevention of HIV and STDs.

It may not be possible to demonstrate whether a sex education programme succeeds in meeting these aims or to distinguish the impact of the programme from other factors such as family, peer group and socio-economic influences. To seek to do so would involve subjecting sex education to much more exacting standards than are expected of other subjects taught at school. For example, the aims of mathematics teaching might include enhancing understanding of financial matters, but we do not judge its success by how well pupils manage their finances in later adult life.

In some large research studies it may be possible to look at the relationship between sex education and long-term outcomes such as people's well-being, relationships and sexual health. But to help practitioners to develop and improve the quality of sex education we also need measures of effectiveness which relate closely to the school environment - ie how well is it taught, and what are pupils learning?

Outcome measures for sex education can be grouped under four main headings:

• Preconditions and implementation
• Understanding, knowledge and attitudes
• Skills
• Behaviour and health

Preconditions and implementation

Sex education cannot be effective unless there is an infrastructure for its delivery and a supportive school ethos. Thus, while having the right preconditions in place does not prove that pupils are learning well, it is an important first step and a relatively straightforward way of identifying schools which are likely to be providing good quality sex education. Outcomes to look for might include:

• Sex education being identified in the school management plan and the existence of formal policies on sex education, confidentiality, equal opportunities, and bullying
• Evidence that the sex education curriculum is planned, eg clear schemes of work which specify intended learning outcomes, and that the defined programme is actually completed
• Evidence of identified lead personnel and clear communication systems
• Evidence of the availability of trained staff and opportunities for further staff development and training
• Evidence of involvement of teachers, pupils, parents, governors, support staff, and outside agencies
• Evidence that a variety of teaching and learning methods are used and that pupils' knowledge is assessed, eg from observation of lessons, sampling of pupils' work, and interviews with teachers and pupils
• "Consumer" satisfaction - pupils, parents and teachers
• Evidence of mechanisms for monitoring and evaluating the programme and sustaining its development

Research tools for measuring these indicators could include checklists and inspection frameworks. Qualitative research methods may be needed to develop, test and standardise the best measures.

Understanding, knowledge, and attitudes

The effectiveness of teaching of most subjects can be judged by assessing what pupils have learnt, and sex education is no exception. Learning means much more than knowledge, however, and in sex education assessment of factual knowledge should be accompanied by qualitative measures of how far pupils have been enabled to gain a critical understanding of sexuality and the social and cultural climate in which they live, for example by:

• Individual and small group interviews asking young people (in confidence, not in the classroom) whether and how the lessons have made them think about issues such as:
  sexual values
  responsibility, eg helping others
  being helped, eg using services
  sexism
  homophobia
  influences such as peers, media and parents
• Asking young people to tell stories or describe "critical incidents" which demonstrate their understanding
• Observations of teaching to show that it has engaged young people in group discussions, roleplays etc, and assessment of how the pupils have participated
• Interviews with young people and parents to assess how far they were involved in negotiating and planning the sex education curriculum and their satisfaction with it.

The aim is not to assess whether the pupils' attitudes conform to a particular set of values, but how clearly they have thought about and are able to discuss the issues.

Skills

The development of skills is crucial in enabling young people to translate understanding and knowledge into practice. It may be possible to evaluate whether a sex education programme succeeds in developing the following:

• Intrapersonal and communication skills around being able to express one's emotions, resisting pressures, negotiating, making decisions and managing relationships and conflict
• Getting condoms and using them correctly
• Giving and receiving sexual pleasure
• Skills and confidence such as coping with periods and managing contraception
• Finding out about sexual health services and getting what one wants out of them.

Ways of measuring the acquisition of skills include looking at young people's profiles and records of achievement, observing how they role-play different scenarios, and asking them to judge their own confidence and ability in doing specific things (eg raising the subject of condoms or contraception with a partner) or to give practical demonstrations (eg how a condom is used). Some major studies measure skills with scales or questionnaires developed and piloted using qualitative research methods.

Research can also help in identifying the skills-base that teachers need to deliver sex education, and what sorts of training are most effective in developing such skills.

Behaviour and health

Behavioural and health outcomes of sex education include fairly straightforward concepts such as safer sex, using services or avoiding unwanted conceptions and STDs but also more subtle aspects of sexual experience and relationships. However, the fact that some outcomes are easy to define may not mean they are easy to measure or to relate to the education which young people have received (see box on conception rates). Except in very large studies, it may be unrealistic to expect research to be able to show that school sex education has any directly measurable behavioural or health outcomes, in view of all the other factors which influence sexual health and lifestyles. Qualitative work is important in identifying and prioritising the most useful outcome measures, but possibilities include:

• Self-reported data on use of condoms/contraception, unwanted conceptions, STDs, unsafe sex
• Self-reported age at first intercourse and numbers of partners - this information is undoubtedly useful in understanding sexual behaviour, but there is not a consensus as to whether increasing ages at first intercourse or decreasing numbers of partners are always desirable outcomes
• Prevalence of chlamydia antibodies at specified age(s) - this shows whether people have ever been infected with this STD so the outcome is not affected by whether or not they have received treatment. But it is awkward to measure because it requires blood samples
• Self-reported use or non-use of sexual health services by those who needed them - but how is "need" defined, given that it will depend in turn on people's sexual behaviour?

In terms of more general aspects of sexual experience and couple relationships, it may be possible to look at outcomes such as:

• Self-reported pleasure in sexual activity, eg specifically on the last occasion when the person had sex - there may be quite strong gender differences in this measure as men and women may attach different priorities to different aspects of sexual satisfaction
• Positive experiences of non-sexual aspects of couple relationships (or of not being in a relationship for those who aren't)
• Fewer regrets resulting from unsatisfactory sexual experiences or missed opportunities - though this may be hard to interpret since sex education might lead to increased regrets if it raises people's expectations more than it improves their actual experiences
• Reductions in coercion or harrassment
• More considered decisions about whether to have sex within any relationship - perhaps this might be measured by asking whether people expected to have sex on the occasion when they first did so in a particular relationship, whether they planned for it and whether they discussed it in advance with their partner (or anyone else)

Clearly, these types of outcome are only really suitable for use in long term studies where people are followed up some years after receiving sex education, and considerable development work may be needed to devise and test reliable measuring instruments such as questionnaires.

Conception rates as a measure of the effectiveness of sex education - some pitfalls

The government has set a target for reducing conceptions among under-16s, and school sex education should help to achieve this. However, under-age conception rates are not an ideal measure of the effectiveness of sex education, because:

• About 8 in every 1000 girls aged 13-15 get pregnant each year. This is higher than in other European countries, and a serious public health concern. But it is still a minority of girls, making it hard to measure small statistical changes which might result from improved sex education. A 25% cut in the rate would be an excellent outcome, but would mean only one fewer pregnancy per 500 girls. A large sample would be needed to distinguish this from a random fluctuation.
• Under-16 conception rates are affected by many factors, eg socio-economic conditions and access to family planning services, making the effects of sex education hard to disentangle.
• Using under-16 conception rates as the only measure of effectiveness would miss the opportunity to study the effects of sex education for the majority of young people who don't get pregnant - eg less concrete benefits such as knowledge or the quality of relationships.
• The under-16 conception rate combines two variables: the proportion of girls having sex before the age of 16; and the proportion of heterosexually active (young) people who use contraception effectively. Because most people first have intercourse at an age fairly close to 16 (from 15 to 19), a small and perhaps not very important fluctuation in age at first intercourse can have a big effect on the under-16 conception rate. This makes any change hard to interpret. It suggests that under-age conceptions may not be a good proxy measure for overall sexual health.

An alternative might be to measure conception rates among older teenagers aged 16-19. But this requires longer-term research which costs more, and quite large samples would still be needed. Also, while many would agree that pregnancy in a girl under 16 is a "bad" health outcome even if it is deliberate and wanted by the girl herself, for older groups it is more debatable which pregnancies are "bad" outcomes and which are desirable.

Researchers in one major study hope to use the proportion of women who have had an abortion before the age of 20 as one indicator of the effectiveness of sex education - since an abortion clearly represents an unwanted pregnancy. But such research is complex - to confirm a link between abortions and sex education researchers need either to track down women some years after they have left school and ask them whether they have had an abortion or to match lists of women who attended different schools with official abortion records. Such use of strictly confidential abortion records requires special authorisation. Also, whether or not a woman with an unwanted pregnancy goes on to have an abortion is affected by socio-economic factors and by her attitudes and beliefs, which could in turn have been influenced by the sex education she received.

Hence although in large complex studies it may be possible to show directly that improved sex education leads to fewer unwanted conceptions, this is unlikely to be feasible in smaller studies.

Criteria for judging the quality of research

Where the aim is to measure the effectiveness of sex education in achieving concrete behavioural or health outcomes, then firm criteria can be laid down for assessing the quality of the research:

• random assignment of control groups and groups to receive the sex education programme being studied
• large sample sizes to show meaningful results
• measurement of longer term outcomes, eg effects at least 12 months after the programme is taught
• measures of actual or intended behaviour as well as knowledge and skills
• clarity about the proper unit of analysis - eg the school rather than individual pupils - and use of statistical methods which reflect this.

But this does not mean that large randomised studies are the only useful sort of research. Qualitative methods are valuable for a variety of purposes:

• enabling local or smaller scale evaluation of the quality of teaching and identifying needs for training or support
• assessing broader, less quantifiable outcomes of sex education such as those relating to understanding and attitudes
• interpreting the results of randomised studies and relating them to practice - eg what particular aspects or features of a programme make it effective and/or well-liked, and how could this knowledge be built upon in improving teaching still further?
• devising and testing methods of measuring outcomes - this can require a trade-off between ease of measurement, transferability across different studies and settings, and appropriateness in terms of what one is actually trying to assess
• developing theories about the mechanisms and pathways by which education and other factors interact to determine people's behaviour
• identifying how sex education can best meet the particular needs of groups such as young people who are growing up gay, lesbian or bisexual, and/or are from minority ethnic communities.

What do we already know about the effectiveness of different types of sex education?

Reviews of studies and evaluations of sex education have shown that this is a difficult area of research and many studies have some methodological drawbacks. However, some studies have shown that sex education programmes can have desirable effects such as delaying first intercourse or encouraging use of condoms or other forms of contraception. No reliable study has shown undesired effects on these outcomes. One major review concluded that programmes which have been found to be effective in terms of behavioural and health outcomes share the following characteristics:

• They focus narrowly on specific behavioural objectives - eg delaying intercourse or using condoms
• A theoretical grounding in terms of how action is affected by understanding and beliefs
• At least 14 hours tuition or teaching in small groups which allows more efficient use of time
• Active learning methods which involve pupils and help them to personalise information
• They convey accurate information about risks of unprotected sex and how to avoid them
• They address social pressures on young people - eg ways of responding to specific "lines" which a partner might use to persuade someone to have sex
• Clear values and messages appropriate to the pupils' age and experience - eg promoting delay in first intercourse for younger ages, but emphasising condom use for older groups who are more likely to be already sexually active. No programme using an "abstinence-only" message (ie no sex before marriage) was found to be successful.
• Activities to practise relevant communication and negotiation skills and increase confidence in using such skills
• Effective training for the teachers or educators delivering the programme.

Some caution is needed, however. The emphasis on closely defined behavioural outcomes makes it difficult to know whether the programmes also achieved broader outcomes such as enhancing the quality of relationships. Moreover, few studies of individual programmes or curricula have been replicated. This is of concern, because expensive mistakes can be made in assuming that an approach which has worked in research in a few settings will yield the same results if it is adopted more widely. Many factors may affect the success of a programme, eg the ethos of the schools, the expertise and commitment of the teachers (not only to sex education but to the particular programme methods), the pupils' background and wider social context. These factors cannot be replicated exactly when a programme is scaled up for widespread use. Indeed the mere fact of taking part in research may increase pupils' and teachers' enthusiasm and motivation, and hence the chances of success. So it is important to re-evaluate programmes in use and not to place all one's faith in experimental studies.

Translating research into practice

When commissioning research, it is important to be clear about what questions it is intended to answer and whether these are realistic. Given current resource constraints, the focus of research should not be abstract enquiry, but finding out more about how to meet young people's needs and prepare them for adult life. Clearly, it would not be possible for a study to measure all the different outcomes suggested in this paper, and some sense of priority is needed. In addition, possible new research questions include:

• what classroom and whole-school processes shape the effectiveness of a sex education programme?
• how does sex education affect other aspects of the school - eg does a good sex education programme reduce problems of bullying or help pupils to perform better in other subjects?
• to what extent are life-skills transferable between different subject areas - eg whether practice in finding out and making decisions about career options helps young people's sexual health skills and vice versa - and whether methods of measuring skills developed for other areas (eg management training) can be adapted for sex education?
• what is the relationship between sexual well-being and wider emotional/psychological well-being (eg including the Health of the Nation targets for mental health)?
• how do interventions without any sexuality component influence sexual behaviour - in the US one programme which involved young people in volunteer work and gave them regular contact with a caring adult led to a lower conception rate - and how could such interventions be combined with sex education?

What are the factors that lead the UK to have the highest teenage pregnancy rate in Europe, as opposed to other northern European countries such as the Netherlands and Sweden?

If research findings are to be translated into better sex education for young people, then it is essential to disseminate these in ways which are meaningful and accessible to practitioners. This means more than just publication in research reports, academic journals and conference presentations. Opportunities are needed for researchers to join with advisors, trainers and practitioners to share in the interpretation of their work and consider its implications for the future development of sex education. This in turn implies a need for closer strategic cooperation between agencies which fund research and agencies which support sex education development and training, to ensure that research funding is linked to appropriate development work. Similarly, because cross-fertilisation between research groups using different methodologies and approaches can stimulate better quality work and help to ensure comparability of findings, opportunities are needed for researchers to exchange ideas throughout the course of their research projects, and not only when presenting final results.

Acknowledgements

The BMA Foundation for AIDS, the Health Education Authority and the Sex Education Forum would like to thank all the workshop participants who contributed their insights towards this paper.
List of workshop participants

Researchers
Dr Charles Abraham, School of Social Science, University of Sussex
Sarah Blacksell, Institute of Population Studies
Susan Charleson, Institute of Education, Social Science Research Unit
Dr Stephen Clift, Reader in Health Education Centre for Health Education and Research, Canterbury Christ Church College
Katie Froud, Senior Research Officer, National Foundation for Educational Research
Diedre Fullerton, Centre for Reviews and Dissemination, University of York
Dr Roger Ingham, Reader in Health and Community Psychology, Centre for Sexual Health Research, University of Southampton
Anne M Johnson, Reader in Epidemiology, University College London Medical School
Mary Jane Kehily, Institute of Education, University of London
Dr Douglas Kirby, Director of Research, ETR Associates, USA
Kevin Lowden, Scottish Council for Research in Education (SCRE)
Dr Alex Mellanby, Research Fellow, University of Exeter, Postgraduate Medical School
Dr Judith Stephenson, UCL Medical School
Dr John Tripp, Department of Child Health, University of Exeter, Post Graduate Medical School
Ian Warwick, Assistant Director, Health & Education Research Unit, Institute of Education, University of London
Daniel Wight, MRC Medical Sociology Unit, Glasgow

Practitioners
Max Biddulph, Centre for the Study of Human Relations, School of Education, University of Nottingham
Alice Cruttwell, Adolescent Programme Manager, North Birmingham FHSA
Jane Hood, Independent consultant (Sex Matters)
Gill Lenderyou, Independent trainer/consultant
Marilyn Toft, General Advisor for Health Education, Quality Assurance Department, Lewisham LEA

Facilitators
Hilary Curtis, Executive Director, BMA Foundation for AIDS
Rachel Thomson, Senior Development Officer, Sex Education Forum

Rapporteur
Caroline Ray, Information Officer, Sex Education Forum

Planning group
Adam Crosier, Health Education Authority
Hilary Curtis (as above)
Lorraine Hoare, Health Education Authority
Judith Stephenson (as above)
Rachel Thomson (as above)
Daniel Wight (as above)