case study – training evaluation project

Back in the 1990s, the UK Offshore Operators changed the basic offshore safety & emergency training standards which are mandatory for most people working offshore in the UK sector. Two years later, they commissioned me to evaluate the effectiveness of these changes.

All training evaluation exercises are difficult. This one was particularly unusual, in that the training programmes give people skills and knowledge they should never have to use. It was also sensitive in terms of industrial relations.

I applied a ‘responsive’ style of evaluation, based on interaction with respondents. In the absence of assessed outputs, and given the significance of perceptions and attitudes, it was not appropriate to apply quasi-empirical methods. There were 10 phases to data-gathering for the project:

1. Desk research to establish quantitative data on training activity.
2. Discussion sessions with small groups of offshore personnel.
3. Structured interviews with representatives of duty holders.
4. Structured interviews with a sample of employers and suppliers, supplemented by a postal questionnaire.
5. A confidential postal questionnaire of a random sample of course delegates.
6. Review of in-house evaluation exercises conducted by employers.
7. Interviews with trades union officials and representatives of regulatory authorities.
8. A confidential postal questionnaire issued to over 900 offshore safety representatives.
9. Visits to a cross-section of UK training centres.
10. Review of research literature.

The results were published across the industry in a series of presentations on all UK installations. The client described this as the most extensive and successful consultation exercise they had ever undertaken.