Evaluating a national skills training programme

What is shaping government

Government departments, PSUs and public programmes need independent evidence on citizen needs, service delivery and programme outcomes, often gathered across large and diverse geographies.

• Programmes are expected to show measurable outcomes
• Citizen feedback is shaping service delivery
• Large field surveys need rigorous quality control

A skills development ministry, working in Eight states. Client identity is withheld to protect confidentiality.

Organisation

A skills development ministry

Sector

Government

Team engaged

Market Research

Services

Government research, Quantitative research, Qualitative research

The ministry needed evidence on employment outcomes after training. Placement figures came from training centres themselves, and the ministry wanted independent evidence. The ministry needed independent evidence on whether the programme helped trainees find jobs.

• Deliver on the brief: evaluating a national skills training programme.
• Understand the views, needs and behaviour of trainees.
• Measure how widespread each pattern is, and explain the reasons behind it.
• Give the client evidence it can act on within 14 weeks.

The work focused on 10,000 trainees.

• trainees10,000

Audiences we typically reach in government

Citizens and householdsProgramme beneficiariesFrontline workersLocal officialsBusinesses using public services

Coverage: Eight states. Market: India.

• India

A mixed-method design: structured research measured how widespread each pattern was, and qualitative conversations explained why.

1. 1Brief
2. 2Tracer survey
3. 3Employer interviews
4. 4Insight

We traced trainees and interviewed employers. Trainees were traced six and twelve months after completion by telephone and field visits, and employers were interviewed about job readiness. Interviews with employers explored skills gaps and satisfaction with trainees. Training centres were also visited to assess equipment, trainer qualifications and links with local employers.

Government researchQuantitative researchQualitative research

trainees

10,000

• Business respondents were screened on role, company size and involvement in the decision before being invited.
• Qualitative participants were recruited purposively to cover the full range of experiences that mattered, rather than for statistical representation.

• Interviews and discussions were led by experienced moderators using a guide agreed with the client.

• Recordings and transcripts for every session
• Re-screening of qualitative recruits
• Logic and range checks on the final data
• Work carried out under our ISO 20252 quality procedures

• Cross-tabulation by key groups and locations
• Significance testing on the differences that mattered
• Thematic coding of interviews and discussions
• Journey and decision mapping

• Outcomes were strongest where training linked to local employers.
• Training providers with weak outcomes received additional support.
• Placement data became part of funding decisions for training partners.

Tracer studies showed real outcomes.

• Build on the core insight: tracer studies showed real outcomes.
• Focus effort where the evidence was strongest: outcomes were strongest where training linked to local employers.
• Re-measure after the changes to confirm the effect.

• The ministry expanded employer partnerships.
• The ministry also introduced tracking of employment outcomes for all training centres.

What changed in the business as a result of the work. Impact is described qualitatively.

• Insight report and management presentation
• Cleaned data file and data tables
• Interview summaries and key verbatims
• Evaluation findings with programme recommendations

• Repeat the measure after changes are in place to track progress.
• A natural follow-on question: what changed after a programme?