Terms of reference for external evaluation for the project “Keeping Children Safe in Sports”

 

 

Terre des hommes Romania is seeking a qualified external evaluator for evaluating “Keeping Children Safe in Sports” (KCSS) – co-funded by the European Commission (EC) and implemented by Terre des hommes (Tdh) Romania in partnership with Tdh Hungary and Tdh Hellas.

 

Terre des hommes (Tdh) is the leading Swiss child relief agency. Founded in 1960, we are committed to improving the lives of millions of the world's most vulnerable children. Through our innovative protection and health projects, we provide assistance to over three million children and their families in almost forty countries each year. In Europe, Tdh intervenes directly or indirectly in many countries, relying on teams based in Budapest, Albania, Greece, Kosovo, Romania, Moldova and Ukraine. Every year thousands of children and their families benefit from projects implemented either at a national or regional level.

 

In Romania, Terre des hommes (Tdh Ro) has been active since 1992. For the past few years, the delegation has been working on three main intervention areas: (1) Strengthening child protection and juvenile justice systems, (2) Protection of children affected by migration and/or trafficking, and (3) Social inclusion of minorities.

 

The project KCSS aims to promote child safeguarding in sports settings in Romania and Greece, as a means to protect and safeguard children. The specific objectives are to ensure children are protected in sports and camp settings through dedicated safeguarding policies, to build capacities of professionals in contact with children, to empower children to recognize and address safety risks, and to encourage multi-country exchange and cooperation regarding child safeguarding in sports and recreational settings.

 

In recent years, the topic of child safeguarding has gained more and more prominence in the sports world. By the concept of safeguarding, we understand the responsibility of an organization “to make sure their staff, operations, and programmes do no harm to children, that is that they do not expose children to the risk of harm and abuse, and that any concerns the organisation has about children’s safety within the communities in which they work, are reported to the appropriate authorities” (definition of the Keeping Children Safe Coalition).

Childhood rights are protected by the Romanian and Greek Constitution as well as through EU Directives (e.g. Law for Responding to Domestic Violence 3500/2006; Adoption of the EU Council Convention for child protection from sexual abuse Law 3727/2008) and the UN Convention on the Rights of the Child (Ratification Law 2101/1992). Child protection issues are further mainstreamed and regulated by the Romanian and Greek Civil Code and the Penal Code, for instance under family law, basic individual protection principles, and breaches of physical and emotional integrity, or personal freedom.

 

Despite the safeguarding risks in sports clubs and summer camps, child-safeguarding training was almost non-existent for most staff in these settings before the implementation of KSSS. In recent years, this gap has been marked by several high-profile abuse cases in Greece and Romania reported in the media pertaining to extra-curricular activities such as in sports clubs and summer camps.

 

Project description

 

Title: “Keeping Children Safe in Sports “(Mediu Sigur pentru Copii în Sport), co-funded by “Rights, Equality and Citizenship” Program of the European Union (REC 2014-2020)  

Start and End date – duration: 01.09.2019 - 01.12.2021

Total budget: 447,222

 

General and specific objective:

Overall objective: Promote child safeguarding policies in sports and recreational settings in Romania and Greece, to protect and safeguard children, as well as to equip staff with the necessary training and guidance.

Specific objective 1: Ensure children in sports clubs and summer camps in Romania and Greece are protected through dedicated safeguarding policies, developed and institutionalized by sports and camp associations and their members.

Specific objective 2: Build capacity of professionals in contact with children to raise awareness and identify and address gaps in child safeguarding policies in sports and recreational settings.

Specific objective 3: Empower children and youth in sports and recreational settings to claim their rights and expect adherence to established child safeguarding policies.

Specific objective 4: Support multi-country exchange and cooperation regarding child safeguarding in sports and recreational activity settings.

Expected results:

Expected result 1 (R1): The child safeguarding systems and knowledge, attitudes and practice of key stakeholders in sports and recreational settings are surveyed and researched and solutions are proposed.

Expected result 2 (R2): Sports federations and recreational associations have developed, adopted and implement child safeguarding policies, encouraging their members to respect them.

Expected result 3 (R3): Sports clubs and summer camps have sound safeguarding systems in place.

Expected result 4 (R4): Professionals in contact with children in sports and recreational settings have an increased knowledge on child safeguarding principles.

Expected result 5 (R5): Increased commitment of professionals in sports and camps settings to respect child safeguarding principles.

Expected result 6 (R6): Children’s awareness of abuse and safeguarding principles and reporting mechanisms is raised.

Expected result 7 (R7): Best practices in the area of child safeguarding are shared between the 2 countries and on the European level through Terre des hommes’ ChildHub online community of practice.

 

Project monitoring:

Monitoring is a continuous function designed to provide Tdh with early indications of the quality, quantity and timeliness of progress towards delivering intended results.

Tdh developed and implemented a detailed monitoring plan for the project and employs a comprehensive, results-oriented evaluation strategy to produce timely information about project activities and assess the impact made by projects.

 

For each project component, Tdh identified specific outcome indicators that reflect overall program results. These include a mix of quantitative and qualitative indicators and have been and are compiled to create a composite picture of program impact across each outcome.

 

The following indicators were measured in order to assess the progress of the project:

● # of reports on child abuse in sport settings registered by authorities;

● % of sports clubs signing the child safeguarding policies;

● # of sports associations and federations supporting project implementation and adopting child safeguarding policies;

● # of professionals trained on child safeguarding;

● # of policy makers who are publicly supportive of the awareness raising campaign;

● # of mass and social media reports on the child safeguarding in sports;

● # of capacity building events;

● # of users accessing the project specific content on ChildHub platform;

● # of young people and children in sports settings who participated in public debates about child safeguarding in sports.

The data collected during the baseline and end-line evaluation provided the results for the following indicators:

% of children who have improved their level of safeguarding knowledge, attitudes and practices.

% of adults who have improved their level of safeguarding knowledge, attitudes and practices

 

The data was gathered or analyzed by the project managers or the organization’s M&E expert. Such data comprises agendas, minutes, attendance sheets, actors involved (participants in trainings, children involved etc.), various types of deliverables (methodologies, reports, policies, training materials developed etc.), self-assessments, feedback forms, pre/post training tests. The additional case studies help to verify if the following qualitative indicators have also been met: Children can give clear examples of practices that ensures their safeguarding and know how to implement them; Adults can give clear examples of child safeguarding practices and know how to implement them.

 

Type and number of deliverables:

48 deliverables, out of which the most important ones are the following:

• research developed: the ToT and training methodologies about safeguarding, for adults and for the children from the sport and summer camp staff,
• the e-course developed,
• the safeguarding policy and procedures developed for Football, Gymnastic federations, sport clubs (Romania) and summer camps (Greece),
• the self-assessment reports conducted in Romania and Greece,
• the procedure to award the quality label for the sports clubs and camp associations,
• the drawing competition for the logo of the quality label in Romania and the one in Greece,
• the media campaigns conducted in Romania and Greece,
• the collection of best practice,
• the final international conference.

Activity summary:

The project implementation started with research (R1) on child safeguarding systems in targeted sports federations, clubs and camps and on the knowledge, attitudes and practices of professionals in contact with children in sports and camps settings. The research methodology included qualitative research tools and quantitative online tools. Focus groups, semi-structured interviews and a questionnaire will be applied to collect the data. The research targeted camp and sports personnel, child protection professionals and children. This baseline informed the further capacity building activities and awareness raising. Support was provided to sports federations and camp associations to develop and further promote the child safeguarding policies (R2). After these policies were adopted, the sports clubs and camps were encouraged to take over these policies and respect them. A special quality label was  created in partnership with associations and federations to publicly acknowledge the clubs and camps which have sound safeguarding systems in place (R3). In the same time, professionals in contact with children in sports and recreational settings were involved in several capacity building activities in order to inform them about the child safeguarding principles and support them in committing to respect these (R4, R5). In this sense, cascade trainings were organized. Child participation was ensured by creating a Child Advisory Board in each country and children participating in sports clubs and camps were involved in a series of sessions challenging the existing social norms and increasing their knowledge about reporting mechanisms (R6). To make the interventions sustainable and to share best practices, an exchange of experience was ensured all along the project, using the ChildHub European online community of practices and facilitating exchange meetings among project beneficiaries.

 

This way, the project contributed to the embedding of child safeguarding policies in sports and recreational settings, both to protect and safeguard children, as well as to equip staff with the necessary training and guidance. The project covered activities such as the adoption and formalization of appropriate child safeguarding and child protection policies, protocols and frameworks, capacity-building of professionals on child protection, and awareness-raising of existing standards, in line with the standards of the "Keeping Children Safe - Child Safeguarding Standards and how to implement them" guidance.

 

The project also included capacity building for adults in contact with children in sports and camps on practical application of Keeping Children Safe standards to specific settings.

 

 

 

The external evaluation has three main objectives:

1. Accountability: to obtain an objective and independent assessment of the results achieved compared to the initial project plan (objectives, results, target group, indicators, outputs and deliverables) by looking at compliance, relevance, effectiveness and efficiency.

The accountability is towards both the funder (EC): the evaluation report is one of the project deliverables; and the project partners and beneficiaries, exploring the value of the project for them, considering their views and feeding back lessons learnt to them.

2. Learning: understand the effects of the project and promote common thinking and learning; Identify lessons learned from the main stakeholders and provide recommendations that can be fed into future related activities on safeguarding in sports

3. Scope of the evaluation

 

The evaluation refers strictly to the project “Keeping Children Safe in Sports” (KCSS). The evaluation will measure the quality of the intervention and the progress towards the objective and will emphasize learning.

It requires exchange of information between the Evaluator and project partners, namely Tdh Romania, Tdh Hungary and Tdh Hellas. It also requires communication with project beneficiaries and stakeholders involved in Romania and Greece, as well with the official partners in Romania, meaning Football and Gymnastic Federations and National Authority for the Rights of Persons with Disabilities, Children and Adoptions.

 

The external evaluation should focus on gathering data and information from:

• direct beneficiaries: coaches and other specialists from sport clubs and summer camps from Romania and Greece; children and young people from the project’s Children Advisory Boards, and from sport clubs and summer camps in Romania and Greece.

 

Potential limits of the evaluation:

• Evaluator’s direct access to beneficiaries: almost 200 coaches from Romania and more than 120 professionals from the summer camps from Greece. These adults already provided consistent feedback based on internal monitoring instruments. They can be reached online or by phone.
• More than 150 children and young people benefit of awareness sessions in Greece and Romania. Their feedback may be interpreted mostly based on their written responses to questionnaires or via online evaluation meetings. They can be reached online or by phone.

• Terre des hommes Foundation (project staff, national and regional delegations, HQ) and the other partners involved will directly benefit from the external evaluation results. The most important findings will be shared with the stakeholders involved, namely the sport institutions (camps or clubs) and sports federations we worked with.
• The evaluation results will be used by the project team for reporting towards the donor (EC);
• The evaluation results will be used by programme and project developers for future actions/interventions for developing and implementing child safeguarding procedures.
• The external evaluation results will help:
  • the project team with an objective view over the project development,
  • design future programs and projects focused on safeguarding children in sports,
  • Terre des hommes’ HQ to foster future programmes and projects which could develop and expand child safeguarding policies and procedures in sport and other settings,
  • the donor (EC) to evaluate the project success.

1. Relevance:

• To what extent has the project adequately targeted the real needs of the population?

• Which decisions were taken to consult and adapt project activities to beneficiaries’ needs and views, including based on their age, gender and diversity, within the project`s possibilities?
• Was there an adequate contextual analysis and stakeholder analysis in place before and during the project to assess what child safeguarding policies, procedures and training methodologies to be developed and age, gender, and diversity-specific safeguarding risks and needs exist?
• Did the project respond to the specific needs of the following target groups, including age, gender and diversity-specific ones, if any:
  1. Children from sport clubs and summer camp settings
  2. Professionals from sport clubs and summer camp settings
  3. Sport federations and summer camps internal procedures and activities

• What are the main lessons learned and recommendations suggested for increasing the relevance of future projects focused on safeguarding in sports?
• How the project could address challenges posed by the Covid-19 pandemic, and how has this affected relevance?

 

2. Effectiveness:

• To what extent has KCSS achieved the objectives set and the expected results? What have been the reasons for achievement and lack thereof?
• What overall lessons can the management team of the project learn from the implementation of the project?
• What overall lessons can the organization learn for increasing the effectiveness of its future safeguarding in sports projects?
• What recommendations are suggested for increasing the effectiveness of the management team and the organization’s initiatives focused on safeguarding in sports?

 

3. Efficiency:

• How were the financial/material, human and time resources used during the project? How have they impacted project results, deliverables and outputs?
  • Financial and material resources allocation with direct impact on beneficiaries;
  • Human resources allocation with direct impact on beneficiaries;
  • Time resources management related to project calendar and adjustments;
• What overall lessons can the organization learn for increasing the efficiency of its future safeguarding in sports projects?
• What recommendations are suggested for increasing the efficiency of future safeguarding in sports projects?

 

 

The Evaluator will design the methodology considering the following limitations:

 

Data available. Most of data available are in digital form. The Evaluator will have access and will analyse:

• Initial project document (annex to the Grant agreement), EC imposed regulations and subsequent agreed changes;
• Deliverables already reported to the donor on EC reporting portal or which are in development
• Instruments and tools prepared and documents already collected for ongoing activities, such as self-assessment tests, pre/post tests, feedback surveys etc.;
• Promotion and communication materials distributed or posted on internet

People available:

• Project team comprising: 6 persons in Tdh Romania; 1 person in Tdh Hungary; 4 persons in Greece. Besides these persons, other experts involved in decision making are available in each organization – to be contacted in their national language and/or in English.
• External experts involved: trainers, speakers, participants – to be contacted in their national languages (i.e. Romanian or Greek).
• Beneficiaries: coaches, children and other summer camp staff involved in activities – to be contacted in their national languages.
• Policy makers and top representatives of national and international institutions will be contacted through Tdh Romania in national languages and, when possible, in English.

Distance and COVID 19 constraints:

• Project partners are in 3 countries: Romania, Greece and Hungary.
• Face to face interaction with project team members and the beneficiaries are possible, respecting all protective measures. As the project does not cover travel or accommodation costs for the Evaluator, the meetings can be organized online, via phone/ e-mail if the respondents are not in the Evaluator’s country of residence.

Time constraints:

• The evaluation will be most probably carried out during the final 6 weeks of the project.

 

Ongoing activities:

• The external evaluation report must be completed, delivered, accepted and paid by the end of the project (1 December 2021). During the evaluation process, some of the last thematic project activities will still be ongoing. For these activities, the Evaluator will analyse the current state of development and the foreseeable results in the remaining time until project end.

 

Given the above constraints, the main methods suggested to be used by the Evaluator are:

• Desk research based on documents provided by Tdh Romania and its partners.
• Face to face / phone / online individual and group interviews and focus-group discussions, as most appropriate with project team and other experts involved.
• Phone / online interviews (individual or in group) and focus-group discussions with beneficiaries, as most appropriate.

 

The Evaluator may add to these suggestions other methods and instruments deemed appropriate that take into consideration the above limitations and constraints, making the evaluation process feasible and of expected quality.

 

 

 

 

During the contract, the Evaluator will provide as deliverables (all of them in English):

1. INITIAL (INCEPTION) REPORT – in 1 week within contract start – comprising:

• Context analysis based on project basic documents: Initial project (annex to the grant agreement), EC imposed regulations and subsequent agreed changes, Internal Tdh regulations, Consortium agreements, Deliverables already reported to the donor on EC reporting portal, instruments and tools prepared and communication and promotion materials;.
• Data collection and analysis plan, methodology and instruments for the: • desk research on documents (types of documents to be analysed and needed from partners, key analysis points to use during the document review), • interviews and/or focus groups with team members and associates (list of people to be interviewed, interview and focus group guides), • feedback/views collected from beneficiaries and participants like children, safeguarding focal points, coaches or summer camp staff (contacting beneficiaries online, list of beneficiaries to be contacted, discussion guides).
• Tdh has a specific template to use for the inception report, which will be shared with the selected evaluator in advance.

2. DE-BRIEFING and presentation of findings from the field – in 3 weeks (after the previous stage) – comprising:

• Data collection report (documents analysis, interviews with the team and associates, feedback from beneficiaries and participants);
• Short review indicating the first findings, provisional conclusions, main trends identified etc.

3. PRELIMINARY REPORT – in 1 week (after the previous stage) - comprising

• Narrative report (min. 20 - max. 25 pages);
• Summary table with the main conclusions, main lessons learned and recommendations ;
• Short review and PPT with main lessons learned, recommendations and conclusions.

4. FINAL REPORT – in 1 weeks (after the previous stage) - comprising:

• Executive summary (max. 3-4 pages);
  • Narrative report (max. 25 pages);
• Summary table with the main conclusions, main lessons learned and recommendations
  • Annexes: Containing the technical details of the evaluation, surveys protocols and questionnaires when the case, protocols of interviews, tables or graphics, secondary review references, persons and institutions contacted, a PowerPoint presentation of the findings and recommendations, transcription of relevant selection of interviews, focus groups, observations

 

After each stage a feedback meeting is organized with Tdh and feedback is incorporated in the evaluation document.

Please see the Chronogram and full Terms of reference in the document attached below. 
 

The interested bidders are expected to send their Offer until 7th of October 2021