Organization: UN Children’s Fund
Country: Panama
Closing date: 16 Oct 2017
Background
The new UNICEF Strategic Plan (2018-2021) highlights the importance of evidence and data. UNICEF needs to be able to understand the situation of all children and provide the conditions so that governments have the capacity to understand it as well, with solid and timely information. We need to be able to measure the situation and progress of the Agenda 2030. As a first step, we need to understand who are the most vulnerable and why, and secondly why and how results for children are being achieved or not.
As part of the advocacy upstream role, UNICEF requires a solid foundation of data and evidence to: i) position children’s rights at the top of development agenda in high and middle income countries, ii) prioritize programmatic areas of focus that use and create evidence, and iii) strategically document “what works” and “what does not work” to achieve results for the most disadvantaged.
UNICEF Regional Office for Latin America and the Caribbean (LACRO) contributes to the generation of quality knowledge products that use data, analyze the situation of children identifying barriers and bottlenecks for the realization of their rights (Situation Analysis of Women and Children), analyze the results of interventions/models and national policies and programmes (Evaluations), and explore hypothesis that may proove or not prove to be correct to generate new knowledge (Research). In some cases, knowledge products may involve the use of methods that include participation of children and/or address sensitive issues that should have the right approach and the most adequate ethical considerations.
In 2013, the UNICEF Executive Board approved the ‘Revised Evaluation Policy’ to ensure that UNICEF has timely, strategically focused and objective information on the performance of its policies, programmes and initiatives. The policy defines evaluation as “an assessment, as systematic and impartial as possible, of an activity, project, programme, strategy, policy, topic, theme, sector, operational area or institutional performance”. All evaluations in UNICEF are supposed to follow the same guiding principles of rigour and transparency and share the same purpose of organizational learning and accountability. UNICEF provides guidance on the format and content of all evaluation terms of reference and report and UNEG (United Nations Evaluation Group) Norms and Standards are the basis for an independent grading system for all final reports (GEROS).
In 2015, UNICEF has also approved the ‘Procedure for Quality Assurance in Research’ to guide UNICEF’s research and to support the quality. As per the procedure, research is “the systematic process of the collection and analysis of data and information, in order to generate new knowledge, to answer a specific question or to test an hypothesis. Research 0methodology must be sufficiently documented to permit assessment and replication. Research in UNICEF should examine relevant issues and yield evidence for better programme and policy advice”. All research activities in UNICEF are supposed to follow the aforementioned research quality procedure.
The Director, Division of Data, Research and Policy at UNICEF approved the UNICEF Procedure for Ethical Standards in Research, Evaluation and Data Collection and Analysis to guide UNICEF’s evidence generation activities and to support the integrity of UNICEF’s evidence base in order to ensure that UNICEF’s programmes, policy and advocacy activities are grounded in ethical principles and practices. All evaluation and research/studies in UNICEF involving human subjects or analyzing sensitive secondary data are supposed to follow the aforementioned ethical standards procedure.
Situation analysis of women and children are key knowledge generation product in which country offices analyze, at the national or subnational level, the situation of the children and women’s rights in different child-related areas. In 2012, UNICEF published a guidance to conduct a rights-based, equity-focused Situation Analysis (SitAn). The guidance defines the SitAn as a disaggregated assessment of the status of children and a trend in the realization of children’s and women’s rights. The SitAn is intended to make an important contribution to shaping national development strategies to accelerate achievement of the child-related goals with equity. The SitAn is also part of the UNICEF country programme preparation, and is therefore done in collaboration with counterparts, as part of UNICEF advocacy strategy in the region.
Between 2016 and 2017, UNICEF Country Offices in the Latin America and the Caribbean (LAC) region have planned to conduct 183 knowledge generation-related activities, including Evaluation (14%), Research (25%), SitAns (14%), and other studies and M&E capacity building activities. According to UNICEF policy and procedures, one of the Regional Office functions is to ensure that evaluations and research reach the expected quality (Quality Assurance function).
As part of its quality assurance and oversight function, UNICEF LACRO supports the creation of quality evidence through evaluations that feed into programming, assess models that are working or not working, communicate solid evidence, and contribute to the corporate agenda for evaluation with models that can be scaled up and adapted in other country/regional realities. Since mid-2015, the LAC Regional Office has established a quality assurance system for terms of references and evaluation reports. The quality assurance system has interesting results and has, in general terms, contributed to increase the quality of evaluations in the region. For SitAns, the LAC Regional Office provides quality assurance support to all COs conducting SitAns. For research, the Regional office needs to implement a system that enforces quality of research products. Ethics reviews are conducted in specific cases where there is participation of children, and when sensible issues have been addressed. In addition, in order to enforce the quality of all knowledge generation products and encourage office to use knowledge generation as a strategic management tool to support the UNICEF advocacy agenda and programmatic work, the LAC Regional Office has made available three different templates for the preparation of concept notes for evaluations, studies and research since January 2017.
Objective
With the objective of reaching the highest quality of research, SitAns, evaluations, and studies that also include ethical standards, when applicable, the LAC Regional Office intends to reinforce the existing quality assurance system for evaluation and SitAns and propose a system for research and studies, and also to include ethical reviews in knowledge products when needed. This consultancy has three main sections:
An assessment that includes specific analysis of existing quality assurance systems for different knowledge products:
Evaluations: A specific analysis of the current quality assurance system for evaluations, including: a) analysis of the quality mechanism in place; b) review of templates, procedures, guidance, current financial arrangements, feedback from country office colleagues using the current system, etc.; c) review and comparison of quality assurance system of the current LAC system vis a vis GEROS review; and d) identification of lessons learned and good practices.
SitAns: A quick review of the current status of the quality assurance of national, sub-national, and thematic SitAns in the region. The assessment includes: a) analysis of the current quality assurance system of SitAns, according to the guidance received and documentation available; b) review of templates, procedures, guidance, etc.; and c) identification of lessons learned and good practices.
Recommendations for a) a reinforced quality assurance system for evaluations b) a reinforced quality assurance system for SitAns, c) the establishment of quality assurance system for research knowledge products; d) the establishment of a quality assurance mechanism for studies that promote higher quality, but also focuses on the use of these knowledge products.
Preparation of practical tools for the use of quality assurance system for each type (Evaluations, SitAns, Research, studies) of knowledge products and Ethical reviews (both in English and Spanish), including:
Template for ToRs, inception reports, and final reports
Template for quality review of ToRs, inception reports, and final reports
Quality Assurance procedure and guidelines
A list of potential centers of excellence, academic organizations, research centers or other organizations, institutions, or companies that may be suitable to implement quality assurance reviews for each or all specific products or a set of products (considering the language needs of UNICEF LAC – Spanish capacity is a must).
Estimated cost of the proposed quality assurance systems by knowledge product, considering the Latin American Region context.
In order to benchmark from different experience, the assessment should include the review of knowledge generating mechanisms in two other UNICEF regional offices.
Scope of Work and Deliverables
Under the supervision of the UNICEF Regional Monitoring and Evaluation Specialist (for evaluations, research and ethical reviews) and the Regional Monitoring Specialist (for SitAns), the professional consultant will undertake the following outputs:
Assessment report of current quality assurance system for Evaluations. This assessment should include a review of a sample of Draft ToRs and Draft Evaluation Reports and final versions (since mid-2015), a comparison with GEROS review, mechanisms in place, tools and procedures, and financial arrangements.
A proposal of a quality assurance system for evaluations, research, SitAns, studies, and Ethical reviews. It should include clear guidelines and procedures for their implementation, and proposed costing of quality assurance system (based on the number of working days needed for conducting the reviews).
A toolkit with practical tools, as specified in paragraph 13 of this document (in English and Spanish). Be aware that most of these tools already exist and only need to be refined.
Scan of centers of excellence, academic organizations, research centers or other organizations, institutions, or companies that may be suitable to do quality assurance reviews for specific products or a set of products (considering the language needs of UNICEF LAC-Spanish is a must).
These outputs should be based on the UNICEF’s Evaluation Policy, Policy of Research and will be conducted in accordance with guidance, norms and standards defined by the United Nations Evaluation Group (UNEG, ethical considerations guidelines, SitAns guidelines, etc.). The Institution/organization/company must commit with the ethical Code of Conduct for Evaluators in the United Nations System.
Approval of Products
The Regional M&E Specialist and Monitoring Specialist will review products delivered by the Consultant and will approve their final version. The consultant will inform the M&E Specialist and Monitoring Specialist of any remarks on the product, within five (5) working days after the product has been delivered.
Qualifications
The required consultant will be a qualified professional with the following characteristics:
Excellent and proved knowledge of Evaluation methods and approaches
Excellent and proved knowledge of Research and Ethical Reviews and approaches
Previous experience with Quality Assurance systems
At least 10 years of proven professional experience in designing and conducting evaluations, research, situation analysis, and/or ethical reviews.
Excellent analytical and writing skills in Spanish and English
Excellent interpersonal skills with ability to work in a multi-cultural environment
Post-graduate University degree in subject related matters
Working knowledge in the LAC region
Familiarity with UNICEF programmes
Duration of contract
The implementation of this consultancy has a duration of 3 months.
To Apply
Applicants will submit their proposals which should include:
A CV and/or P11
A short technical proposal (2 pages max.) describing methodology, timeline, and workplan
An economic proposal including a total fee to conduct this consultancy, and disaggregated by product, as described in paragraph 17.
Incomplete proposals (technical and economic) will not be considered.
Additional considerations
UNICEF reserves the right to withhold all or a portion of payment if performance is unsatisfactory, if outputs are incomplete, not delivered of for failure to meet deadlines
All materials developed will remain the copyright of UNICEF and UNICEF will be free to adapt and modify them in the future.
How to apply:
UNICEF is committed to diversity and inclusion within its workforce, and encourages qualified female and male candidates from all national, religious and ethnic backgrounds, including persons living with disabilities, to apply to become a part of our organization. To apply, click on the following link http://www.unicef.org/about/employ/?job=507526
