Accountability is a concept used with many definitions and in many contexts (bureaucratic, political, professional, legal). In governance and management it can be defined as the duty to ensure and report that the use of authority is aligned with rules, standards, policy and interests of the program, organization and the broader group of stakeholders. Its assessment looks at the extent to which it is defined, accepted, and exercised along the chain of command and control within a program. In the context of research for development, accountability can be seen the obligation to take responsibility for performance in light of commitments, to the extent that performance is in the control of the program/project. Accountability requires ownership and acceptance of responsibility and the ability to deliver or influence the delivery of the desired results.
We consider size 250K USD multi-year activities. We need to ensure that we are capturing also activities that are smaller than that, but yet key in achieving the set outcomes. Therefore, we need to enfold them as an activity within your project larger activities. Activities like this will be led by an Activity Leader.
In research for development impact pathways, adoption, use and uptake refer to the primary and direct use of the research output by beneficiaries. This use may lead to changes; i.e. outcomes and impacts. Influence is a related term that implies an effect of research results on the primary beneficiaries or course of events.
An ex‐ante assessment of the quality, relevance, feasibility and potential for impact and sustainability of a research program or activity, usually prior to a decision on funding it.
The causality between observed (or expected) changes and an output from research or related activity. Attribution refers to both isolating and estimating the particular contribution of a program/project to the outcome/impact.
Financial and management audit in the CGIAR provides accountability to management at the level of the Center Boards, Consortium and Fund Council on finances and assets and also provide elements of oversight in human resources and business efficiency.
An analytical description of the situation prior to research activities, against which progress can be assessed or comparisons made. This can include stakeholder analyses, social network analyses, policy analyses, other on-going initiatives inventory, etc.
Objectivity and impartiality on the part of evaluators (which is not guaranteed by structural independence; for example evaluators may be reluctant to be critical of people they think may provide them with future contracts).
The individuals, groups, or organizations, whether targeted or not, that benefit, directly or indirectly, from the chain of events that research has contributed to (see also Target Group).
Is what is put forth through CCAFS and covers Window 1 (W1), Window 2 (W2), Window 3 (W3) and bilaterally raised funds by CGIAR centers, excluding leveraged funds from non-CG partners.
- W1: Funds to support the entire CGIAR program portfolio including through CRPs as well as to proposals from the consortium for support to other critical activities that are vital for successful implementation of the strategy and results framework.
- W2: Funds designated to one or more of the CRPs
- W3: Donor funds specifically earmarked to a CRP through a center
- Bilateral: Funds raised directly by CGIAR Centers through concepts notes and proposals. Centers decide to which CRP they want to map their bilateral and not the donor who decides.
In these standards this term refers to the primary clients of evaluation – those requesting or receiving the evaluation (for example senior managers or donors of a CRP). Elsewhere it is often used to refer to the target group for a research project.
Clusters of Activities
Are structural components of a CRP flagship. They are the summary description of a range of products that are linked and related with each other e.g. through their contribution towards an outcome.
In economic terms, a comparative advantage in producing or selling a good is possessed by an individual, firm or country with the lowest opportunity cost (as opposed to absolute cost) in producing the good. In these standards the term refers more broadly to the role and mandate of the CGIAR in producing international public goods where there are no alternative research suppliers that are better positioned to produce those goods.
The extent to which the program has achieved or is expected to achieve its results at a lower cost compared with alternatives. Cost‐effectiveness analysis is distinct from cost‐benefit analysis, which assigns a monetary value to the measure of effect. In research programs costing of outputs is more feasible than outcomes that typically depend on conditions and activities outside of research.
The situation or condition which (hypothetically) would have prevailed if there had been no program/project.
A deliverable is a specific, tangible product generated by a project, such as a workshop report, paper, dataset, scenario, etc. Each deliverable contributes to one key-output. Unlike key outputs which are generic categories, deliverables are specific and time bound, with an expected year of completion.
The extent to which the program or project objectives were achieved, or are expected to be achieved, taking into account the exploratory nature and risks inherent to research.
A measure of how economically resources/inputs (funds, expertise, time, etc.) are converted to results. In research context, assessment of efficiency refers to activities and outputs that are in the control of the research programs or cut across several CRPs and takes into account the exploratory nature and risks inherent to research. In the private sector “value for money” is commonly used for efficiency.
Are the beneficiary population, usually quite massive, making it unfeasible for a project or program to work with them directly.
The systematic and objective assessment of an on‐going or completed project, program or policy, its design, implementation and results. In the CGIAR evaluation refers to an external, completely (IEA commissioned) or largely (CRP commissioned) independent and systematic study of an in‐depth nature that uses clear evaluation criteria. In addition to research, it applies also to central CGIAR institutions, support programs and themes, and the System as a whole. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision‐making processes of major stakeholders.
Different aspects of quality of a program which are used internationally to develop evaluation questions and serve as a check that all major issues have been considered. In the CGIAR these include relevance, efficiency, quality of science, effectiveness, impact and sustainability.
Evaluation reference group
A structure set up to work with the evaluation managers to ensure good communication with, learning by, and appropriate accountability to primary evaluation clients and key stakeholders, while preserving the independence of evaluators.
The information presented to support a finding or conclusion. Such evidence should be sufficient and relevant. There are several sources for evidence: observations (obtained through direct observation of people or events); documentary (obtained from written information); analytical (based on computations and comparisons); self‐reported (obtained through, for example, surveys) and experiential (based on professional understanding and expertise that is accumulated over time) and is based on credible and legitimate sources.
Focuses on program/project implementation and is improvement-oriented.
Global public goods
These are defined as goods with the three following economic properties: ‘non‐rivalrous’ (i.e. consumption of this good by anyone does not reduce the quantity available to others), ‘non‐excludable’ (it is impossible to prevent anyone from consuming it) and available worldwide. In the CGIAR also the term International public goods is used. It refers to issues that are deemed to be important to the international community; and typically cannot, or will not, be adequately addressed by individual countries or entities acting alone.
Positive and negative, primary and secondary long‐term effects resulting from a chain of events to which research has contributed, directly or indirectly, intended or unintended. These effects can be economic, socio‐cultural, institutional, environmental, technological or of other types. Note that sometimes the term impact is used to refer to more immediate results, here defined as Outcomes. It is the change in welfare of end-users (or beneficiaries), mostly farmer communities and others, as in the case of environmental impacts in the long-term (20, 50, 100 years), resulting from a chain of events to which research has contributed, directly or indirectly, intended or unintended.
In the CGIAR this term is generally used for an ex post study which uses specialized methods to estimate the changes in selected development parameters and the extent to which these are attributable to defined research activities or programs of the CGIAR. The Standing Panel on Impact Assessment (SPIA) has an oversight and capacity building function for impact assessment in the CGIAR.
The systematic assessment of the effects ‐ positive or negative, intended or unintended ‐ of program/project output(s) or intervention(s) on the outcomes/impacts of the affected groups and environments, and the extent to which these outcomes/impacts can be attributed to the program/project.
The causal pathway for a research project or program that outlines the expected sequence to achieve desired objectives beginning with inputs, moving through activities and outputs, and culminating in outcomes and impacts. Assumptions underpinning the causal chain and feed‐back loops are usually included (Closely related terms include Logical Framework and Theory of Change.)
In conducting an evaluation, the absence of bias in due process, in the scope and methodology, and in considering and presenting achievements and challenges. The principle applies to the clients of the evaluation, donors and partners, management, beneficiaries, and the evaluation team.
An evaluation that is carried out by entities and persons free from the control of those involved in policy making, management, or implementation of program activities. This entails both organizational and behavioural independence, protection from interference, and avoidance of conflicts of interest.
A quantitative or qualitative variable that represents an approximation of the characteristic, phenomenon or change of interest (for instance, efficiency, quality or outcome). Indicators can be used to monitor research or to help assess for instance organizational or research performance.
The financial, human, and material resources used in research.
Intermediate Development Outcome (IDO)
At CGIAR’s research program level targets representing CRP‐specific thrusts and target domains that are generated as a result of multiple activities by diverse actors outside the CGIAR. Their scales reflect CRP target domain and estimated volume of benefits. At System level IDOs represent accumulation of CRP outcome results with the scale corresponding to the CGIAR’s target domains.
International public goods
see Global public goods
A deliverables is interoperable when:
- The deliverable is published on a repository that implements a metadata schema (In the CGIAR we use the CGCore Metadata Schema which is based on Dublin Core standard).
- The deliverable is published on a repository that implements an interoperability protocol (i.e. CGspace https://cgspace.cgiar.org/ or Dataverse).
- The (meta) data uses a formal, accessible, shared, and broadly applicable language for knowledge representation.
- The (meta) data uses vocabularies that follow FAIR principles (Findable, Accessible, Interoperable and Reusable).
- The (meta) data includes qualified references to other (meta) data.
As a criterion for assessing governance and management, the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program — including shareholders, other stakeholders, implementers, beneficiaries, and the community at large.
Is the person who controls the budget for the project, they are responsible for it from the programmatic, either regional, flagship or CU portfolio perspective and will be the ones that 1) pre-fill a) their respective portfolio plans (impact pathways, with indicator and targets), b) the (contractual) project details and 2) will coordinate feedback to the projects at the various stages (e.g. planning, reporting) and 3) they will be the ones who need to summarize and synthesize from what they get reported from the projects.
Major Output Groups (MOGs)
The large groups of outputs, products and deliverables (e.g. new knowledge from research together with new capacity in a specific group of stakeholders to use that knowledge and enact CSA policy) that are necessary for achieving an outcome. They are in correspondence to what are now called clusters of activities.
A process of continuous or periodic collection and analysis of data to compare how well a project, program, or policy is being implemented against expected progress and results, in order to track performance against plans and targets, to identify reasons for under or over achievement, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff, while evaluation as defined in the CGIAR Policy and Standards is carried out by external evaluators. Monitoring is also used for research purposes to guide decisions on research design and adjustment.
In the context of the CGIAR, this refers to the accountability of all partners, including donors, for the efficiency of outputs, outcomes and impacts of a program, institution or policy and sustainability of research.
Are actors such as national research institutions, extension organizations, NGOs and others, which access CG products directly. Next-users can create an environment that enables the target impact for end-users; decision makers that we want to influence to achieve outcomes.
The intended or unintended short‐term and medium‐term effects resulting from an intervention’s outputs. Note that the term impact is sometimes used as a generic term for effects, including the short‐term results, here defined as Outcomes. See also Research outcomes. At CCAFS we define outcomes as the changes in next-users that need to happen so that an enabling environment is created and the targeted impact can occur. These are medium-term practice changes that occur through the adoption, use or influence of the research product by the next-users, who end up doing things in different ways. These practice changes are underpinned by the related necessary changes in knowledge, attitudes and/or skills. What do next-users need to DO to contribute to the enabling environment needed for achieving the targeted impact? In CCAFS we have 2 levels of outcomes, with 2019 outcomes shared between regions and flagships contributing to 2025 outcomes. The timestamp on the outcome represents the year by which CCAFS anticipates meeting the related outcome target in each flagship.
see Project Outcome Statement
see Project Outcome Story
The products, new knowledge and services which result from research, capacity building and other activities related to research for development. Can be research findings, insights, experience, problems solved, etc., resulting from research and measured by achievement of deliverable and process indicators.
Can be individuals and organizations that we work with to generate our outputs and products and to interact with next-users.
- Expertise, network and influence with next-users.
These can be 1) budget receiving partner to lead an activity or 2) budget receiving partner contributing to implementation of parts of an activity or 3) only leverage funding providing implementation partners or 4) a combination of any of the three afore mentioned.
A process of review involving qualified individuals within the relevant field. Peer review methods are employed to maintain standards of relevance and quality to improve performance and provide credibility. A peer review may be an input into an evaluation.
The continuous process of setting goals, measuring progress, giving feedback, coaching for improved performance, and rewarding achievements
The ongoing monitoring, measurement and reporting of program accomplishments and progress toward pre‐established goals, which involved collecting data on the level and type of activities (inputs) and the products and services delivered by the program (outputs).
The individual projects of research of development that contribute to a major research action. Each Flagship is made up of multiple projects, which in turn may be made up of multiple activities.
Is the person in the lead and coordinating role for the project. S/he is responsible for the ultimate delivery of the project and the coordination with the project partners as well as the contact person for the management liaison.
Project Outcome (Statement)
Describes ‘who (exactly) is doing what (exactly) in a specific way (and often in a different way from before)’ to achieve our set goals. (see image: http://screencast.com/t/RsKgyrbzMl
Project Outcome Story
We are asking at the planning stage for you to anticipate any story that gives evidence of your project activities contributing to some behavioral changes in the coming year. This is just a guess and you might identify a different story at the reporting stage, but we have found it useful for the projects to aim for these stories as it helped project teams to be more aware of looking out for these kind of stories to report.
Are other contributing organizations and nominated people within these organizations.
The extent to which the objectives of a development intervention are consistent with global and national priorities and policies, as well as those of intended beneficiaries, partners and donors. In these Standards, it also refers to the extent to which the program is consistent with the goals, the System Level Outcomes, comparative advantage and reform agenda of the CGIAR, and program activities are consistent with the objectives of the program and its Intermediate Development Outcomes.
The effects from research outputs applied by intermediary users, for instance by national partners or international research or development organizations.
The output, outcome or impact (intended or unintended, positive and/or negative) of an intervention.
A management strategy focusing on performance and achievement of outputs, outcomes, and impacts.
An assessment of the progress and performance of an intervention (including research), periodically or on an ad hoc basis. The words evaluation and review are often used interchangeably, but in the CGIAR, an evaluation refers to an external, completely (IEA commissioned) or largely (CRP commissioned) independent and systematic study of an indepth nature using clear evaluation criteria, whereas reviews may be more flexible and narrow in focus.
Scaling up and scaling out
In agricultural development the terms are used nearly interexchangeably to refer to the expansion of beneficial impacts from agricultural research and rural development. Scaling up/out relate to expanding, replicating, adapting, and sustaining successful policies, programs, or projects in geographic space or over time to reach a greater number of people. Scaling is typically preceded by piloting the model, idea or approach initially in a small scale. Scaling‐out may refer specifically to the adoption and adaptation to local circumstances by users; while scaling‐up may refer to extension and institutional support related to scaling.
Agencies, organizations, groups or individuals who have a direct or indirect interest in the CGIAR or its component, for instance research program or its evaluation.
Focuses on assessing worth of the program/project and lessons learnt (results and consequences), for instance to enable assessments with respect to change, continuation or enlargement of the program/project.
The continuation of benefits from a program intervention after research has been completed; the probability of continued long‐term benefits or scalability of the benefits; the resilience to risk of the net benefit flows over time.
System Level Outcomes (SLOs)
The high level impact goals of the CGIAR: Reduction in rural poverty; Increase in food security; Improving nutrition and health; More sustainable management of natural resources.
The individuals or organizations for whose benefit the research or activity is ultimately undertaken, for example farmers or consumers in particular regions or agroecologies. (The word client can also be used, but is reserved in these Standards for primary clients of evaluation.
Is the explanation that we will need to ensure that we understand your target value contribution towards the indicator and that we are when aggregating these numbers. See example above (‘in at least 10 countries) and (Colombia, Honduras, Guatemala, Nicaragua), would be some narrative explaining the target value.
Is the number that your project or region is contributing towards a selected indicator. The target value has the format of a number only, as the indicators wordings have been standardized. E.g. Outcome indicators from Flagship 1: # of national and subnational development initiatives and public institutions prioritize and inform project implementation of equitable best bet CSA options using CCAFS science and decision support tools. Target value FP1 for 2019: 15 (in at least 10 countries). Target value contribution LAM 2019: 4 (Colombia, Honduras, Guatemala, Nicaragua). When it comes to unpacking the target values and break them down by years, we have agreed that this would be accumulative. E.g. Target value FP1 2015: 2 and 2016: 4 (this means that each year 2 initiatives have started using CCAFS support tools).
Theory of Change (ToC)
Presents a hypothetical identification of the ways by which change is expected to occur from output to outcome and impact along an impact pathway. The TOC questions the assumptions about causality underlying the relationships between outputs, outcomes and impact. In TOC the assumptions present the mechanisms of change. There is no single method or presentational form agreed for TOCs. In research it is often used as a framework for testing hypotheses of how the intervention worked or is expected to work and incrementally building up the evidence base for the assumptions.
The costs of planning, adapting and monitoring tasks completion. Transaction cost analysis includes comparison of transaction costs under alternative governance or operating structures.
The use of different sources or types of information, evaluators or types of analysis, to verify and substantiate an assessment, in order to overcome the potential bias that comes from a single source or method.
Last Update - October 22, 2015