AUST GOVT's $11M contract with Clinton Foundation, no goal, no purpose, no objectives but we paid anyway
Here is a link to a comprehensive Independent Completion Report (ICR) into an $11M agreement with the Clinton Foundation HIV/Aids Initiative Inc.
My only criticism of the review is that it did not examine the corporate structure of the entity it was reporting on, the Clinton Foundation HIV/Aids Initiative (Inc) which had ceased to file financial returns in 2005 and was dissolved as a corporation at the end of 2007. This is not a criticism of the review team as the terms of reference did not ask the reviewers to examine that structure.
Beyond that omission the ICR paints a frightening picture of an Australian Government aid program that handed over more than $11M to the Clintons - and as the report says did it with:
- no statement of project goal or purpose,
- no clear Program Area objectives and
- no targets
Here are some extracts from the ICR.
The William J. Clinton Foundation is a charitable organisation established by President Clinton to construct the Clinton President Centre and to pursue four Missions; Health Security, Economic Empowerment, Leadership Development and Citizen Service and Racial, Ethnic and Religious Reconstruction.
In February 2006, the Australian Government and the CHAI signed a Memorandum of Understanding. Under the agreement, the Australian Government through AusAID is to provide funding up to $25 million over four years, complemented by funding from the Clinton HIV/AIDS Initiative.
Under the MOU, AusAID agreed to fund CHAI to work with public health authorities in three countries including Papua New Guinea to scale-up treatment and care for people living with HIV and AIDS. The other countries were Vietnam and China, with an option to extend the initiative to other countries. Indonesia is the fourth country to be included.
In PNG, AusAID is supporting the CHAI to improve access to HIV and AIDS treatment. Total agreed funding is $11,080,000 for the period 2006-2009.The goal of the CHAI in PNG is to work with the National Department of Health (NDoH) to improve clinical, laboratory and management capacity in Papua New Guinea to enable an expansion of access to antiretroviral treatment for HIV positive people.
Clarity of objectives, indicators and targets
The project is briefly outlined in Section 2 above. The only defining project document is the Clinton Foundation HIV/AIDS Initiative Workplan for 2006-2009. In this document there is no statement of project goal or purpose, thus no indicators at that level, no clear Program Area objectives (with indicators) and no targets. If this is indeed the equivalent of a design document there are serious weaknesses in the overall logic, statement of objectives to be achieved and indicators that will measure their achievement. The absence of a logical framework (or equivalent), particularly with respect to the distinction between outputs and outcomes, and articulation of indicators and means of verification, may have almost certainly contributed to identified weaknesses with ongoing project monitoring and evaluation, discussed further in Section 3.7.
One finds that project terminology changes throughout quarterly reporting and AAPs, which only further contributes to confusion. The ET has, with assistance from the CHAI management, put together what is probably now the accepted Program Area objectives (see Annex 6). They still lack indicators and targets.
Appropriateness of management and institutional arrangements
A Funding Agreement was signed in July 2006 between the government of Australia and CHAI. This outlined the terms and conditions of the arrangement between AusAID and CHAI for the implementation of CHAI in PNG. This agreement states that “the parties agree to monitor the Program against the Program Milestones and evaluate it against the Program Outcomes.” The defining document against which this occurs is the Program Workplan 2006- 2009. As noted above there is no stated objective/outcome in this document (or any documents subsequent to this) for the whole project (and associated indicators). Outcomes or objectives for each of the AoC are not articulated (thus no indicators provided). It is therefore impossible for this part of the agreement to be adhered to. The elements of the project have also changed over time, and these changes have not been documented systematically. Had there been a monitoring matrix developed at the start of the project these problems could have been avoided. With such a document in place it would have been very easy to document changes so that donor and partners (and evaluators) were abreast of any changes to the project profile.
The Funding Agreement is further compromised by the description of the reporting requirements (Attachment B) which at no time specifies that the project must report systematically against all activities that are part of each of the AoC (or subsequent Program Areas), or on any indicators of outcomes (even though it states it must report on progress of the Program against the Annual Plan).
The evaluation is using the latest structure of the program and reporting on effectiveness of each of the Program Areas, including Program Management
“Effectiveness is a measure of the extent to which an aid activity attains its objectives”. As already stated the overarching goal/purpose/objective of the CHAI was never clearly articulated, and thus no indicators to measure its achievement have been presented in any project documentation. The Evaluation Team consulted with the CHAI team to arrive at possible indicators that might be used in the remaining six months of the program to indicate effectiveness of the whole program.
In the absence of a clear statement of objectives at the overall project level, and objective statements without indicators at the Program Area level it is not possible to say with any degree of confidence whether CHAI has been effective. The only reports on progress are the Quarterly Reports (QRs) (and minimal reporting in the AAPs) and these are limited to describing activities and outputs for each Program Area, and are not linked to indicators for the Program Area.
Reporting on this is compromised by the lack of clear and consistent program logic that has indicators attached to every level of the program. The overall objectives (not originally articulated) that are now stated for each Program Area have been arrived at after discussion between the ET and the CHAI Country Director.
CHAI Human Resources
Human resource issues have challenged the project from the start. There was unusually high staff turnover in the program during the initial two years 2006-2008, including some key positions. Between 2006 and the end of 2007, five of seven original staff transitioned out of the program (see Annex 9). Since the beginning of 2008, there has been program stability though a recent departure of the second Clinical Director has prompted CHAI (at the direction of Principal Technical Adviser, HIV/AIDS/STI, Communicable Diseases Branch and the Health Secretary) to recruit for this position from within PNG going forward. Recruitment of some members of the in-county team appears to have not been thoroughly explored, resulting in appointment of people not suited to the PNG environment, project management roles, and not acceptable to the GoPNG.
Monitoring and Evaluation (M&E)
This section of the report assesses whether the M&E framework effectively measures progress towards meeting project objectives. It therefore assumes that there is an M&E framework for the project with clearly stated objectives and indicators to use for measuring their achievement. This was not the case with CHAI.
Monitoring is a “continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds”.7
For CHAI monitoring is achieved through quarterly reporting and Annual Activity Plans. Whilst much of this is informative there is no systematic approach to reporting against each element of each Program Area. From one document to another the number of technical Program Areas varies from year to year, and even in the most recent AAP only five are listed (there are now six). AAPs sometimes list achievements of the previous year, and sometimes they do not. Key achievements of the previous year are never listed against each Program Area. This is key information that should then inform the planning for the next year.
Evaluation is the “systematic and objective assessment of an on-going or completed project ..., its design, implementation and results.” Evaluation is undertaken to determine the achievement of project objectives and outcomes, among other things.
Evaluation requires that objectives and outcomes are clearly stated at the overall project (goal) and component (Program Area) level, and indicators for measuring achievement of those goals and objectives are stated. There is no overarching project goal statement. In the 2006 workplan (the first definitive description of the project) no objectives were stated for each of the AoCs (later Program Areas). Only in the 2008 AAP do Program Area objectives become evident for some Program Areas, but these are interchangeably objectives for the duration of the project or for the year of that AAP. At this point in time no indicators are provided to measure their achievement.
Systematic M&E is integral to effective project management. For CHAI an M&E framework has never been developed, despite frequent requests by AusAID for one. The problems identified above would have almost certainly been overcome had there been a project monitoring matrix developed at inception. This could be a “living” document that evolves as new elements to the project are introduced in response to government needs.
The ultimate beneficiaries of this project are people living with HIV. There is no indication CHAI will measure the outcome of the project on them. There is a significant body of published literature on measuring the outcome of treatment and care programs for PLHIV using Quality of Life indicators. There are opportunities to collect “significant stories” (as distinct form using the “Significant Change” methodology) as evidence of the effect of this project on PLHIV, particularly to do with the Micro Credit Livelihood Enhancement and participation in activities such as those available through the Well Baby Centre. Literature is available on the measurement of stigma. To date CHAI does not appear to have considered any of these.
$11M of taxpayer money donated to the Clintons and treated like a gift for their pet organisation to do with what it wants.