UCSF Organizational Overview:

UCSF Global Health Sciences (GHS) program is uniquely positioned to provide the technical assistance (TA), support, and guidance for the proposed project. GHS has a long history and successful track record of providing training, mentoring and support in all aspects HIV/AIDS M&E, surveillance, strategic information and planning, data use, education, and training. GHS's Prevention and Public Health Group (formerly the Institute for Global Health) includes over 70 affiliated faculty and senior staff members who carry out a variety of interdisciplinary research and training activities directed at diseases of public health significance in the resource-constrained world. GHS is recognized internationally for its faculty's expertise in the field of evidence-based guideline and program development and policy; it houses the Cochrane Collaboration's HIV/AIDS Group. Building on its many years of experience in the field, GHS recognizes that, when it comes to putting new knowledge and skills into practice, on-site mentoring is one of the most effective and flexible approaches to training. This is because on-site mentoring promotes an atmosphere of trust and respect between mentors and mentees, it gives mentors a chance to better appreciate the systemic context in which mentees operate, and it allows mentees to establish a long-term relationship with their mentors that in turn gives them a channel to address their specific needs and to ask for advice and help beyond the formal mentoring period.

GHS has core expertise in providing training, TA, mentoring, and capacity building to in-country partners for their M&E needs - such as improving the collection, management, and use of HIV prevention, care, and treatment program data, including PEPFAR indicators and surveillance data - and in improving data use at the regional, provincial and national level. This includes organization and presentation of data and data analysis across time to examine trends. GHS also has considerable experience in providing guidance and services to develop, implement, and test evaluation information systems, including the effective use of data and strategic information (SI) for program planning and improvement. Specifically, we have assisted sites to develop data collection strategies and have trained staff to clean study data, ensuring high quality datasets through local or remote database infrastructure. We have helped sites generate reports and translate results into SI. We have helped program staff use data from key indicators that are collected routinely for program evaluation and improvement; assessed program needs for capacity in M&E; and established flexible solutions for collecting, managing, analyzing, and providing information to country programs. GHS has experience in implementing the Global Fund's M&E Strengthening Tool and in conducting assessments of M&E capacity and systems at all levels: programmatic, sub-national, and national.

Goal: To empower CDC IPs with knowledge, skills and systems to efficiently manage, use, harmonize and disseminate M&E data both internally and externally and to build national capacity to train and mentor IPs for M&E.

Objectives:

1. To assess the general capacity for IP data management among CDC IPs (year 1)

2. To improve capacity among 36 selected CDC-IPs to collect, manage, analyze, interpret, use and report program data by the end of the project period

3. To develop an M&E training & mentoring program adapted to IP needs

4. To improve harmonization, accuracy, and timeliness of indicator reporting for selected IPs by end of each training cycle

5. To improve capacity of IPs to conduct outcome evaluations by the end of project period

6. To assist CDC Uganda and selected IPs to conduct evaluations of HIV care & treatment programs

Geographic Coverage and Target Populations:

-National coverage

-Target population are the implementing partners and all those to whom they provide services

UCSF will work with MUSPH and MOH to create a network of M&E experts. This network of experts will constitute the seed of a Centre of Excellence in M&E that will be organized and coordinated by the MUSPH team of M&E officers with continuous support from UCSF and will constitute the main exit strategy of this project. This will include establishing satellite training sites to cover the four major regions of the country. UCSF will contribute directly to this Centre through the training of M&E specialists, trainers, and mentors as mentioned above, and will provide additional support in the form of personnel, equipment, and other supplies necessary to establish functional M&E systems within IPs in Uganda. In this way, when fully functional by the end of the project period, the MUSPH Centre of Excellence will have the capacity to provide on-going support to the IPs mentored by this project, to train and mentor additional PEPFAR and non-PEPFAR IPs, and to help establish the MUSPH-planned academic track in M&E within the MPH program at MUSPH.

Monitoring and Evaluation Plan

UCSF will be responsible for organizing the monitoring and evaluation of the activities proposed in this application. A framework summarizing the detailed M&E plan and related indicators has been created. The monitoring and evaluation plan for the proposed activities will track the processes, outcomes and impacts of interventions on the beneficiaries and will be integrated in the program of activities in each annual cycle. The results monitoring will attempt to examine the causal linkages between the inputs, outputs and the results. Tracking will be conducted through several different methods. First, evaluations of workshops and training sessions will be conducted using an on-line system for assessing knowledge gained, participant satisfaction, and participant activities post-training. Second, meta-evaluations will be conducted to assess the success of the outcome evaluations that selected IPs will be doing throughout the project period. This will include an assessment of the protocol-writing process and non-research determination application process. Third, assessments of indicators being collected, data quality, reporting and analysis of programmatic data will be conducted with each participating IP. Reports and data use within IPs will be compared post-training to pre-training. Finally, post-intervention reviews will be conducted with each IP to ensure that harmonization of indicators across partners has been successful. The results of our project evaluation will establish whether the overall M&E capacity building strategies implemented both at the IP level and at the national level are successful in raising the level of expertise in M&E for each IP and in increasing the numbers of M&E mentors and experts in Uganda.

The evaluation of the proposed activities will include tracking the utilization of inputs (resources) in the production of outputs including processes/activities. The main tools of analysis will be the annual budget and work plans, and periodic progress report. Tracking will be done though administrative data recording, both narrative and financial progress reporting, spot checks/follow-up and financial audits. The outcome of implementation monitoring will determine efficiency in the utilization of resources and whether the set targets were met.

Results will be disseminated to IPs and key partners at the end of each cycle and at the end of the project nationally. The final evaluation will be distributed to all stakeholders.

The UCSF-MUSPH team will evaluate the M&E capacity and needs of all CDC Care and Treatment IPs, including MOH, and will train and mentor every year a selected number of IPs with different areas of programmatic focus to implement a comprehensive project tailored to the specific needs of each IP. This will be accomplished through a combination of formal short-course training and on-site mentoring that will aim to ensure that beneficiaries not only acquire M&E knowledge but fully understand how to apply this knowledge to their programs and develop mentoring skills of their own in the process. This will equip a cadre of staff with knowledge and skills that will enable them to address identified M&E gaps, to ensure timely and accurate M&E reporting, and to develop and implement program-specific outcome evaluations and related protocols in compliance with ethical standards and protection of human subjects.

UCSF will assist selected care and treatment IPs to conduct evaluations of their care and treatment programs, with the goal of performing at least one ART outcomes project per year of the overall project. Clinical outcomes will include mortality, ART initiation, average CD4 counts, drug regimens in use, loss to follow up.

UCSF will be assessing systems in place in care and treatment clinics for tracking adherence and loss to follow-up.

Over the course of the 5-year project UCSF plans to assess the monitoring systems in place at all CDC-supported care and treatment clinics in Uganda. We will not be providing treatment or clinical services, but rather will be conducting assessments, trainings, and making recommendations for improvements to existing systems.

Key Strategies of the SI Component of the Project:

1. To work in close consultation with key stakeholders in planning and implementing the different phases of the M&E capacity building project. In order to ensure that efforts are complementary, coordinated, and supportive of the national HMIS, the project will establish strong partnerships with key players during the planning and implementation phase of the project, including MEEPP, the PEPFAR SI TWG, UAC, CDC-Uganda Informatics and Program Units, the HMIS TWG, and the MOH M&E Section. The project will also support one or more staff positions within NACP to help coordinate the national M&E harmonization effort, to provide increased capacity and expertise within MOH, and to be involved with the IP training and mentoring program.

2. To assist, train, and mentor IPs in improving all aspects of their M&E capacity and systems in partnership with MUSPH. The UCSF-MUSPH team will evaluate the M&E capacity and needs of all CDC IPs, including MOH, and will train and mentor every year a selected number of IPs with different areas of programmatic focus to implement a comprehensive project tailored to the specific needs of each IP. This will be done through a combination of formal short-course training and on-site mentoring that will aim to ensure that beneficiaries not only acquire M&E knowledge but fully understand how to apply this knowledge to their programs and develop mentoring skills of their own in the process. This will equip a cadre of staff with knowledge and skills that will enable them to address identified M&E gaps, to ensure timely and accurate M&E reporting, and to develop and implement program-specific outcome evaluations and related protocols in compliance with ethical standards and protection of human subjects. By combining UCSF's international M&E experience and training skills with MUSPH's role and expertise in HIV/AIDS research, education, and capacity building in Uganda, we will maximize the chances of success of this project and the potential for its continued growth beyond UCSF assistance.

3. In an effort to provide support to national capacity building relating to collection, management, analysis and use of data, UCSF will work with collaborating institutions to build a Centre of Excellence in M&E at MUSPH. This Centre of Excellence will provide training and academic leadership in M&E and will further support and strengthen the M&E capacity of USG IPs, the Uganda MOH, as well as of other governmental organizations and NGOs in Uganda. By building a cadre of staff and professionals with the academic depth and capacity to provide training and mentorship in all aspects of M&E at the governmental and IP levels, the project will contribute to MUSPH's plans to develop a comprehensive curriculum on M&E with both formal degree-granting and short-course offerings for NGOs and professionals in Uganda. This strategy will promote the development of capacity for robust and sustainable M&E in Uganda in the long term.

Training The MOH, UCSF-MUSPH Team and CDC-Uganda will come together to develop three levels of M&E and Mentoring courses for PEPFAR IPs. These courses will aim to provide participants with a practical understanding of M&E and mentoring concepts and methods, and how to integrate them into program planning, implementation, and management. Emphasis will be placed on imparting knowledge and skills that will allow participants to use M&E to generate meaningful program data for supporting decision making within the context of Uganda's national HIV/AIDS strategy. The novel use of mentoring using an M & E project tailored to the particular needs of each IP in addition to exercises, small group work, large group discussions, and facilitator-led presentations, will enable participants to gain hands-on experience in M&E that will be applicable in their own work. The course's exercises will reinforce concepts covered within a course module. Small group work and large group discussions will provide participants with opportunities to share their various perspectives on the concepts covered and clarify their level of understanding. This training and mentorship will build the selected IPs' capacity in M&E data management, analysis, and reporting and help IPs define a harmonized set of indicators relevant to their areas of focus (prevention, care & treatment, HCT, PMTCT, OVCs, etc.) with guidance and support from the SI TWG and under the coordination of the NACP Harmonization Coordinator.

Trainings: Monitoring, Evaluation and Mentoring -I

Description The Monitoring and Evaluation (M&E) Team of MOH, UCSF-GHS, MUSPH and CDC-Uganda have come together to develop 3 levels of Monitoring, Evaluation and Mentoring. This first course is designed to provide participants with a practical understanding of M&E concepts, methods and how to integrate them into program planning, implementation, and management. The course emphasizes the development of useful knowledge and skills that will allow participants to use M&E to generate meaningful program data for supporting decision making within the context of a country's national HIV/AIDS strategy.

The novel use of mentoring an individual project in addition to exercises, small group work, large group discussions, and facilitator-led presentations, will enable participants to gain hands-on experience in M&E, which can be applied in their own work. The course's exercises introduce content or reinforce concepts covered within a course module. Small group work and large group discussions provide participants with opportunities to share their various perspectives on the concepts covered and clarify their level of understanding. Participants are given tools for planning program-specific M&E and assessing M&E readiness, which can provide a starting point for developing M&E action plans.

Trainings: Monitoring, Evaluation and Mentoring -II

Description The second course is designed to provide participants with a deeper understanding of data quality and how to assess it, how to get there, and innovative ways of finding data sources and data use. The course builds on to what was covered in M&E&M-I but goes in more depth. The course emphasizes the development of relevant knowledge and skills that will allow participants to use M&E to generate meaningful program data for supporting decision making within the context of a country's national HIV/AIDS strategy.

The novel use of mentoring as well as individual projects, exercises, small group work, large group discussions, and facilitator-led presentations, will enable participants to gain hands-on experience in M&E, which can be applied in their own work. The course's exercises introduce content or reinforce concepts covered within a course module. Small group work and large group discussions provide participants with opportunities to share their various perspectives on the concepts covered and clarify their level of understanding. Participants are given tools for planning program-specific M&E and assessing M&E readiness, which can provide a starting point for developing M&E action plans and their individual M&E projects.

Training: Monitoring, Evaluation and Mentoring - III:

Description The third course is designed to provide participants with a deeper understanding evaluation theory, and designing and implementing evaluation studies. This course will cover cost effectiveness, impact evaluations, data triangulation, and advances in the field of evaluation studies.

Implementation of Activities:

Building and sustaining the M&E capacity of PEPFAR IPs will be the core focus of this project. To achieve this in a relevant and efficient manner, UCSF will listen to and work with key M&E stakeholders in Uganda, support the creation of a new M&E Harmonization Coordinator position at NACP, and team up with MUSPH. The main operational strategy will be to review the M&E capacity of successive groups of IPs with diverse programmatic specificities, geographical locations and capacities, and to support them to address their needs and harmonize their indicators in a way that integrates training with day-to-day responsibilities. We will implement this strategy by working with a limited number of IPs every year, including centrally located IPs with relatively developed M&E capacity as well as more peripheral IPs working in underserved rural areas with limited M&E capacity. We will conduct in-depth reviews of their M&E capacity and systems, and addressing their needs through a combination of instruction and hands-on design and implementation of a comprehensive M&E or outcome evaluation project related to their existing programs. This exercise will also serve as a platform to generate or improve electronic M&E data capture and reporting systems for each main programmatic area. After completing their project, the focal M&E persons from each IP will be involved in supporting (on a part-time basis) a new group of IPs selected in subsequent years as a way of honing their training and mentoring skills.

This project will be implemented in close consultation with CDC, MOH, UAC and the TWGs on SI and Care and Treatment. In addition, UCSF will team up with MUSPH to combine their technical and programmatic strengths. The operational team will be comprised of a UCSF in-country project coordinator and three M&E trainers/mentors that will be identified and/or hired through MUSPH in year 1 and subsequently trained by UCSF, using a curriculum customized to their experience and needs. The team will work in close collaboration with CDC-Uganda and the MOH NACP Harmonization Coordinator to identify IPs and their training and mentoring needs. The in-country team will have continuous technical support from UCSF experts in M&E, information technology (IT), surveillance, HMIS, and data analysis.

A general assessment of CDC IPs' M&E capacity will be done by the UCSF-MUSPH team in collaboration with CDC, MOH, and MEEPP to inform the adaptation of training and mentoring curricula in M&E at three different levels. Six IPs with national coverage and various programmatic specificities will then be selected in year 1 with input from MOH and CDC and will be asked to designate an M&E focal person (FP) and a back-up to work closely with the UCSF-MUSPH team and identify their own M&E gaps and needs (in staffing, data collection, management, analysis, and reporting). The UCSF-MUSPH team will train the IP focal persons using the relevant M&E curricula and will mentor them to evaluate in-depth their own organization's capacity for data collection, management, analysis and reporting.

The results of this evaluation will be used to design an on-the-job, hands-on comprehensive M&E project (for all IPs) and an outcome evaluation (for one of the six IPs) focusing on the specific gaps and priorities of each IP. This project will be planned over a 12-month period during which the IP focal persons will be regularly trained, supported and mentored by the UCSF-MUSPH team. The MOH, UCSF-MUSPH Team and CDC-Uganda will come together to develop three levels of M&E and Mentoring courses for PEPFAR IPs. These courses will aim to provide participants with a practical understanding of M&E and mentoring concepts and methods, and how to integrate them into program planning, implementation, and management. Emphasis will be placed on imparting knowledge and skills that will allow participants to use M&E to generate meaningful program data for supporting decision making within the context of Uganda's national HIV/AIDS strategy. The novel use of mentoring using an M&E project tailored to the particular needs of each IP in addition to exercises, small group work, large group discussions, and facilitator-led presentations, will enable participants to gain hands-on experience in M&E that will be applicable in their own work. The course's exercises will reinforce concepts covered within a course module. Small group work and large group discussions will provide participants with opportunities to share their various perspectives on the concepts covered and clarify their level of understanding. This training and mentorship will build the selected IPs' capacity in M&E data management, analysis, and reporting and help IPs define a harmonized set of indicators relevant to their areas of focus (prevention, care & treatment, HCT, PMTCT, OVCs, etc.) with guidance and support from the SI TWG and under the coordination of the NACP Harmonization Coordinator.

The proposed activities will support the broader technical program areas for monitoring, evaluation and for information systems. The proposed assessment and subsequent improvement of M&E capacity of staff, the assessment of information systems in use for monitoring service uptake, process and clinical outcomes, and the harmonization of indicators will cut across all HIV program areas and include information systems in use for all types of service delivery. Implementing partners working in each of the program areas will be included in M&E and systems improvement activities for year 1, as will MOH staff.

This activity will address gaps in systems in place for monitoring and evaluation of activities supported by implementing partners working in Uganda across all HIV service areas.

UCSF in partnership with CDC, MUSPH and MOH, will conduct assessments of current systems in place, make recommendations for improvements, and provide technical assistance in implementing these improvements. The overall goal is to harmonize indicators across implementing partners and in line with those set out in the national level M&E plan.

The plans to strengthen strategic information systems across all HIV services areas and implementing partners by definition links functional areas and will have intentional spillover, especially as regards training Ugandan nationals both at MOH and MUSPH as M&E experts and mentors who will work closely with IPs as they improve their own M&E systems in use in their service areas.