About

Program Review Procedure

Division: Institutional Effectiveness
SCC Policy Number: SPR4.27
OPR: Vice President of Institutional Effectiveness
Issued: April 2023
Revised:
Approved SCC President/CEO: May 2023
Reference KCTCS Policy:

Program Review Overview

Program review is a structured and holistic process of evaluating all academic, support, and administrative programs within the college to ensure that the programs and services offered at the college support and advance the mission of the college and meet the higher education needs of the communities in which we serve. Program review guides continuous improvement of programs and helps to make changes that support and contribute to the success of our students. Data, analyses, and insights associated with program reviews are used to evaluate the effectiveness and vitality of the program, prescribe changes to programs, and inform the annual planning within a program including development of action plans.

A program in the context of program review is a distinct planning unit within the organizational structure of the college and may include but are not limited to academic/technical/workforce programs, support/service programs, individual planning units, and/or organizational/administrative divisions.

Types of Program Reviews

Academic and support programs are evaluated on two distinct cycles each with specific processes associated with an annual review (minor cycle) or a multi-year, comprehensive review (major cycle) which occurs at least once every five years. Academic and support programs utilize similar formats for review but contain different evaluation elements based on the differences in the types of programs being reviewed. Program reviews utilize both quantitative and qualitative analyses. The program review processes contained herein are based on best practices in program review.

Roles and Responsibilities Associated with Program Review

The following describes the various roles and responsibilities associated with the program review process at SCC.

Leadership Council – approval of policies and procedures related to program review are evaluated and approved through the SCC Leadership Council. The Leadership Council is composed of representatives from the shared governance structure at SCC. Information provided for review to the Leadership Council is reviewed across the shared governance structure via representation to the Leadership Council from the Executive Council, Faculty Council, Management Council, Staff Council, and Student Government.
Administration – Vice President of Institutional Effectiveness manages the program review process and works closely with the Chief Academic Officer (CAO) and Vice Presidents (VPs) of college divisions to ensure that the program review process is implemented according to the applicable program review policies and procedures. The CAO develops a schedule for program reviews and works with the Dean, Associate Deans, Program Coordinators, and others to evaluate the programs, provide guidance, identify goals and objectives, develop action items, and to make program determinations. The VPs of divisions develop the schedule for program reviews within support areas and work with Program Staff and others to evaluate the programs, provide guidance, identify goals and objectives, develop action items, and to make program determinations.
Program Faculty (Academics) – Deans, Associate Deans, Program Coordinators, and program faculty are responsible for completing the program reviews for the Annual Program Review (APR) and Comprehensive Program Review (CPR) processes.
Program Staff (Support Programs) - Unit Planning Supervisors, Department/Unit Leads and support program staff are responsible for completing support programs reviews for the APR and the CPR processes.
External Program Review (EPR) Team - The EPR is comprised of the VP of Institutional Effectiveness, IR department staff, and a CAO-appointed Program Coordinator (from an unrelated program). The EPR reviews the program Self-Study and completes the EPR Report Form which summarizes findings and makes recommendations. Composition of the EPR team may be altered based on the specific programs being reviewed to ensure external review.
Program Review Assessment (PRA) Team – The PRA team is comprised of staff from the IR Department and representatives from the Institutional Effectiveness and Research Steering Team (ST). The PRA team assesses the program reviews (both Annual and Comprehensive) in terms of the overall quality and completeness and provides recommendations and guidance for improvement.
Institutional Research (IR) Team – provides data for program review, manages database for storing program reviews, provides data analysis, support, and training to those involved in the process of program review and provides coordination and oversight for program review process.
Institutional Effectiveness and Research Steering Team (ST) – The ST is comprised of representatives from across the college who provide guidance and advise related to matters associated with institutional effectiveness and research efforts. For the purposes of program review members of the ST are/can be included in the program review assessment process evaluating the quality of program reviews.

ANNUAL PROGRAM REVIEW (APR)

Minor Cycle – Annual Review

Process Overview and Chronology for Annual Program Review (APR)
Program reviews are conducted annually for all academic and support (non-academic, administrative, operational) programs. Annual program reviews (APRs) are completed by program coordinators, planning unit supervisors, and/or designees and in accordance with an annual program review cycle for Academic programs that begins in May and concludes in December and an annual program review cycle for support programs that begins in May and concludes in June. The office of Institutional Effectiveness and Research provides oversight and coordination of this process. Each Academic and Support program at the college is responsible for conducting an annual program review utilizing the following process:

The Institutional Research (IR) team activates the annual program review template in Compliance Assist, a database software utilized to store program review data, and notifies program coordinators/planning unit supervisors that the cycle for annual program review has opened. A schedule for completing the annual program review process is included with notification.
The IR team provides annual program review-specific training related to the types of data available and provides advice on how to use the data for analysis of the program. The IR team also provides training on how to utilize Compliance Assist to record the program review information and supporting documentation. Compliance Assist is accessible via the Institutional Research page on the SCC website. Program coordinators/planning unit supervisors are supplied a rubric to guide the annual program process. The rubric is intended to enhance the overall quality of the review and is utilized to provide guidance to program reviewers for each of the areas of program review.
The IR team assists program coordinators/planning unit supervisors by providing program related data. Common data sets are provided to all program coordinators utilizing Program Snapshot data, Student Success Continuum data, and other program specific data from KCTCS Decision Support System (DSS) and specific data requests from the program coordinators/planning unit supervisors.

Program coordinators/planning unit supervisors conduct the program review with input from program faculty and address the following categories (subject to change) and elements of annual program review:

Table 1 – Annual Program Review Elements
Category	Element
Academic Program Profile	Program Profile - overview information including mission
Key Indicator Trends	Description of measures and metrics that the program uses to assess its progress toward goals Key indicator trends affecting the program
SLO Assessment	Evidence of Student Learning/Support Outcomes Performance and success of the program
Resource Impact Summary	Evaluation of program resources, resource uses, and resource needs (Basis for Resource Requests)
Program Cost Analysis	Analysis of program cost/expenditures
SWOT Analysis	Changes impacting the program Program Strengths Program Weaknesses Program Opportunities Program Threats Challenges and Issues associated with the program Adjustments/Modifications needed in program
Overall Program Review Summary	Status of the program Program goals and objectives Program planning considerations – new goals, objectives, initiatives Adjustments/Modifications needed in program Opportunities for improvement (Basis for Program Planning)
Action Plan	Action plan for next year (goals, objectives, resources)

Program Coordinators/Unit planners complete the program review by entering the program review narratives and data into Compliance Assist by the scheduled due date.
A Program Review Assessment (PRA) team is assembled to assess the program reviews. The PRA team is comprised of members of the Institutional Effectiveness and Research Steering team (ST) team and the IR team. The PRA team utilizes a program review rubric to assess the quality and completeness of the review and provides feedback and recommendations and assigns an overall quality rating to the review utilizing a PRA feedback form. The timeline for the PRA Assessment process is included in Table 2.
The PRA feedback form is provided to the Program Coordinator, Associate Dean, Dean for academic programs or to the unit planner and their supervisor for support programs. The VP of the person completing the program review will review the PRA feedback form and respond with feedback to the person who completed the program review within 30 days of receipt of the PRA feedback form. Program reviews not achieving an acceptable rating are referred to the program coordinator for revision and resubmission. The program reviewer has 30 days to make the revisions. The IR team monitors the revisions and indicates status within Compliance Assist and communicates with the Division Chair/Unit Supervisor regarding status. Deans/Unit Leads/Supervisors are provided a summary indicating the status of the program reviews within their divisions/departments/areas.
A summary report indicating the overall status of the annual program reviews is provided to the Chief Academic Officer (CAO) and to Vice Presidents of support divisions once the annual program review period is complete.

Program Review, Unit Planning, and Resource Requests
The annual program review process results in program planning and resource requests for the following year. Program reviews are intended to inform unit planners and program coordinators of the vitality of their respective programs and provide insight into the modifications and adjustments that are needed including new and/or continuing resources. Goals and objectives for the next year are identified during this review and help to inform the formal resource request process that occurs during the early spring in advance of the College budget development cycle.
Unit planning (UP) associated with the annual program review is conducted once the program review period is complete with suggested modification and/or additional action as identified by the APR process, action plan, and information from previous APR cycles and reviews. Unit plan (UP) development occurs from December to March each year.

Divisional Review of Programs
At the completion of the annual program review divisional Vice Presidents (VPs) review all their respective program reviews through a process called the Division Program Review (DPR). The DPR process is intended to provide an opportunity for divisions to analyze the status and vitality of their programs and to identify issues, trends, opportunities, and challenges in summary form. This review is based on the SWOT situational analysis model and involves evaluating the overall strengths, weaknesses, opportunities, and threats identified through the annual review of programs. This review also provides an opportunity for division leadership to make divisional plans, set goals and objectives, and evaluate and prioritize the resource needs of the division. This review is recorded in the DPR section in Compliance Assist. Divisions/Units share their analysis and review with the College President and the SCC Executive Council. Divisional reviews for Academic Programs are completed by the division VP by January 31 and divisional reviews for Support Programs are completed by the division VP by August 31. Feedback to program reviewers within each division is provided within 30 days of these deadlines. The timeline for DPR is provided in Table 2 below.

Table 2 – Timeline for Annual Program Review and Assessment
Program Type	Annual Program Review	Divisional Review	Assessment
Academic Programs	May–December	December–January	February–April
Support Programs	May–June	July–August	September–October

Comprehensive Program Review (CPR)

Major Cycle - 5 Year Review

Process Overview and Chronology for Comprehensive Program Review (CPR)
All programs at SCC are to be reviewed periodically via Comprehensive Program Review (CPR). Each Academic and Support Program at the college participates in CPR process at least once every five years. The CPR is a separate, more detailed review and is conducted in addition to the annual performance review process. The office of Institutional Effectiveness and Research provides oversight and coordination of this process. The following outlines the CPR process.
1. The CAO develops and distributes a five-year preliminary schedule to Deans, Associate Deans, and Program Coordinators. This schedule lists the programs and their planned review cycle. The number of academic programs reviewed per year will be determined by dividing the total number of academic programs by five (number of years in review cycle). The number and type of academic programs reviewed in any review cycle may change at the discretion of the CAO.
  The support program review schedule is distributed by the OIER to divisional VPs. The number of support programs reviewed per year will be determined by dividing the total number of support programs by five (number of years). The number and type of support programs reviewed may change at the discretion of the OIER staff.
2. Programs Coordinators/Planning Unit Supervisors review the proposed schedule and make suggestions based on other programs reviews that may be scheduled in the program area (CPE Program Review, Individual program accreditation cycle). Program Coordinators provide input to CAO for academic programs and Planning Unit Supervisors provide input to the appropriate VP regarding scheduling of support program reviews.
3. Final CPR schedule developed by the CAO/VPs and distributed to Deans, Associate Deans, Program Coordinators (Academics) and to the Planning Unit Supervisor (Support), and to the VP of Institutional Effectiveness for implementation.
4. Training provided to Deans, Associate Deans, Program Coordinators, and Program Faculty and Planning Unit Supervisors/Staff (Support Areas) for conducting self-study and the overall program review process. CPR template is provided and reviewed.
5. Data needed for program reviews are identified by Institutional Research (IR) in collaboration with Program Coordinators/Planning Unit Supervisors and timeline developed for provision. Data developed, compiled, and provided into the CPR template.
6. Programs under review will meet with IR team to review CPR milestones and schedule for completion established.
7. Program Coordinators/Program Faculty (Academics) and Unit Planners/Supervisors (Support) will meet with their respective areas to develop a review schedule with key milestones, review draft data for program review.
8. Program Coordinators and Program Faculty conduct a self-study and review of their respective program by evaluating and responding to all required components in the CPR template, develop preliminary recommendations and draft an action plan for improvement that includes specific goals and objectives for program development and improvement.
9. Program Coordinators and Program Faculty review and approve the self-study findings included in the CPR template. Planning Unit Supervisors/program staff approve the self-study findings included in the CPR template for support areas.
10. Completed and approved CPR template submitted to the External Review Team (ERT).
11. ERT meets to review CPR template and completes the ERT Report Form which summarizes findings, makes recommendations. ERT submits the CPR template and the ERT Report Form along with any other pertinent information to the appropriate Program Coordinator and Chair (Academics) or to the Unit Planning Supervisor/VP for review and comment.
12. Program Coordinator/Chair (academics) or Unit Planning Supervisor (support) reviews the findings of the ERT and then completes the CPR Response Form. The CPR Response Form is then submitted along with the completed CPR template, the ERT Report Form to the appropriate Dean (academics) or VP (support) for review.
13. Dean reviews CPR Template, ERT Report Form, Program Response Form and makes program recommendations and suggestions to Chief Academic Officer (CAO) utilizing the Program Review Status (PRS) form. The Dean in coordination with the Program Coordinator/Chair develop draft action plan for program improvement. Dean submits the PRS form and draft Action Plan to the CAO.
  For support areas, the VP of the support area reviews CPR Self Study, ERT Report Form, Program Response Form and makes program recommendations and suggestions to the Program/Planning Unit Supervisor utilizing the Program Review Status (PRS) Form. VP in coordination with the Program/Planning Unit Supervisor develop draft action plan for program improvement.
14. CAO meets with appropriate Dean, Associate Dean, and Program Coordinator to review CPR template, ERT Report, Program Response Form, Program Review Status form and reviews draft action plan. CAO provides guidance, feedback, and suggestions for the action plan, establishes goals and objectives for program, identifies resources needed, description of assessment measures, expectations of program and a timeline for completion and review.
15. CAO completes the Program Review Summary Form which includes a summary of the key findings and recommendations, identifies Action Plan goals and objectives, and classifies the program according to the following classifications:
  - Grow (a program with high demand and/or an area of emerging growth)
  - Sustain (performing well and the outlook is positive)
  - Fix (previously strong programs that can be turned around)
  - Sunset (small or specialized programs with limited growth potential)
  For support programs, the appropriate VP completes the Program Review Summary Form to include a summary of key findings, recommendations and an Action Plan with goals and objectives.
16. Program Review Summary Form is reviewed with the appropriate Dean, Chair, and/or Program Coordinator (academic programs) or the Unit Planning Supervisor (support programs) to discuss and acknowledge the findings, recommendations, and/or status of the review and to provide feedback/modifications/edits/comments regarding the program review. Based on this review, a timeline for update and review of progress towards the Action Plan the goals and objectives identified.
17. CAO (Academic Programs) and VP (Support Programs) provides summary of program review status to Executive and Leadership Councils. All documents associated with program review are archived in Compliance Assist.
18. Dean/Associate Dean (Academic Programs) or Unit Planning Supervisor (Support Programs) will submit progress report of action plan (if prescribed) to CAO/VP based on timeline established.

Timeline for Comprehensive Program Review and Assessment
The Comprehensive Program Review process will take place during the fall and spring semesters with assessment of submitted reviews being completed during the summer months as indicated in Table 3. Each year the CPR process reviews 7-8 programs during the Fall and Spring semesters. Each summer the IR team will assess the quality of the CPR process and make any suggestions and/or modifications to the current and/or next cycle of CPRs.

Table 3 - Comprehensive Program Review Timeline
Program Type	Comprehensive Program Review	Assessment
Academic Programs	August–May	June–July
Support Programs	January–May	June–July

Alignment with External Program Reviews
Programs that are reviewed by external or program-specific accreditation bodies may coordinate the schedule of their CPR with the CAO to align program review efforts and to avoid unnecessary duplication of review (CPE Program Review, Program Accreditation review, etc.). Analysis of program reviews associated with external accreditors or other external program review requirements will identify areas in the SCC CPR that may be omitted/modified and those areas that will be required. The intent of the CPR is to provide a comprehensive review which should contain all or most of the elements associated with any external reviews. External program reviews may be utilized in lieu of the local APR process with approval from the CAO.
Program Review Training
All employees with program review responsibility are required to complete annual training associated with conducting high-quality program reviews. Training is provided by the SCC IR team in conjunction with the SCC Teaching and Learning Center (TLC) and is geared to support program reviewers both before, during, and after the program review process. Each phase of training focuses on specific aspects of the review process and is modified based on training needs identified by the APR team. Training is provided in three distinct phases: Pre-Review, Review, Post-Review with each session providing pertinent information and guidance based on the phase and type of review taking place.
- Pre-Review Phase. Pre-Review training focuses on collecting the appropriate and relevant data and information necessary to complete the review. This session includes details surrounding the use of Compliance Assist, types and sources of program data, overview of different data collections, and timelines for completion. Training outcome – understanding how to best prepare for program review.
- Review Phase. The Review training focuses on the actual program review processes and is led by the IR team. These sessions focus on best practices in writing the narrative and utilization of data as part of the review. Training during the review process focuses on capturing the big picture of the program, identifying trends, exploring challenges and strengths to accurately portray the status of the program. This session also includes one-on-one assistance to program reviewers to assist with questions and to share ideas and suggestions. Training outcome – understanding how to complete an accurate and thoughtful program review.
- Post-Review Phase. Post-Review training focuses on interpreting feedback from reviewers and to correct common errors. Post-review training involves looking at the suggestions for improvement and incorporating into the program review document. This training also includes guidance on how to develop unit plans based on the program review. Training outcome – understanding how to improve program reviews and identifying next steps.
Evaluating the Quality of Program Review
Program review rubrics are utilized to evaluate the quality of program reviews both for the annual program review and the comprehensive program review processes. Rubrics help to guide the program reviewer by identifying key components and considerations for inclusion in the review. Review/Assessment teams utilize the same rubrics to guide the evaluation and assessment of the program reviews and provide and method for structuring feedback and recommendations. Rubrics are scored and returned to the appropriate program coordinator and classified as either complete or incomplete. Complete reviews are ready for publishing and implementation. Incomplete reviews require additional information, development, and/or modification. If reviews are determined to be incomplete, the program reviewer can resubmit with revisions and the review team will reassess the program review.
Periodic Review and Assessment of the APR and CPR Process
The policy and procedures associated with program review are reviewed annually. The Institutional Effectiveness and Research Steering team reviews and modifies the policy and procedures as necessary with input from the IR team and by evaluating the program reviews conducted throughout the year. Program review criteria and measures are also reviewed annually to ensure alignment with other applicable and/or related program review guidelines including, but not limited to, the Kentucky Council on Postsecondary Education and SACSCOC requirements for effective program review. Additions/modifications may need to occur throughout the year regarding the information reviewed in the CPR and APR. Such modifications will be incorporated as needed and communicated.
- Publication of Program Review – Once programs are classified as complete, they are electronically stored in Compliance Assist. Program Reviews including self-studies, external review reports and program action plans are available to all SCC faculty and staff via the Institutional Effectiveness and Research site on the SCC Intranet.