The PiP Project engaged in an extensive and detailed phase of evaluation activity throughout the period Oct 2011 - July 2012.  The evaluation was designed to enable a better understanding of C-CAP’s impact within particular stakeholder groups and on approval process efficacy.  More generally the findings were intended to inform the HE sector about the technology-supported approaches that can be deployed to improve curriculum design and approval processes. A .zip file containing all the PiP evaluation outputs listed below is available (7MB).

Evaluation outputs

PiP evaluation activity generated four "strand reports", each relating to the evaluative strands as described in the evaluation plan.

1. Evaluation of systems pilot (WP7:37) - Heuristic Evaluation of Course and Class Approval Online Pilot (C-CAP)
2. Evaluation of systems pilot (WP7:37) - User acceptance testing of Course and Class Approval Online Pilot (C-CAP)
3. C-CAP impact and process evaluation (WP7:38) - Piloting of C-CAP: evaluation of impact and implications for system and process development
4. Evaluation of impact on business processes (WP7:39) - Critical analysis of BPI technique and C-CAP within class and course approval

Evaluation activity has also generated a final evaluation report.  This report attempts to synthesise the results from the strand reports while simultaneously providing a high level appraisal of PiP and C-CAP, the evaluation outputs, and provides summary conclusions and recommendations. 

• Final project evaluation synthesis (WP7:40)

The PiP Evaluation Plan sets out the principal phases of evaluation undertaken as part of the PiP project and fulfils workpackage 7, activity 17 (WP7:17).

• PiP Evaluation Plan

Additional evaluation outputs and resources

Throughout the Project the PiP Blog has been used to document and disseminate reflections on PiP Project developments.  This has been particularly true of the evaluation activity.  A number of lengthy contributions pertaining to the PiP evaluation have been posted on the Blog, some of which later influenced Project thinking on the evaluative approach. Additional contributions were made during the embedding phase of the Project (August 2012-April 2013).

• Revisiting user acceptance… or resistance? Insights from tech-supported curriculum design
• Understanding the social system when embedding tech-supported curriculum design and approval
• Thin end of the wedge: system resistance and its implications for C-CAP
• Academic quality: at the centre of the curriculum approval universe
• Process as myth: understanding the mythic core of organisational process with ideal types
• Two sides of the same coin: qualitative and quantitative approaches to process analysis
• Evaluating C-CAP: reflecting on the dichotomy of curriculum design and approval
• Evaluating PiP: optimising user acceptance testing via heuristic evaluation
• Reflecting on evaluation: JISC Curriculum Design Programme meeting

The Institutional Approaches to Curriculum Design Programme represents a unique example of funded innovation and development. Very little academic literature is therefore available to influence the evaluation of technology-supported approaches to curriculum design and approval. The PiP Project was ambitious in its desire to develop a programme of evaluation that was structured, rigorous and where possible replicable, i.e. can be deployed by other HE institutions that may wish to improve their curriculum design systems and approval processes with technology.  The rationale behind this evaluation approach is explained in the resources below, including a JISC webinar contribution, an accessible and generic evaluation plan, and a 20 min explanatory video.

• What was best about your evaluation? Principles in Patterns (PiP): evaluation approach. JISC Curriculum Design Webinar, 25 April 2012
• Evaluating technology-supported curriculum design and approval: suggested approach (A resource on the JISC Design Studio providing an overview of a general framework for evaluating technology-supported approaches to curriculum design and approval)