Programme Review - Guidenotes and Forms
Evaluation, Consultation and Planning within Programme Review
These guidenotes provide advice on activities to be carried out to inform the programme review.
Documentation submitted to Programme Review Boards
These guidenotes describe the standard University format for the Programme Evaluation Summary and the Programme Development Plan, which are prescribed elements of the Programme Review documentation. University policy allows alternative formats to be used in the case of a joint University / PSB review. If alternative formats are potentially necessary, this should be discussed beforehand with the college Head of Quality Assurance (or equivalent) or the Director of Quality Assurance. Guidance notes are also provided regarding other supporting material. These are submitted or made accessible three weeks in advance of the Programme Review Board meetings, but a single set should also be made available for reference by the Review Board at the time of the event.
Evaluation, Consultation and Planning within Programme Review: Guidenote on Programme Evaluation
The aim of the evaluation process during programme review is to judge the effectiveness of the programme(s) considered and explore ideas for future improvement.
This guidenote suggests an approach to programme evaluation focussing on three areas:
- past performance and standards
- developments and good practice in the subject and in teaching
- management of quality and standards.
Specific guidance is provided below, including suggestions on how to:
- plan and organise the evaluation processes
- identify and collect relevant information
- communicate and document the process.
This guidance note comprises a set of broad self-assessment questions and possible sources of relevant information.
The document Considerations of Good Practice for Taught Programmes should guide the programme evaluation and it should also be informed by appropriate consultation with University support services and students and graduates.
The end point of this reflective process is the production of a Programme Evaluation Summary which in turn is used as a source for the Programme Development Plan. These two documents are submitted to the Review Board.
Planning and organising the programme evaluation process
The school dean (or school deans) should allocate responsibility for the evaluation process among the Programme Team and administration staff. The overall process should be as inclusive as possible, involving discussions at school board and with student representatives, as deemed appropriate. In some cases external advisors may be involved (e.g. a past external examiner or industrial board member).
It can be helpful to summarise the evaluation in a set of reports or commentaries that identify key points and issues to be considered by the Programme Team, for example:
- answers to specific questions
- specific needs or opportunities for improvements
- ideas, examples of good practice.
These internal, working documents need not be submitted to the Programme Review Board, but might be cited as references in the Programme Evaluation Summary and might then be made available as supporting material.
It may be helpful to share evaluation tasks by allocating responsibility for different aspects. Some schools have successfully employed a dedicated My Dundee module as a forum for lodging and discussing draft documentation (consult the Learning Centre for further advice).
Identifying and collecting relevant information
Past performance and standards
In considering aspects of the operation of the programme, the programme team may wish to consider trends and issues in the following:
Function of component modules - for example, are any modules particularly successful or problematic? Annual Module and Programme Monitoring reports, should contain summaries of most, if not all of the relevant information.
Student recruitment data - trends in student number(s), qualifications, and characteristics.
Retention, progression and continuation - these issues may be complex at programme level, for example in the case of college or school level entry that enables students to exercise choice between a number of programme pathways.
Final award - including consideration of the pattern of final awards (for example, honours classification bands) and also consideration of the overall performance of cohorts of students from admission/intake.
Specific groups or types of student - trends or significant issues regarding specific groups or types of students, for example:
- students admitted with advanced standing, including any admitted under 'Articulation' agreements
- students admitted via wider access programmes
- students with disability or special needs
- part-time students.
Graduate destinations - this should include consideration of graduate first destination statistics (available from the Careers Service) and also any graduate-related feedback from other sources.
Feedback and observations
From students:
- summaries of questionnaire data, including the National Student Survey, where available
- minutes of Staff-Student Liaison committee(s).
From External Examiners:
- External Examiner Reports
- minutes of Examination Board meetings
- comments on exam. papers and/or coursework assignments.
From staff:
- notes and minutes of discussions at Programme/School Board, any sub-committees on teaching, any 'away-days', etc.
Distinctive or innovative teaching practice - approaches to learning and teaching within the programme that might inform or enhance the way other teaching is carried out, for example:
- known good practice, for example that highlighted by student feedback or within annual monitoring reports
- innovation recognised through the University's teaching awards
- research or scholarship projects in teaching
- staff publications related to pedagogy in the discipline
- Special engagement with QAA Quality Enhancement Themes or Higher Education Academy subject centres.
Reports from any internal or external reviews - typically over the past 5 years:
- the previous programme review (or papers related to the introduction of the programme)
- external reviews, such as accreditation reports by professional bodies.
University and College learning and teaching strategies and school learning and teaching development plan(s) - how might these impact on the future of the programme?
Recent changes - developments actually or potentially influencing the programme; for example, changes in staffing, facilities, funding.
- Developments and good practice in the subject and in teaching
In considering whether the programme is up-to-date in terms of developments in the discipline and in methods of teaching and assessment, the Programme Team may wish to address the following questions:
- What is 'state-of-the-art' in teaching this subject in higher education?
- What can we learn from the 'leading-edge' centres of excellence in teaching this subject?
- What is best practice relevant to teaching the types of students admitted?
- How can accessibility to this Programme be improved for students with disability?
- How can achievement by students with disability on this Programme be improved?
The following UK and international sources may be relevant:
Discipline-related:
- subject groups - for example, associations of Heads-of-Department (in the discipline)
- subject journals, books, conferences
- websites of other universities
- Staff members with experience of acting as external examiners elsewhere
- Professional and Statutory Bodies (where relevant)
- HEA subject centres
- QA reports - both subject overview reports and reports from highly-rated departments.
General developments in teaching
- exemplars of good practice elsewhere in the University
- resources published by the Higher Education Academy, for example in HEA subject centre websites
- The QAAS Quality Enhancement Themes
- books, guides, journals on teaching
- specifically regarding promoting accessibility and achievement for students with disability:
- the University's Disability Support Centre
- QAA Code of Practice, Section 3 -Students with Disability
- Teachability project: this provides guidance and a methodology to audit accessibility.
Consideration of Equality and Diversity in the Curriculum
The programme team should complete the inclusive curriculum checklist and submit this as Appendix 6 of the programme evaluation summary. The programme development plan should include appropriate action points arising from the completed checklist.
Management of quality and standards
In considering the curriculum, learning outcomes and assessment methodology, the programme team should consult a range of internal and external reference points for standards and quality.
External reference points:
Internal reference points:
- information describing aims and intended learning outcomes - Programme Specification(s), Module Specification(s) etc
- evidence of assessment and actual student achievement:
- exam questions
- coursework assignments
- marking criteria
- samples of student exam scripts and work
- External Examiners' reports and related correspondence
- professional body accreditation reports and/or QAA Subject Review reports (where applicable)
- the document Considerations of Good Practice for Taught Programmes
Aspects that might be considered include:
- Do credit definitions of awards adhere to SCQF?
- Do statements of aims and intended outcomes at each level reflect the level descriptions of the SCQF?
- Do statements of aims and intended outcomes at undergraduate honours level reflect the relevant QAA Subject Benchmark statement?
- Do actual practice and actual student achievement reflect intended aims and outcomes and the relevant level descriptions of the SCQF?
- (where relevant) do the curriculum and student achievement reflect expectations of relevant Professional Statutory Bodies?
- Evidence of practice and implementation of procedures relating to standards and quality
- Does practice reflect the guidance given in the University's Considerations of Good Practice in Taught Programmes
- Does practice reflect specific precepts of the QAA Code relevant to the Programme (for example, regarding placements)?
It can be helpful to complete a mapping exercise for each policy reference point:
- list the criteria in the reference point
- list where in the Programme curriculum these criteria are taught, developed and assessed.
Information gained from this part of the evaluation should form the primary input to the Section 3 of the Programme Evaluation Summary 'Evaluation of Standards'
- The strategic and planning context for the School and University
The programme team should evaluate the programme(s) in the context of the School Learning and Teaching Development Plan, the College Learning and Teaching Strategy and the University Learning and Teaching Strategy (see http://www.dundee.ac.uk/qaf/ltstrategiesplans.htm).
Where relevant, the profile of staff available to teach the programme(s) should be reviewed in relation to its sustainability (e.g. in relation to anticipated retirals) and its financial sustainability should be assessed.
It may also be appropriate to evaluate changes or likely changes to the facilities available for teaching (and learning).
This material should be included in section 2.6 of the Programme Evaluation document 'Resources and the Learning Environment' and Appendix 4 'Learning resources'.
Evaluation, Consultation and Planning within Programme Review: Guidenote on Consultation with Support Services
Programme teams should consult with specific support services (see below) to optimise effectiveness of support provided by each service to the programme and in particular to consider:
- adequacy of support resources
- effectiveness of communication between the support service and staff and students on the programme
- how support for the programme could be improved for the future - involving actions by either or both the programme team and the support service.
Support services to be consulted
The support services listed below should be consulted for all Programme Reviews.
- Admissions and Student Recruitment
- Academic Professional Development
- IC Services
- Library & Learning Centre
- Student Services including the Careers Service
Where services nominate staff with responsibilities for different colleges and or schools, then the first point of contact should be those staff members; otherwise it should be the service director. A representative of the Programme Team should make contact the relevant staff member in each support service during the evaluation phase to arrange an initial discussion.
Discussion should take the form of a dialogue about the issues noted in 'Scope' below for each support service, plus any additional issues considered relevant. It would be helpful if the programme team representative could provide an outline of any:
- major changes being considered by the programme team that would impact on support services, for example:
- changes to student numbers
- shifts to open / distance learning and / or mode of using My Dundee
- new collaborative arrangements
- future needs regarding the specific support service.
The outcomes of the discussions should inform the programme team's reflection and preparation of documents for the Review Board. Any notes prepared may be cited as references in the submitted documentation
Surveys of student views associated with Programme Review may include opportunities for students to comment on the effectiveness of support services. Heads of the support services listed above should be:
- given an opportunity to comment on any survey forms prior to issue to students
- provided with copies of results of surveys and consultations that relate to their service.
Documentation
The general outcomes from consultation should be included within the standard documentation for the Review Board prepared by the programme team. Heads of support services may ask schools to use a standard form of documentation, in which case these documents should be submitted to the Review Board as an Annex to the Programme Evaluation Summary.
Scope of discussions
Admissions and Student Recruitment
Admissions and Student Recruitment will prepare a standard market analysis report on request. Discussions should consider:
- trends in student applications and admissions to current programme
- trends in market demand for planned programmes, and highlighting growing areas of demand
- competitor analysis and assistance in market positioning
- student entrance qualifications - any trends associated with student progression and achievement
- effectiveness of liaison between the programme team and Admissions and Student Recruitment
- accuracy and helpfulness of promotional information describing the programme
- effectiveness of marketing and admissions
- how student recruitment for the future could be improved, by either or both:
- actions by Admissions and Student Recruitment and/or External Relations
- actions by the programme team.
Academic Professional Development
Discussions should consider:
- staff development related to learning, teaching and assessment
- how support for the programme might be improved, by either or both:
- actions by Academic Professional Development
- actions by the programme team.
Information and Communications Services (ICS)
Discussions should consider:
- adequacy of IT facilities to support the programme
- effectiveness of communication between programme team and IC Services
- the school or programme team's future plans regarding student numbers and IT use, with a forward look that includes:
- servers and storage capacity
- printing requirements
- any special remote access requirements
- any special open/distance learning requirements
- use of central IT suites for teaching
- any specific IT requirements for students with disability or special needs
- IT induction.
- How IT support for the programme could be improved by:
- actions by the ICS
- actions by the programme team.
Library
The programme team should send the Subject Liaison Librarian a list of the modules associated with the programme and hence included in the Programme Review. This should take place as early as possible in the evaluation phase. The Subject Liaison Librarian should submit the Library's standard form to the School Secretary prior to the Programme Review Board (View Programme Review: Library consultation form). Discussions should consider:
- adequacy of library resources to support the programme
- effectiveness of communication between programme team and library
- how support for the programme could be improved by actions by the Library or actions by the programme team.
Learning Centre
Discussions should consider:
- effectiveness of existing Learning Centre support within the programme (e.g. academic skills support, eLearning, online assessment, online student feedback and IT induction)
- quality of communication between the Learning Centre and the programme team
- how the programme could be improved by actions of the Learning Centre or the programme team
- specific issues for Academic Skills Support
- effectiveness of liaison between programme team and academic support team
- level of awareness of staff and students about the academic support programme organised by the Learning Centre
- general extent of uptake of academic support opportunities by students on the programme and the suitability of elements of the support programme for this cohort of students
- trends in the needs of this cohort of students for academic support
- opportunities to embed academic support within the curriculum, where necessary involving contributions by staff from the Learning Centre
- how retention, progression and achievement of students might be assisted via academic skills support
- specific issues for eLearning and Online Assessment
- effectiveness of liaison between programme team and eLearning support team
- current and projected role of My Dundee modules, organisations and content system in the delivery of the curriculum
- current and projected role of online assessment systems in both formative and summative assessment
- use of other eLearning-related systems within the programme (e.g. online plagiarism detection, online group or peer assessment, personal response systems)
- student and staff views on eLearning and online assessment (e.g. as provided in student feedback and in the most recent Student and Staff eLearning surveys)
- how retention, progression and achievement of students might be assisted via eLearning (e.g. early information about student engagement)
- additional Learning Centre activities, such as: aspects of pedagogic design; administration of online student feedback (Survey Service); support for Personal Development Planning (PDP).
Student Services
Discussions should consider:
- general issues relating to any of the student services, but principally and specifically relating to: Careers Services; Counselling Service; Disability Services; Student Advisory Service (but not Academic Skills Support - see Learning Centre above)
- effectiveness of liaison between the programme team and the support services
- awareness by students of the programme of support offered by Student Services
- general extent of uptake and the kinds of support needed/provided at a general level
- how student support for the future could be improved, by actions by Student Services or actions by the programme team
- specific issues for Careers Service
- trends in destinations of graduates from the programme
- relevance of the curriculum to the needs of employment
- employment market and opportunities for graduates of the future
- how employability of future graduates from the programme could be improved, by actions by Careers Service or actions by the programme team.
[Note: Copies of graduate first destination statistics should be made available to the Programme Team by Careers Service]
- specific issues for Disability Services
- effectiveness of support at programme level for any students with disclosed disability
- accessibility of this programme with reference to Teachability criteria
- how accessibility, and the achievement and experience of students with disability could be improved by actions by Disability Services or actions by the programme team
Evaluation, Consultation and Planning within Programme Review: Guidenote on Consultation with Students and Graduates
Students and graduates should be included in the evaluation process in ways that:
- are systematic, representative and rigorous, but are appropriate to, and sensitive to the nature of the student body
- positively encourage students and graduates to contribute their views on the effectiveness of the programme and to make suggestions on how it could be improved.
Accordingly, the evaluation phase of programme review should include:
Analysis of 'routine' feedback
This should include consideration of student views expressed:
- in annual, or end-of-module questionnaires
- at meetings of Staff-Student-Liaison-Committee, School and College boards
- in the National Student Survey or, for taught postgraduate programmes, the Postgraduate Taught Experience survey (PTES)
Analysis should consider broad trends, and in particular any student concerns or suggestions that have not been resolved. It should be possible to obtain relevant information by analysing past Annual Programme Monitoring Reports.
Specific surveys
Targetted surveys can be used to inform staff evaluation seeking student views on specific issues, for example:
- what students think of the current programme
- students' suggestions for improvement
- and possibly student comments on any changes that staff are considering.
Where relevant, the views of appropriate categories of students should be sought (for example part-time vs full-time, early years vs final years, groups with different entry routes or qualifications).
Methods may include:
- cohort questionnaires - involving all, or a sample of current students on the programme
- discussions - within a class or focus groups.
The University Survey Service (Library & Learning Centre) can provide advice in relation to:
- questionnaire design
- questionnaire issue
- analysis and reporting
- any focus group or face-to-face work arising from survey results.
The Alumni & Development Office may be able to facilitate contact with graduates.
Student class representatives should be invited to comment in advance on the methods used and questions asked.
A similar approach could be used to survey the views of recent graduates, especially in relation to the suitability of the education and training received on the programme to their careers. However, generating contact lists could be more problematic, and any follow-up activities would probably have to be by e-mail rather than face-to-face.
Documentation submitted to Programme Review Boards: Guidenote and Outline for the Programme Evaluation Summary
The objectives of the Evaluation Summary are:
- to summarise and communicate the findings of the programme team's evaluation
- to demonstrate that evaluation has been comprehensive, rigorous, evidence-based and that it reflects key contextual factors and reference points.
The Evaluation Summary should be reflective in nature, and evidence-based. It should be concise; detailed factual information should be selectively presented, and tables of information should be confined to appendices or cited as reference papers.
The following table provides guidance on structure and content of the individual sections. No limits are prescribed on length, but the following are suggested as indicators of upper limits: Section 1, five sides A4; Section 2, 20 sides A4; Section 3, five sides A4.
Programme Review: Programme Evaluation Summary
| Headings |
Suggested content |
| SECTION 1: INTRODUCTION |
| 1.1 Programmes included in the review |
Names and awards of the programmes included in this Review. This should normally include, as applicable within the cognate area: undergraduate and postgraduate awards, CPD, collaborative and overseas provision , supervision of research students and online and distance learning provision. |
| 1.2 Context |
Brief statement of the context in which the Review is taking place, highlighting any key issues. For example:
- recent developments at University, college school or programme levels, including reference to learning and teaching strategies and plans
- any significant developments associated with the subject (e.g. changes to QAA benchmark statement or accreditation requirements of professional / statutory bodies)
|
| 1.3 The evaluation process |
Brief explanation of how the evaluation was undertaken - e.g. how it was structured, who did what, how the views of students and graduates were taken into account |
| SECTION 2: REFLECTION |
|
2.1 The students
2.2 The curriculum
2.3 Assessment
2.4 Teaching
2.5 Student support and guidance
2.6 Resources and the learning environment
2.7 Communication, management and enhancement |
This section should:
|
| SECTION 3: CONCLUSIONS |
| 3.1 Summary of strengths and good practice |
Points of strengths and perceived good practice. The objectives are:
- to identify and acknowledge good practice;
- to encourage dissemination across the University
|
| 3.2 Summary of needs and opportunities for change |
Points of needs and opportunities for change. This sets the agenda of issues for the Programme Development Plan |
| LIST OF REFERENCES |
A simple list of all references cited in the text of the Programme Evaluation Summary. This can also form an index and 'List of contents' for any Supporting documentation. |
| APPENDICES |
| Appendix 1: Programme and module specifications |
The programme and module specification(s) describing what the programme is, the aims, intended learning outcomes and structure of the existing programme, using the University's standard EDS-compliant forms. If, as a result of their evaluation, the programme team proposes radical changes to the programme aims and/or structure, a draft proposed Programme Specification should also be included. |
| Appendix 2: Evaluation of standards |
Mapping of curriculum against relevant external reference points, to include Scottish Credit and Qualifications Framework and where relevant QAA Subject Benchmark Statements at Honours Undergraduate level, and Professional Body criteria. |
| Appendix 3: Staff |
Listing of staff, including part-time staff, tutors and teaching assistants who deliver the programme |
| Appendix 4: Learning resources |
Listing of principal learning resources associated with the programme |
| Appendix 5: Student recruitment materials |
Copy of any publicity or marketing material for prospective students prepared at school or programme level. The Review Board should also be provided with relevant excerpts from University-level publicity and marketing information including the Prospectus and WWW pages Contact: Admissions and Student Recruitment |
| Appendix 6: Completed inclusive curriculum checklist |
Copy of the checklist with observations and action points relating to the programme(s) under review |
| Appendix 7: Student numbers and progression |
Summary of recruitment, retention, progression and achievement Contact: Admissions & Student Recruitment Service (recruitment data via ARMI database); Registry (SITS data but note that this information should be calculated annually as part of annual programme monitoring). |
| Appendix 8: Graduate destinations |
Summary of graduate destinations Contact: Careers Service. |
Documentation submitted to Programme Review Boards: Guidenote and outline for the Programme Development Plan
The objectives of the Programme Development Plan are to:
- explain how and when the programme might be enhanced in the future;
- outline proposed future enhancement actions;
- indicate how and when changes will be implemented.
In constructing the Programme Development Plan, it may be valuable for the programme team to consider the following aspects.
Likely student intake of the future
Where to look:
- recent historic data on student applications and intake
- consultation with Admissions and Student Recruitment.
What to consider:
- recent trends in student intake
- the nature of the likely student intake of the future - numbers and characteristics.
Likely opportunities for future graduates from the Programme
Where to look:
- recent historic data on graduate employment
- consultation with the University's Careers Service
- information from relevant subject associations, Professional and Statutory bodies.
What to consider:
- trends in employment for graduates in this subject
- the nature of likely opportunities for future graduates from the Programme.
Likely context and environment in which the Programme will operate in the future
Where to look:
- discussion by staff who teach the Programme
- The University Strategic Framework and Learning and Teaching Strategy, the College Learning and Teaching Strategy, the School Learning and Teaching Development Plan
- consultation within the College - for example with the Secretary, Vice-Principal, Planning Committee etc
- consultation with University Directorate of Planning and Information Services.
What to consider:
- needs and likely availability of resources to support the Programme in the future, considering:
- staff
- learning resources -teaching accommodation, library, computing and IT, specialist equipment, class grant etc
- sustainability of the Programme - both current and projected future situations.
- modifications required to ensure that the delivery of the programme complies with the Equality Act (2010).
The following table provides guidance on structure and content of individual sections of the development plan.
Programme Review: Programme Development Plan
| Headings |
Suggested content |
| SECTION 1: INTRODUCTION |
|
| 1.1 Programmes included in the development plan |
List names and awards of the programmes included in this Review (copy from section 1.1 of Programme Evaluation Summary) |
| 1.2 Context |
Brief statement of the context in which the Review is taking place, highlighting any key issues highlighting any key issues. e.g.:
- recent developments at University, college, school, or programme levels
- any significant developments associated with the subject (e.g. changes in accreditation requirements of professional / statutory bodies)
(copy from section 1.2 Programme Evaluation Summary)
|
| 1.3. Summary of needs and opportunities for change |
Copy from section 3.2 of Programme Evaluation Summary) |
| SECTION 2: PLANNED DEVELOPMENTS |
|
| Development plan for the future - typically the next six years. |
This should indicate how the programme team proposes to manage and enhance the programme for the future.
The Plan:
- should include proposals to address needs and opportunities and to build on strengths identified in the Programme Evaluation Summary
- may include provisional or conditional elements or highlight issues for which further investigation is required
- will likely include greater detail for early actions, with later actions being described in more general terms
- should be a primary reference point for future annual monitoring and should be updated annually
|
Documentation submitted to Programme Review Boards: Guidenote on other supporting material
The following documentation should be available for reference by the review board in advance of the review event. Additionally, members of the review board may request a copy of specific documents from the School Secretary in advance of the review event. It is encouraged that all or parts of this documentation is made available in electronic format with suitable access facilities - for example, on a website, My Dundee module or CD.
This supporting documentation should normally include:
- all references cited in the Programme Review Summary (with the exception of any deemed by the Dean to be commercially confidential - e.g. student recruitment and marketing reports)
- copies of publicity and promotion material
- student handbooks (see Senate policy on content)
- module specifications for all core and options
- Annual Module and Programme Monitoring reports since the last programme review and relevant extracts of School Board minutes where issues arising from these have been discussed
- External Examiner reports (most recent two years)
- PSB (Professional and Statutory Body) reports, where applicable - most recent
- set of examination papers and coursework assignments (most recent year)
- other materials deemed relevant by the programme team.
Conduct of Programme Review Boards: Composition of the Programme Review Board
Membership of programme review boards should be negotiated by the college head of learning and teaching, or similar nominee of the college VP, and the relevant dean. They should together agree on Review Board membership that is appropriate for the programme, ensuring proper representation by external subject experts.
Membership will normally include:
- a convener, normally the college head of quality assurance (or equivalent) or another senior member of academic staff within the college
- an experienced member of academic staff from a different school within the college. If the programme is delivered in open or distance learning mode, or if open or distance learning is proposed for the future, the review board must include an academic who has knowledge and experience of open/distance learning
- an experienced member of academic staff from another college
- a student representative, nominated by the president or deputy president of Dundee University Students' Association (DUSA), normally a member of the executive of who has attended training events organised by Academic Affairs and/or Student Participation in Quality Scotland (sparqs). The DUSA President or Deputy President should be contacted directly to identify the student representative.
- at least one Student Services' representative (in addition to the ASRS member noted in 6 below), and more if desired. The review Convenor and Dean / Programme Leader should together decide which representative or representative(s) would be best able to contribute to the review process, according to the nature and needs of the programme. Services that might be considered include: the Careers Service (for example, in relation to employability and employment); the Library and Learning Centre (for example, in relation to access to learning resources, e-learning and educational development); Information and Communications Service (for example, in relation to IT facilities). The Director of the relevant service(s) should be consulted regarding appropriate participants and their availability in advance of the event.
- the Director of the University's Admissions and Students Recruitment Service (or nominee). The Admissions and Students Recruitment Service participation may be by correspondence, rather than attendance at the full review board. This should be agreed by the Head of the Admissions and Students Recruitment Service and the Convener of the review board in advance of the event
- a minimum of one external expert in the subject and ideally two (consistent with the range and volume of provision being reviewed). One should be an academic from another Scottish university or a professional/practitioner with current knowledge of higher education in the subject. The other should come from an institution or body outwith Scotland. Consideration should be given to including external representatives chosen from employers, recent graduates or a professional body as well as academia.
The organising secretary for the event should contact proposed members well in advance to agree availability. For DUSA executive members, the President should be approached in the first instance and will allocate duties as he or she deems appropriate. Regarding University of Dundee academic staff, an approach should be made first to the dean of the relevant school (or equivalent) to ask for a specific member of academic staff to sit on a board. If a particular staff member cannot attend for valid reasons, the expectation will be that the dean who has been approached will designate an alternative.
The Review Board cannot include any staff who teach on the programme. At the invitation of the Convenor, observers may attend Programme Review meetings, but they should not take part in discussions, unless specifically asked to do so by the Convenor.
The Review Board Convener should be supported by a Review Board Secretary nominated by the College Secretary, whose chief role is to support the convenor in the preparation of the report. It is recommended that the Review Board Secretary is not the secretary for the school running the programme, to allow that secretary to attend the board as a member of the senior management and academic staff panels.
Conduct of Programme Review Boards: Composition of Review Board Panels
The Programme Review Board examines submitted documents and other information and meets with panels of
(a) school executive staff
(b) students
(c) academic staff who teach on the programme.
These panels may involve administrative staff for (a) and (c) where the Programme Team considers this would be useful and constructive. Panel (b) should consist of students and graduates only - no staff members should be in attendance.
Panel (b) may include student representatives and recent graduates where deemed appropriate.
Panel (c) should not include members of panel (a) but the panels may be merged in the case of a very small programme team.
Membership of the panels should be proposed by the School Secretary and approved by the Convenor ahead of the meetings.
Conduct of Programme Review Boards: Conduct of Programme Review
This guidenote has been prepared to provide a broad starting point to help members of review boards to contribute to the process effectively and efficiently. It is not intended to form a set of 'rules'. Conveners should feel free to organise and structure the review board event in ways that reflect the subject, participants and context of the particular review. A recommended model, which assumes a review lasting one and a half days, is as follows:
1. Before the Programme Review Board event. The School Secretary should ensure that members should receive hard copies of the following documentation a minimum of three weeks prior to the review board event (preferably earlier)
Organisational information:
- membership of the Review Board and panels
- timetable for event, including locations and maps
- for external members, an expenses claim form.
Reference documentation:
- information on University programme review policy and procedures
- relevant excerpts from the Scottish Credit and Qualifications Framework
- relevant QAA subject benchmark (honours undergraduate programmes)
- professional statutory body (PSB) guidelines (where relevant)
Programme documentation:
- Programme Evaluation Summary
- Programme Development Plan
- other supporting materials, as appropriate.
If a member has any questions, or wishes a copy of any reference cited in the Programme Evaluation Summary, they should contact the School Secretary who issued the documentation, noting that it is also possible to check or obtain a copy of any cited references at the event. The convener may also request preliminary written comments by Board Members to be submitted prior to the review board event
2. First (private) pre-meeting of Programme Review Board. This may be held on the afternoon before the meetings with the programme panels. Here, members of the Board will be introduced, and will give initial impressions of the submitted materials. Any prior interests or connection with the programme on the part of a Board member should be declared at the first meeting (for example, an external expert having been an external examiner or a student having participated in a programme module). Convenors may then call for topics and issues that the Board members would like to pursue with the different groups and the Board should then agree:
- who will lead on these issues during meetings
- the panels at which each issue will be discussed.
3. Second pre-meeting of Programme Review Board. This is often held immediately before the meetings with the programme panels. The Convenor may present a plan of action based on the previous day's discussions for agreement by the Board.
4. Meetings with programme panels. There is no prescribed order, but there may be benefits in interviewing teaching staff last as they may then be in a position to answer points raised by the Review Board as a result of earlier discussions with executive staff and students. The Convenor may wish the Board members to provide initial impressions after each meeting. For convenience, the meeting with students and graduates is often held informally over lunch. For the meeting with the Programme Team, the Convenor may allow a short presentation by staff - however, this should not dominate the meeting.
5. Third meeting of Review Board. The Board discusses its decision and agrees on its conclusions. This should cover:
- recommendation to school board regarding the future operation of the Programme
- any conditions associated with recommendation
- any suggestions regarding the future operation and enhancement of the Programme
- any aspects of good practice that the Review Board wishes to highlight
- any suggestions regarding the University's Programme Review policy and procedure
- any additional comments.
Before the meeting with the Programme Team, the Convenor should read the decision, any conditions and recommendations, and points of good practice back to the Board members, to ensure all are in agreement with the wording.
6. Meeting of Review Board with Programme Team. The Convenor conveys the decision and any conditions or recommendations. This is a reporting rather than a discussion meeting. The convener should read out key points of the conclusions, and then close the event, indicating approximately when the Programme Team can expect to receive the report.
7. Drafting of Report. The convener and secretary are responsible for preparing the report. They may invite individual members to prepare drafts of specific sections. Having drafted the Report, the Convenor and Board Secretary circulate this to panel members for comment and suggestions for revision. The draft report should be submitted to the Programme Leader, School Secretary and Dean to check its accuracy.
8. Submission of final report. The convenor submits the agreed report to:
- The Dean of the School and/or the School Secretary
- The Programme Team leader.
Conduct of Programme Review Boards: Suggested areas of focus for Review Board members
Suggestions on areas of particular focus for individual members are noted below. However it is emphasised that all members have both the authority and responsibility to comment on any aspect or issue.
External subject experts
- current and advancing knowledge in the subject
- needs of employment in the subject
- aspects of international good practice, where relevant
- relevant external reference points associated with the subject -e.g. QAA Subject benchmark statement, any professional / statutory body criteria or guidelines.
Extra-college and extra-school member(s)
- appropriateness of credit rating and reference to Scottish Credit and Qualifications Framework (SCQF)
- approaches to teaching, learning and assessment
- the sustainability of the Programme(s)
- those parts of the Evaluation Summary and Development Plan related to the inclusive nature of the curriculum and its delivery
- extent to which this review reflects SFC guidelines to institutions on the characteristics of internal (subject) review:
- all provision should be reviewed on a cycle of not more than 6 years
- reviews should take full account of benchmarks and the code of practice
- reviews should take full account of the Scottish Credit and Qualifications Framework
- reviews should provide an objective review of provision based on an understanding of national and international good practice: each review team should include at least one external member with a relevant background
- internal processes should take full account of student feedback, and include procedures to obtain student views of the provision being reviewed
- internal reviews should consider the effectiveness of annual monitoring arrangements and follow-up action for programmes covered by the review
- internal reviews should provide assurance and promote enhancement review.
Careers Service member
- generic needs of employment, including personal transferable skills within the curriculum and employability
- careers guidance support to students
- operation of the University's PDP scheme within the programme.
ASRS member
- programme title and recruitment and publicity material for the programme
- the Programme's recruitment profile in relation to national and international trends for the subject and related subjects
- projected student numbers within any business models presented.
Student member
In general, read and comment on documentation and plans from a student perspective and specifically:
- examine the ways in which the curriculum is taught and assessed
- consider how student learning is supported, including, where appropriate, web-based information, My Dundee modules, library resources, programme and module handbooks
- scrutinize the ways in which student representation, feedback and survey information is taken into account and fed back to students
- review the operation of academic and other support systems for students (for example, advisors of study, personal tutors; student support services)
- comment on recruitment strategies and materials
- explore the accessibility of the course to students.
Distance Learning expert (where relevant)
- effectiveness of approaches to distance learning.
Conduct of Programme Review Boards: Reporting and reviewing for Programme Review
The Review Board Report should be structured in four sections, as outlined in the table which follows:
Programme Review: Review Board Report
| Headings |
Suggested content |
| Introduction
|
|
|
Observations and questions to staff and students
|
|
|
Summary of the Review Board's views
|
- strengths and good practice
- the effectiveness of the programme team's evaluation and justification of standards of the programme
- needs and opportunities for change
- commentary on the inclusive nature of the curriculum and its delivery
- the programme team's proposals for future change
- the sustainability of the Programme(s)
- the accuracy and clarity of publicity and marketing information describing the programme
|
|
Conclusions
|
- a formal recommendation to School Board regarding the future operation of the programme
- any conditions or suggestions associated with the recommendation
- any aspects of good practice that the Review Board wishes to highlight
- any suggestions regarding the University's Programme Review policy and procedure
- any additional comments
|
For each programme considered, the Review Board's formal recommendation to school and college board should be stated with reference to one of the following categories:
- 'Approved'
- 'Approved with suggestions' - these would be classed as 'advisory' and not 'required'
- 'Approved, subject to conditions' - when the Review Board considers that certain changes or actions are required
- 'Not approved' - in such a circumstance, the Review Board should note the reasons for making this judgement.
The status of the Review Board Report is that of a recommendation to the relevant School Board(s).
- The School Board should receive the report, discuss it, and consider an action plan, integrating this, where appropriate, within the School Learning & Teaching Development Plan. This must be carried out within one year of the publication of the final review report. The College Board should monitor progress on matters contained within the review reports, with special attention to any conditions that are mentioned.
- The Dean should submit a copy of the Programme Review Report plus the School Board's observations to the College Board, the Director of Quality Assurance and the Vice-Principal (Educational Development).
- The Dean or school secretary should follow-up any issues as appropriate. Those relating to College or University policy, procedures or resourcing should be noted for consideration by the College Vice-Principal or University Vice-Principal (Educational Development) as appropriate.
- Heads of support services should be informed about relevant outcomes of the review process, where relevant.
- If the Review Board's recommendation is that the programme is 'not approved':
- The Vice-Principal (Educational Development) and the Director of Quality Assurance should be informed immediately.
- the Dean and Convener of the Programme Review Board should together agree a plan of action, consulting the College Vice-Principal or nominee
- the Dean should submit a copy of the report, and a note of the plan of action, to School Board, College Board and the University Academic Quality Committee (via The Vice-Principal (Educational Development) and the Director of Quality Assurance).
Outcomes of the Programme Review Report may be followed up by The Vice-Principal (Educational Development), the Academic Secretary, the Director of Quality Assurance and the Academic Quality Sub-Committee (AQSC) of Learning and Teaching Committee. Where the outcomes from a programme review result in modifications of the programme review specification, the procedures described in Changes to Provison must be adopted.
The Director of Quality Assurance prepares an annual report on programme review for submission to AQSC and the Scottish Funding Council. This report includes a digest of significant development issues arising in reports as well as good practice that has been identified. The significant development issues are discussed at AQSC and the good practice reported to the Director of Academic Professional Development for action and dissemination.
Links to other Policies and Guidance Relevant to Programme Review
Policy statement on Considerations of Good Practice for Taught Programmes
Policy statement and guidance on Programme and Module Specifications
Policy statement on Programme Review in the context of Accreditation of Programmes by Professional or Statutory Bodies
Link to relevant external guidelines
QAA Code of Practice Section 7 and SFC guidance.* The University's Programme Review process has been designed to reflect these guidelines.
*see http://www.sfc.ac.uk/information/info_circulars/sfc/2008/sfc3008/sfc3008.pdf for new link