Online assessment policy and procedures
Updated on 12 November 2013
This document outlines policies on the use of centrally supported online examination systems and procedures for the use of digital technologies in the assessment process at the University of Dundee
This document outlines policies on the use of centrally supported online examination systems and procedures for the use of digital technologies in the assessment process at the University of Dundee. The precepts within this document have been developed to ensure that all assessment activity that involves the use of information technology (IT) is aligned with the University of Dundee’s Assessment Policy for Taught Provision and takes cognisance of the international standards document ISO/IEC 23988 (2007) ‘Information technology - A code of practice for the use of information technology (IT) in the delivery of assessments’.
The policies and procedures described here are based on principles that assessments presented to students by electronic means will be accurate, confidential, fair, reliable and secure and run according to relevant legislation and good practice.
Where Schools or individuals members of staff choose to use tools or approaches not explicitly referenced within this policy it is their responsibility to adhere to the principles within this document.
2. General principles
2.1 Development of assessments
The Library and Learning Centre (LLC) can provide tailored advice or training to individuals, module/programme teams or Schools on the effective use of online assessment and feedback technologies. It is recommended that academic staff members considering use of online assessment in their modules or programmes complete the School of Education, Social Work and Community Education accredited modules ‘Assessment Online’ and ‘Learning and Teaching Online’.
2.2 Assessment design and guidance to students
Assessments should be carefully designed with due regard to module and programme learning outcomes, Scottish Credit and Qualifications Framework (SCQF) level descriptors and relevant professional body requirements. Particular care should be taken in the design of objective tests comprising multiple choice or multiple response questions to ensure that a sufficient quantity and quality of distracters are used. Negatively worded questions should be avoided. Negative marking may be used. It is important that the marking regime used (including scoring rules for individual questions) should be made clear to the students in advance of any assessment. Assessments should be checked for accuracy of content, spelling, punctuation and to avoid potential ambiguity. Peer review of new questions is recommended.
It is the responsibility of Schools to ensure that their students are familiar with the tool, system or delivery platform to be used, particularly prior to undertaking a summative assessment. It is recommended that opportunities for students to familiarise themselves with the relevant tool or system be provided through the provision of a formative assessment of a similar format, incorporating question types and content relevant to the specific module of study.
The following information should be communicated by Schools/academic staff to students before an assessment takes place:
- the format of the assessment
- the number and type of questions to be used/answered
- the scoring rules for individual questions and the overall assessment
- the contribution the assessment makes to an overall module
- how the questions are selected for the assessment
- any time limit
- any restraints on navigation between questions, or blocks of questions
- assessment regulations including permitted and excluded resources
- feedback to be provided (including timescales and their mark if appropriate)
- details of the appeals procedure available to students
- how they will be identified for the assessment (i.e. they must bring their matriculation cards).
2.3 Sensitive academic material
For the purposes of assessment, academic staff may find it appropriate to use multimedia, image or text material that has the potential to cause unnecessary offence to users. Examples of such material include: medical images and videos detailing particularly graphic detail, forensic science images of human remains or graphic text describing offensive behaviour from a legal or social work perspective.
Whilst students studying the discipline may be familiar with the material, academic staff have a responsibility to:
- ensure students are aware of the nature of the material
- ensure students are aware that others who casually view the material may be offended by it
- minimise the chances of offence by suggesting the material is viewed in specialist or private IT suites, or by scheduling private sessions.
Academic staff members preparing assessments that contain materials that may be regarded as offensive by those not studying the topic should also advise relevant support staff prior to circulation/presentation of the assessment.
2.4 Disability and accessibility support
The University has a legal duty under the Equality Act 2010 to anticipate the needs of disabled students and to make all reasonable adjustments to meet their needs. The University also has a duty to promote equality of opportunity and to assess the impact of all of its activities on disabled people, including all aspects of teaching and assessment.
The University’s online assessment systems are designed to be as accessible as possible to disabled students and to follow standard University accessibility procedures and good practice guidelines.
They are also designed to be compliant with most assistive technology (e.g. screen readers). However, further progress is required in this respect and the University continues to work with the developers of online assessment tools to improve accessibility for disabled students. Any accessibility difficulties experienced by individual disabled students will need to be taken into account and an alternative to the online assessment provided where necessary as a reasonable adjustment.
Some disabled candidates may be granted additional time to complete an assessment, and the facility to operate an assessment for a single person in an alternative room may be required. These candidates should be identified to the LLC at the time of examination registration. Some disabled candidates may affix a sticker to their ‘Online Examination Attendance Form’ to identify the fact that they have a specific disability. Such assessments should be marked in line with Disability Services’ guidelines e.g. Dyslexia Marking Guidelines unless accuracy of written expression is a genuine competence standard for the assessment. Guidance on this should be sought from Disability Services.
Such reasonable adjustments to the assessment process for disabled students are identified by Disability Services and communicated to School Disability Support Officers (DSOs) following an individual needs assessment, in line with the University’s policy on the provision of academic-related adjustments for disabled students. Further information is available from Disability Services.
General guidelines for the interpretation, detection of, and penalties for plagiarism are provided through the Academic misconduct by students code of practice
It is the responsibility of Schools to determine how the issue of plagiarism will be most effectively addressed when information technology is used in the delivery of assessments (e.g. whether random sampling will be employed, or whether originality software will be used).
2.6.1 Relevant legislation
Section 32 of the Copyright Designs and Patents Act 1988 provides by subsection (3) that:
‘Copyright is not infringed by anything done for the purposes of an examination by way of setting the questions, communicating the questions to the candidates or answering the questions’.
However, this is qualified by subsection (4) thus:
‘Subsection (3) does not extend to the making of a reprographic copy of a musical work for use by an examination candidate in performing the work.’
2.6.2 Summative assessment
Our present University interpretation is that copyright would not be infringed if a photocopy or digitised copy were used within an examination. The material could form part of the paper itself, or be issued to students as a separate item. The exception does not apply to musical works.
It should be noted that even when this exception applies, use of the work must be accompanied by a full citation of title and author except where this is impractical or otherwise impossible.
2.6.3 Formative assessment
The use of copyright material in formative assessments is subject to the same guidance as for learning resources i.e. only copyright-cleared material (whether licensed or where clearance is obtained directly from the copyright holder) may be directly used. This applies to text, images and moving images. Links to publicly available materials on the web may be provided within formative assessments, but these must always be properly credited.
The use of copyright material in summative assessment is permissible. The use of copyright material in formative assessment may be permissible. Musical works are excluded in both cases. Any other or subsequent use is excluded, including the reproduction of copyright material as part of a past paper.
2.7 Health and safety procedures
PCs, chairs and the general assessment environment must conform to standard safe environmental guidelines (clear screen, adjustable screen, adjustable chair, good lighting and ventilation).
While there is no legal limit on the length of time an individual may work with a visual display unit (VDU), Schools should have an awareness of Health and Safety Executive (HSE) regulations relating to the use of VDUs (http://www.hse.gov.uk/pubns/indg36.pdf).
2.8. Reliability of assessment platform
It is recognised that no system is 100% reliable. In the event of technical problems or system failure at network level, examiners may be required to reschedule examinations or extend submission dates. In the case of online examinations it is recommended that a backup system of paper copies should be available (responsibility for the provision of paper-based copies lies with Schools). Spare computers should be available at a ratio of at least 1 per 20 students (5%) with a minimum of 3 spare PCs per IT suite. In the case of the central online assessment systems testing will be conducted by the LLC in conjunction with University of Dundee Information Technology (UoD IT). Where Schools operate their own online assessment systems it is their responsibility to ensure that appropriate testing takes place.
2.9. Exam irregularities and grievance procedure
In the event of any examination irregularity or grievance relating to an assessment involving online assessment, the Director of Policy, Governance and Legal Affairs (PGLA) must be informed to ensure appropriate procedures are followed (as is the case with other examinations).
Electronic logs of assessment details should be retained for a minimum of 90 days after any medium or high stakes assessment. Software used for medium or high stakes assessments must permit the interrogation of individual question answers and must allow human re-marking of questions if judged necessary.
3. Procedures for centrally supported online examination systems
The Questionmark Perception and ExamOnline systems are jointly managed and supported by the LLC and UoD IT. Contact with the software vendors is co-ordinated by staff in the LLC to whom any queries should be directed in the first instance (email@example.com).
- provide training and support in use of the centrally supported systems
- can assist academics with the authoring of questions for summative examinations/class test – firstname.lastname@example.org should be contacted to discuss approaches
- provide guidance on effective question/assessment design
- provide support for online examinations (see detailed Examination Procedures below).
3.2 Summative examination procedures
3.2.1 Pre-examination procedures
Online examinations/class tests to be presented using centrally supported systems should follow the procedures below.
Schools should complete the ‘Online Examination Registration Form’ and email to the LLC at least four weeks before the date of the examination/class test. Completed forms should be submitted to email@example.com.
For examinations/class tests outwith University examination diets Schools should book the IT suite(s) required using the Web Room Bookings form ensuring 5% spare PCs with a minimum of 3 spare per IT suite. IT suites should be booked for an additional 30 minutes before an assessment is scheduled to start and 30 minutes after the assessment is scheduled to finish.
A finalised (approved) version of the examination/class test must be ready ten working days before the date of the examination and confirmation sent to the LLC via the email address firstname.lastname@example.org.
The LLC will send a copy of the ‘Online Examination Attendance Form’ to the relevant academic a minimum of two working days before the examination date. This form will specify the login details to use. In the case of an exam with multiple sittings a minimum of 20 minutes between sittings should be allocated to allow for change of password and collection and distribution of attendance forms. It is the responsibility of the School to print a copy of the Online Examination Attendance Form for each student scheduled to sit the examination.
It is the responsibility of academic staff to ensure that their assessments are checked for accuracy of content, spelling, punctuation, presentation and scoring prior to the delivery of the assessment to students.
It is the responsibility of academic staff/Schools to ensure correct invigilation procedures will be in place (minimum of 2 invigilators per IT suite) and that the possibility of collusion due to overlapping exam times or locations is avoided.
3.2.2 Detailed summative procedures
For summative examinations/class tests Schools must adhere to the University of Dundee Regulations for the Conduct of Examinations (see section 3.2.3).
Students will login to PCs using an account which is associated with secure browser software used for the presentation of high-stakes assessments. This software prevents students from accessing any other applications on the PC.
Student accounts are created by the LLC based on a list of usernames and matriculation numbers provided by the School or obtained from My Dundee. Passwords will be matriculation numbers. A set of spare accounts are created for each examination (see Section 3.2.4 Technical Issues).
The ‘Online Examination Attendance Form’ must be completed, signed and dated by the student. This form should be printed by the School; distributed to students by invigilators; collected by invigilators at the end of the examination and subsequently stored by the School.
Immediately after the examination has concluded the Senior Invigilator should count in the number of attendance forms. A member of LLC staff should check that the number of submission corresponds with the number of forms collected and if any submission is missing should identify the relevant PC(s) and attempt a recovery.
After an examination has finished, a copy of the submissions will be taken by the LLC and emailed to the academic in charge of the examination.
The submissions must be checked by the academic staff member responsible for the exam at the earliest opportunity to ensure accuracy and should be published to the students as appropriate. Results should also be forwarded to the Examination Office as appropriate.
Overall responsibility for the conduct of examinations resides with Schools.
Invigilation of online examinations should be carried out with due diligence. Invigilation should adhere to the University of Dundee Regulations for the Conduct of Examinations (https://www.dundee.ac.uk/qf/documents/details/guidance-for-invigilators-of-examinations.php).
3.3 Technical Issues
In the event of system failure prior to the start of an examination which renders it impossible for students to launch the assessment, a member of LLC staff will contact UoD IT to attempt to resolve the problem. If the issue cannot be resolved within 20 minutes the exam may have to be postponed until the problem can be fully diagnosed and rectified. Should the system fail after or during the exam, rendering it impossible to transmit answer data to the server, a member of LLC staff should attempt (where software permits) to recover the submission file from the local PC.
In the event of individual workstation failure a student may be allocated a spare PC and/or a spare account and allowed to restart the examination. Alternatively, the Senior Invigilator may elect to allow the student to complete the exam using a paper copy. The student should be given the full examination time limit if forced to restart, irrespective of whether online or on paper. The PC number of the workstation that failed along with any error message should be noted by an Invigilator and passed to a member of LLC staff who will log the issue with UoD IT.
Dr D J Walker
Senior Learning Technologist
Library & Learning Centre
Approved by the Senate Learning and Teaching Committee 12 November 2013
Minor revisions to update URLs – March 2015
Minor revisions to update URLs – July 2018
Academic and Corporate Governancequalityandacademicstandards@dundee.ac.uk