Examinarium

In support of planning questions:
Examinations are divided into sections that contain theme-specific questions. You can create a set of theme-specific questions and have the system display the same set of questions to all students or choose a random selection of questions for each student.
There are five types of questions: essay, multiple choice with one correct answer, multiple choice with one or several correct answers, fill in the blanks question and statement question.
Assignments can comprise text and no more than one attached file. The attached file can be an image or a PDF file.
The answers are made up of text with simple formatting, such a lists. (Students can also attach files to their answers, but the software currently imposes restrictions on that type of answer.)
You can also prepare individual examinations accessible to only one student.
 

The pedagogical design of questions
Consider the maximum score for each question. When you create randomised questions, make sure that all interchangeable questions have the same maximum score.
Moreover, you will want to make sure that the interchangeable questions are of the same level so that the students have an equal opportunity to earn the highest grade or pass the examination.
Exams for which students are allowed to bring notes and materials can’t be taken in Examinarium rooms. Background materials can be added to the exam as an attachment.

Book exams in Examinarium
Book exams and other exams that does not involve teaching and are possibly to take freely around the year, are ideal for an Examinarium exam. The teachers prepare and publish the exams with alternative questions before the start of the academic year, and then they can concentrate on assessing the students' answers, for example once a month. Students register for the exam in Sisu and gain access to the Examinarium exam. After the assessment the grading information is automatically transferred to Sisu (see the sections on Grading of examinations and Sisu instructions for more details).

Course exams in Examinarium
The course exam differs from the literature exam in that the exam period and the number of retakes are more limited. Therefore, it is recommended that the teacher divides the exam period into two parts: the course examination immediately after the course and the re-take examination after a short break. During the break, the teacher assesses the exam answers for the course examination.

Personal exam in Examinarium

In some courses, exams are prepared on a student-by-student basis (for example, maturity examinations or exams with optional literature and only few students). In such cases, students are instructed to register in Sisu and contact the examiner who will create a personal exam based on the request. The system notifies the student when the personal exam has been published and the examiner as soon as the exam is ready for assessment.

Log in to Examinarium using a browser (preferably Mozilla Firefox). If Examinarium fails to select a role for you, contact technical support at edutech @ helsinki.fi.

Creating an examination
Go to the dashboard and select ”Create new exam”.
Select the exam type:
•    General exam (open to all students who register in Sisu).
•    Personal exam (the students eligible for the examination are specified in the publishing settings).

Course

• Next, define the ‘implementation’ (i.e., the course or examination) in Sisu to which the examination is connected.
• Begin to type the course code or name until the correct course is found, and select the correct implementation from the list. You can identify the implementation by the fact that it includes not only the course code, but also the name and duration of the implementation. If you cannot find the correct implementation, contact an education planning officer.
• Examinarium only shows implementations that are ongoing or will begin in the next three months. 

Exam info

• Give the exam a descriptive name. The name does not need to include the Sisu code. When students register for an exam, they will see the exam code and name.
• If an exam can be completed in more than one language, be sure to include the title in all the languages of completion.
• The languages students can complete the exam in are listed under “Exam language”. The exam language does not affect the language of the user interface. The user can determine the UI language. You can select a number of languages if you wish. The examiner will use the language of completion at the grading stage.

Save. Complete the basic exam information. The course name, grading scale and credits of the course will be automatically retrieved from Sisu. (The number of credits awarded to the student can be changed at the grading stage.)

Exam assessment

• After being graded, "Whole course grade" examinations are automatically submitted for registration in Sisu. 
  • If an automatic export to Sisu is not requested or if the course grade is not based on the examination alone, select partial course grade as the type of examination. If a completed examination has been locked as the partial course grade, the teacher must enter the final course grade in Sisu on the Study attainments tab. 
• An examination may have multiple examiners or evaluators (an evaluator cannot save the final grade or edit the examination). To add an examiner/evaluator for an examination, they must first log in to the system. 

Attaching files to examinations and special requirements for examination computers
The specifications concerning the software needed to complete an examination limit the choice of computers the examination can be completed in. All computers have pdf-reader, Windows calculator, Wordpad ant Paint programs installed. Examinarium currently offers the following extra choices:

• ”Drawing table”: computers in Kumpula and in Viikki Infokeskus are equipped with drawing tables and Xournal++ program for drawings and calculations.
• “Finlex”, which restricts the exam rooms to K133 in Learning Center Aleksandria, where the computers allow access to www.finlex.fi
• “No Finlex”, which restricts the exam rooms to any room but the K133 in Aleksandria (no access to www.finlex.fi)
• ”Matlab, Geogebra, ChemDraw, Origin, R, R Studio”, special programs installed in some computers in Kumpula.
• ”No Matlab, Geogebra etc.” computers in Kumpula with none of the special programmes.

You can attach a file to an examination. The file is displayed to all students who take the examination.
Use a PDF file or an image to make sure the file can be opened using an examination computer.

Copying an examination
You can copy an examination by selecting the list of examinations, choosing “Copy” and then selecting a general or personal examination.
Then you need to supply the necessary examination information, including the name of the examination and the relevant course.

To create questions, go to the Questions tab or search for questions in the question library.
The question library is a collection of questions created by each teacher. You can add questions directly to the question library and use them for any exam.
If you include the same question in more than one exam, any changes to the question will be visible in all the exam (excluding scores).
You can share the ownership to a question. 

Questions are divided into sections
You can include multiple questions under each section and allow the system to display a randomised selection for each student. You can determine the number of questions displayed. All interchangeable questions must have the same maximum score.
Alternatively, you can include permanent questions in a section and have them displayed to all students who take the exam.
The system displays the exam questions on different pages according to section. Each page shows the section name in question to the student.

It is possible to create an exam with optional sections and to use optional content in the exam. Optional sections may include different book packages or lecture sections. The student chooses the section(s) during the exam booking and only the questions in the chosen sections are shown to the student during the exam. See the instructions for creating optional sections.

You can attach key words to questions and then search for them later.
You can attach a file to a question. The file must be in PDF format or an image file that can be opened using the examination computers.

Creating an essay question
Enter the question in the text field. You can edit questions using the editor function (do not add web links to questions because students can only access the exam system using the exam computers).

You can specify a recommended length for an essay answer. You can only issue a recommendation since the system does not restrict the length of students’  answers. However, students will be able to compare the length of their answers with your recommendation.
Choose a grading scale (points or Pass–Fail). If you choose score-based grading, indicate the maximum points.
You can specify the essay assessment criteria (such as a model answer) for yourself or other evaluators.

Creating a multiple choice question
You can create two types of multiple choice question:
•    One correct answer: the student can select one answer from a list of choices.
•    Several correct answers: students can select one or several answers from a list of choices and the score is based on the combined points obtained for each answer.
Write and phrase a question using the editor function.
Add answer options and points.
For “one correct answer” questions, choose the correct answer and determine the maximum score for the question.
For “several correct answers” questions, determine the points for each answer (points earned or lost)

Creating a fill-in exercise
Each question may have multiple gaps inside a text for the student to fill in. The answers may be text or numbers.
See the instructions for creating a fill in the gap assignment. 

Creating a statement question 

Statement question (can be edited) is a question type, where student can get negative points for a wrong answer which will affect the total points of the exam. 
Default answer options are: True, False and I don't know. Student will see the answer options in this order (it will not be randomized). See the instructions for creating a statement question.

Retrieving questions from the question library
You can add questions you have created previously to an exam by choosing ”Add questions from the question bank”. 
You can use the search function to find questions you have created by entering a search word or section of a question. You can also conduct a search using the course or exam title or a key word.
After you have added a question to an exam, you can edit it by selecting “Settings” and “Edit”. If the same question is included in multiple exams, any changes to the question or the answer options (excluding the score) and attached file will be applied to all the exams.

Question library

In the question library there are all the questions created by you (from all your exams), and the questions where you have been added as an owner. In the question library the examiner can also export selected questions (only essays and multiple choice questions) to Moodle on XML format.

The examination settings in the Publish tab.

Timetable
Define the exam period: The exam start and end dates. That is the period during which students can take the exam..
After a student has registered for an examination, you must submit a request to shorten the examination period to opetusteknologia @ helsinki.fi.
Please note that the examination period cannot exceed the duration of the instruction event in Sisu.
There are six alternatives for the duration of the examination: 30 minutes, 55 minutes, 1 h 25 minutes, 1 h 55 minutes 2 h 25 minutes and 2 h 55 minutes.
The exam duration does not accommodate late arrivals. An “extra” five minutes are reserved for exiting the exam room so that the next person in line can begin an exam on the hour.

If your student's have right to additional time to complete an examination, see the instructions from page "Special arrangements in examination sessions"

Publication settings
The default number of attempts is 1. The examination can be retaken after it has been graded. The examiner can also allow a retake after an interrupted examination. See the instrucions for releasing exam for retake.

Add a participant: If the exam type is “Personal exam”, include all the students entitled to reserve the exam in the list of participants.

• The EXAM users (preferred) option allows you to search by name for students who have logged into Examinarium at least once.
• If the student has not logged into Examinarium before, you can tick the box "New users" and add the student to the exam with his/her helsinki.fi address (the email address associated with their username).
• Since personal exams do not require registration in Sisu, make sure the student in question is entitled to complete the course.

After the exam has been created, click “Save and publish” at the bottom of the page to open registration for students.
If the examination is not ready to be published or you are editing a previously published examination, click “Save”.

You can preview the examination as it is displayed to students. The questions in the preview mode will be randomised.

General information
Once a week, you will receive an email round-up of the general exams in Examinarium to inform you if students have registered for your exams and if you have ungraded examinations. You can also log in to the system to check the current situation. You will always receive an email when a student has reserved a personal exam time and when a personal exam is ready to be graded.

According to the University regulations on examinations, examinations must be grade within one month. After this period, the student’s grade should be published in Sisu. If your examinations are taken infrequently, you can grade all the examinations accumulated over the course of a month in one go. The recording from the examination room is available for one month after the examination.

Study attainments tab
Log in to Examinarium using your personal credentials at https://examinarium.helsinki.fi.
The dashboard will display a list of your examinations. The columns in the table show the number of examinations waiting to be graded (Ungraded/Waiting for the final grade) and the number of students who have registered for your examinations (Reservations).

Select an examination. The “Study attainments” tab shows your examinations in the different stages of the grading process:

• Aborted exams if necessary, you can enable a retake by selecting the title and clicking “Release”. No-shows don't require releasing. 
• Submissions in the assessment process (grading has not begun or been completed)
• Assessed submissions  (a grade has been issued but not locked and the examiner can still change it)
• Locked study attainments (grading cannot be changed and the student has been informed of the grade). NOTE: Assessment process is not finished until it is locked.

Grading an examination answer
Click on the student’s name to open an ungraded exam.
At the top of the grading examinations view, you will see the student’s examination completion time and number of attempts. Below are the examination questions and the student’s answers.
You can select points for each answer from the dropdown menu (or grade it as Pass/Fail if that is the grading scale you have selected).

Feedback to the student
To give students feedback, fill in the “Give feedback” panel in the top right corner of the page. To close the feedback panel, click the red “Hide” arrow. Your comments will be saved automatically.
Finally, select a grade from the “Exam grade” dropdown menu. If necessary, you can change the exam type to (partial or whole course grade) and select the number of credits.

Grading information

• The text in the field Additional information to be sent to the study register is not visible to students or transferred to Sisu.

• If evaluators have been defined for an examination, the field Send an email message to teachers with assessments in this exam enables you to send an email message to the other evaluators, for example, when you have completed your evaluation. These messages are not archived in Examinarium.

Exam grade

• The examination grading scale is retrieved from Sisu and cannot be changed in Examinarium.
• If a) the examination comprises a partial course grade and b) you wish to use a scale differing from the one in Sisu, you must apply the evaluation appropriately. In these cases, it is important that students know how to interpret the grade indicated in Examinarium. This can also be communicated to students in writing in the feedback field in Examinarium.
• Example: If the course grading scale is 0–5 but you wish to evaluate the examination on a pass/fail scale, you can evaluate it, for example, as follows: 0=fail and 1=pass. If the course grading scale is pass/fail but you wish to evaluate the examination on a scale of 0–5, notify students of which scores correspond to which grade.

Exam type

• Partial course grades are not transferred from Examinarium to Sisu. In the case of examinations indicated as partial course grades, the teacher must compile the partial grades and enter the whole course grade in Sisu. If necessary, the settings can be changed for each examination, which means that you can lock a given student’s examination as a partial or completed course grade. You should use partial course grades in the following cases:
  • If the examination grade is not the same as the grade recorded in Sisu
  • If you wish to use Sisu’s feedback field or the Additional information for the attainment field
  • If the examination is based on an old examination connected to a course in Oodi
  • If the language of the examination is not Finnish/Swedish/English/German
  • If you do not wish to use automatic grade confirmation in Sisu for some other reason
  • When recording evaluations locked as partial course grades, you can also use SIMU, or the Sisu converter.
• Examinations locked as whole course grades are automatically confirmed when transferred to Sisu, on the following conditions:
  • Information in the examination feedback field is not transferred to Sisu. Instead, written feedback on the examination is displayed in Examinarium only.
  • Students can raise their grade by retaking the examination or taking another examination connected to the same implementation. They can also raise their grade by taking an examination connected to another implementation if the implementations are under the same assessment item in Sisu. 
  • The language defined for the studies in Examinarium is recorded in Sisu as the language of study. 
  • If an Examinarium examination appears in Sisu as a separate part of a completion method, the completed credits can be locked in Examinarium as a whole course grade, and automatic evaluation can be used. (In this case, the examination comprises a whole course grade in Examinarium, but a partial course grade in Sisu.)
  • If you receive the email message ‘Assessment transfer from Exam to Sisu failed’, it means that the completed credits have not been transferred to Sisu due to missing or erroneous information.
  • In the case of erroneous entries or other problems with grading, please contact arviointi@helsinki.fi. 

Study points (credits)

• The default number of credits for an examination is retrieved from Sisu.
• If the number of credits is based on a sliding scale, the number of credits for the relevant examination is defined in Examinarium. The grade given to a student can be raised using an automatic transfer from Examinarium to Sisu if the number of credits is the same for the retake examination. For the supplementation of credits based on a sliding scale, the examination must be locked as a partial course grade in Examinarium and recorded separately in Sisu.
• If the number of credits for the implementation is not based on a sliding scale, the number of credits must not be changed in Examinarium. If the locked number of credits is incorrect, the completed credits cannot be connected to a degree. 

Saving a grade
When the grading is complete, click “Lock the assessment”. Examinarium will display a notice to remind you that “after saving the grade you cannot change the assessment”. To confirm, click “Yes”. NOTE: only locked grades are completed.
Once grading has been locked, the student will automatically receive an email notification. Students can view their grades in the Examinarium system.
If the examination grade constitutes the grade for the whole course, locking the evaluation will automatically transfer the grade to be registered in Sisu.

After grading
Graded examinations are available in Examinarium for 12 months, after which they are archived, i.e. deleted from the system. Copy any examination answers and grades that you will need after 12 months.

Grade distribution
Students can access the grade distribution for examinations in Sisu. 

Advanced grading functions
•    Grading according to question: you can grade all answers to a specific question in one go. See the instructions for assessment by questions.
•    Speed grading: the examiner can determine the grade for multiple examinations in one go. This function is suitable for grading examination questions that are scored automatically (multiple choice and fill in the gap), according to question (essays) or by an evaluator. See the instructions for quick assessment of an exam.
•    Automatic grading: If the examination is entirely made up of automatically scored questions (multiple choice and fill in the gap), you can create percentage-based grade limits to allow Examinarium to issue a preliminary grade automatically. 

The examination room is equipped with a recording camera surveillance system that allows monitoring the examination retrospectively.
The recording may only be viewed by the relevant examiner. To review the recording, contact edutech (at) helsinki.fi within a month of the examination date.

Each examination must be connected to an ‘implementation’ (i.e., a course or an examination) in Sisu. Examinarium only shows implementations that are ongoing or will begin in the next three months.

The following information is automatically retrieved from Sisu for examinations:

• The course code and, for the course name field, the name and validity of the implementation 

To book a place in a general Examinarium examination, students must be registered in Sisu for the same implementation to which the examination is connected, and the implementation must be ongoing or begin in the future. After the implementation has ended in Sisu, the students can't see the exam anymore in their list of available exams in Examinarium.  Personal exams are also connected to the implementation in Sisu, and students should register in Sisu for the recording of the completed credits.

• Several Examinarium examinations can be connected to the same implementation (e.g., separate examinations for different language versions, a separate examination for a retake, or a personal examination for those entitled to extra time in an examination). 
• If the same examination is connected to several implementations or courses in Sisu, a separate examination must be created for each of them in Examinarium.
• If a separate implementation is created for an Examinarium examination, Electronic exam  must be selected as its type.

Have you created a course in Optime?
See the Optime instructions for administration.
If you create an instruction event in connection with an Examinarium examination, select the type “Examinarium (examination aquarium)”.
At the moment, it is sufficient that the student has enrolled for any instruction event included in the course (as long as the enrolment is valid).

Makeup and retake examinations

For makeup and retake examinations, the duration of an implementation can be extended until the examinations have been administered. Alternatively, a separate implementation can be created for a makeup or retake examination.

Personal examinations

In the case of a personal examination (an examination created separately for each student), the student must have, at some point, logged in to https://examinarium.helsinki.fi at least once, or they cannot be defined as an examination participant. The student should be advised in Sisu to log in to Examinarium and to then contact the examiner (who will create the examination for the student). Technically, a personal examination does not require registration in Sisu, but it is recommended for the recording of credits and to ensure that the student has the right to take the course in question.

Locked grades that make up the grade for the whole course are automatically registered in Sisu. If the examination grade is only part of the grade for the whole course, the teacher must round up all the grade components and then register the whole course grade in Sisu.

Examinarium features an exam visit option. This means that our students can also take an examination created in the system in another university’s EXAM facility (provided that such facility allows exam visits).

Examination answers given elsewhere are displayed in the same way to examiners as those given in the University's own examination facilities. If your examinations have any software restrictions, it might not be possible to take your examination in the facilities of other universities. Consequently, please include information about software requirements, drawing tablets or the Finlex database in the field “Instructions visible for the student during the exam booking”. 

This is how software limitations affect exam visits:

• No Finlex: no effect, students may take the examination as an exam visit
• Finlex: not in use elsewhere except in the K133 facility at Aleksandria, the examination cannot be taken as an exam visit
• No Matlab, Geogebra, etc.: the software in question may be in use in some examination facilities. It is recommended that the software is specified in the instructions displayed during exam booking and the student must check from the target university's facility information that the software in question is not in use.
• Matlab, Geogebra, ChemDraw, Origin, R, R Studio: some software may be in use in some examination facilities. It is recommended that the software is specified in the instructions displayed during exam booking and the student must check from the target university's facility information that the software in question is in use.
• Drawing tablet: may be in use in some examination facilities. It is recommended that the need for a drawing tablet is specified in the instructions displayed during exam booking and the student must ensure that drawing is possible in the facility.