Marker's Black Box

home \| buy now \| marking \| support \| contact us
Frequently asked questions page Q. When giving written feedback to students, is it OK to criticise their work? A. Marking should be constructive and if you say something is incorrect you should also say why and how it can be corrected. You should avoid negative comments as they tend to reduce motivation. Give advice on how to make improvements. Avoid making comments about the student's effort and motivation, as your inference may be incorrect and demotivating. Q. How can I motivate (and avoid demotivating) students in my written feedback? A. To motivate students, be explicit about how improvements can be made. End comments could also be used to motivate the student, such as by highlighting the strong points of the work. Also, try not to notate everything you notice on the student's script itself as over-notated work can make students feel overwhelmed. Written comments should agree with the mark awarded – it frustrates students if all comments are positive and good but they only have a mark of 50%. Q. Who can use Marker's Black Box? A. Anyone who assesses work and is required to provide written feedback on performance can use Marker's Black Box. It is especially useful for tutors at university, teachers at school, and other learning or training environments. Q. How much time can I save using Marker's Black Box if it takes me 20 minutes to mark one item? A. Marker's Black Box can save time in a number of ways. Firstly, as remark selection is automatic, written feedback is instant. So whatever fraction of that 20 minutes is taken producing the written feedback, Marker's Black Box can cut that time down considerably. However, you will find that it's not just that it saves you time but it helps you provide much more feedback than you would otherwise. The application will also save time collating marks if it is interacting with Excel as this is also automated. Q. Can Marker's Black Box improve inter-marker reliability? A. An assessment should have strong inter-marker reliability. Marker's Black Box can be used by several tutors to mark the same course. This way they will be using the same criteria, with the same definitions, and the same criteria weighting scheme. Q. Can Marker's Black Box be used just for marking essays? A. Marker's Black Box can be used for marking essays, research reports, seminar presentations, poster presentations, practical work, portfolios, and dissertations. It can be used to provide detailed written feedback or just to grade student work. It can also be used to write references for students. Q. Apart from marking what else can Marker's Black Box be used for? A. Marker's Black Box can be used for writing personal references for students applying for employment or further study. It could also be used for writing personal appraisal reports. Q. How many marking criteria can I use in Marker's Black Box? A. You can use up to 20 marking criteria in Marker's Black Box. Each one can be named and assigned a weight that determines its contribution to the final grade. Q. Does Marker's Black Box give ‘sensible' marks? A. You can adjust the maximum mark (when all criteria are marked at 5) and the 'minimum mark' (when all criteria are marked at 1). This may yield more realistic marks, as marks above 90% and below about 20% are seldom awarded. Q. In Marker's Black Box, how are remarks generated? A. When grading against a criterion, a remark is selected automatically and is randomly chosen from the remark bank for that mark and that criterion. You can click 'Try again!' to randomly select an alternative remark or you can view all of the remarks for that mark and that criterion and select one from the list. Any remark generated can be edited and saved or it can be edited and saved as a new remark. Q. Can Marker's Black Box read a list of students in an Excel file? A. Yes, you can open a list of students on the course from an Excel file and select each student's name (or ID) during marking. When you create the marker's report, the mark awarded is saved into the same Excel file. You can also save the marks awarded for each criterion so that you can analyse a break down of the marks. You could compare different markers to see if, and in what way, their marking styles differ. Q. Can I create a collection of remarks that I frequently use? A. For a particular assignment, there may be a large number of remarks that the marker would like to use over and over during marking. You can create these by selecting "View Assessment-specific Remarks". Q. Can I obtain a list of criteria and their definitions from Marker's Black Box? A. It is possible to copy all criteria and their definitions to the clipboard for pasting into a word processor, if they are to be used externally, such as printing them for students to see the marking criteria being used. To do this, select 'Display All Criteria and their Definitions' from the 'Tools' menu. Q. Doesn't the text created by such a system read in a rather impersonal or disjointed way? A. An interesting feature is its automatic sentence linking option. The application makes a note of the sequence of marks given for each criterion and can select a link word on the basis of whether the subsequent mark is higher than, less than, or equal to, the previous mark. Sentences linked in this way have good flow to them and text reads in a more coherent way. Also, if you include a general, grammatical, or assignment-specific remark, you can enter a page number to refer to the page in the student's work. The remark includes the page reference (e.g., 'see page 12'). This makes it more personal and the student sees exactly what you are referring to in the script. Q. Can I use existing remarks from one marking template for a new marking template? A. When creating a new marking template, all of the remarks for a criterion from another template can be imported into the current template. Select: 'Tools > Import Remarks from Another Template' Q. How do I share template files with my marking colleagues? A. You can email a template file to fellow markers very easily. From the menu options selecte 'Tools > Email Template to Colleague'. An email is created using your default email application with the files associated with the template added as attahcments. All you have to do is to click 'Send'. Q. How does Marker's Black Box increase inter-marker reliability? A. Marker's Black Box can increase inter-marker reliability because all marking criteria are made explicit to each marker. This means that the markers are all aware of the basis for marking each item and that no criterion can be ignored during marking. Marker's Black Box also increases inter-marker reliability because all criteria are weighted in exactly the same way for each marker. When marking without this application, one marker could give a greater weighting to certain criteria than other markers and this is undesirable. Q. What if my colleague wants to mark at home and away from her computer? A. If one member of a marking team prefers to use an alternative method of marking, they can use offline marking sheets. Select 'Tools' > 'Create Offline Marking Sheet'. If you have a question, why not ask us here? Next >>

Frequently asked questions page

Q. When giving written feedback to students, is it OK to criticise their work?

A. Marking should be constructive and if you say something is incorrect you should also say why and how it can be corrected. You should avoid negative comments as they tend to reduce motivation. Give advice on how to make improvements. Avoid making comments about the student's effort and motivation, as your inference may be incorrect and demotivating.

Q. How can I motivate (and avoid demotivating) students in my written feedback?

A. To motivate students, be explicit about how improvements can be made. End comments could also be used to motivate the student, such as by highlighting the strong points of the work. Also, try not to notate everything you notice on the student's script itself as over-notated work can make students feel overwhelmed. Written comments should agree with the mark awarded – it frustrates students if all comments are positive and good but they only have a mark of 50%.

Q. Who can use Marker's Black Box?

A. Anyone who assesses work and is required to provide written feedback on performance can use Marker's Black Box. It is especially useful for tutors at university, teachers at school, and other learning or training environments.

Q. How much time can I save using Marker's Black Box if it takes me 20 minutes to mark one item?

A. Marker's Black Box can save time in a number of ways. Firstly, as remark selection is automatic, written feedback is instant. So whatever fraction of that 20 minutes is taken producing the written feedback, Marker's Black Box can cut that time down considerably. However, you will find that it's not just that it saves you time but it helps you provide much more feedback than you would otherwise. The application will also save time collating marks if it is interacting with Excel as this is also automated.

Q. Can Marker's Black Box improve inter-marker reliability?

A. An assessment should have strong inter-marker reliability. Marker's Black Box can be used by several tutors to mark the same course. This way they will be using the same criteria, with the same definitions, and the same criteria weighting scheme.

Q. Can Marker's Black Box be used just for marking essays?

A. Marker's Black Box can be used for marking essays, research reports, seminar presentations, poster presentations, practical work, portfolios, and dissertations. It can be used to provide detailed written feedback or just to grade student work. It can also be used to write references for students.

Q. Apart from marking what else can Marker's Black Box be used for?

A. Marker's Black Box can be used for writing personal references for students applying for employment or further study. It could also be used for writing personal appraisal reports.

Q. How many marking criteria can I use in Marker's Black Box?

A. You can use up to 20 marking criteria in Marker's Black Box. Each one can be named and assigned a weight that determines its contribution to the final grade.

Q. Does Marker's Black Box give ‘sensible' marks?

A. You can adjust the maximum mark (when all criteria are marked at 5) and the 'minimum mark' (when all criteria are marked at 1). This may yield more realistic marks, as marks above 90% and below about 20% are seldom awarded.

Q. In Marker's Black Box, how are remarks generated?

A. When grading against a criterion, a remark is selected automatically and is randomly chosen from the remark bank for that mark and that criterion. You can click 'Try again!' to randomly select an alternative remark or you can view all of the remarks for that mark and that criterion and select one from the list. Any remark generated can be edited and saved or it can be edited and saved as a new remark.

Q. Can Marker's Black Box read a list of students in an Excel file?

A. Yes, you can open a list of students on the course from an Excel file and select each student's name (or ID) during marking. When you create the marker's report, the mark awarded is saved into the same Excel file. You can also save the marks awarded for each criterion so that you can analyse a break down of the marks. You could compare different markers to see if, and in what way, their marking styles differ.

Q. Can I create a collection of remarks that I frequently use?

A. For a particular assignment, there may be a large number of remarks that the marker would like to use over and over during marking. You can create these by selecting "View Assessment-specific Remarks".

Q. Can I obtain a list of criteria and their definitions from Marker's Black Box?

A. It is possible to copy all criteria and their definitions to the clipboard for pasting into a word processor, if they are to be used externally, such as printing them for students to see the marking criteria being used. To do this, select 'Display All Criteria and their Definitions' from the 'Tools' menu.

Q. Doesn't the text created by such a system read in a rather impersonal or disjointed way?

A. An interesting feature is its automatic sentence linking option. The application makes a note of the sequence of marks given for each criterion and can select a link word on the basis of whether the subsequent mark is higher than, less than, or equal to, the previous mark. Sentences linked in this way have good flow to them and text reads in a more coherent way. Also, if you include a general, grammatical, or assignment-specific remark, you can enter a page number to refer to the page in the student's work. The remark includes the page reference (e.g., 'see page 12'). This makes it more personal and the student sees exactly what you are referring to in the script.

Q. Can I use existing remarks from one marking template for a new marking template?

A. When creating a new marking template, all of the remarks for a criterion from another template can be imported into the current template. Select: 'Tools > Import Remarks from Another Template'

Q. How do I share template files with my marking colleagues?

A. You can email a template file to fellow markers very easily. From the menu options selecte 'Tools > Email Template to Colleague'. An email is created using your default email application with the files associated with the template added as attahcments. All you have to do is to click 'Send'.

Q. How does Marker's Black Box increase inter-marker reliability?

A. Marker's Black Box can increase inter-marker reliability because all marking criteria are made explicit to each marker. This means that the markers are all aware of the basis for marking each item and that no criterion can be ignored during marking. Marker's Black Box also increases inter-marker reliability because all criteria are weighted in exactly the same way for each marker. When marking without this application, one marker could give a greater weighting to certain criteria than other markers and this is undesirable.

Q. What if my colleague wants to mark at home and away from her computer?

A. If one member of a marking team prefers to use an alternative method of marking, they can use offline marking sheets. Select 'Tools' > 'Create Offline Marking Sheet'.

If you have a question, why not ask us here?

Next >>