As the taking the ‘ass’ out of assessment was such a good conference, I’m back with the lessons of the parallel sessions and final keynote.
What makes a good 1st year module? Anne Llewellyn
Anne talked about the usual issues for students in their first year such as expectations, self-belief, friendships and so on as well as the issues for staff mainly focused around the massification of HE. She used Ruth Pickford’s idea that 1st level assessment and feedback should focus on (taking a Jack & the Beanstalk reference) Fe Fi Fo Fun:
Jack & The Beanstalk (attribution: DaveC71)
- FEelings – around issues such as improving self-belief
- FIt – as in ‘for purpose’ so that they are aligned, empowering, interactive…
- FOrmative – ie help students improve, summative assessment should be used sparingly.
- FUN – in terms of engaging, motivating, authentic.
Some examples of good practice included having residentials, group work on authentic tasks, interaction. Reflection is critical for students to understand what is expected.
She also mentioned ways of reducing the assessment burden on staff and students. The example was using an interview to assess a portfolio. Get students to create a portfolio of evidence but rather than read through the portfolio and assess it the students have an interview which has to draw on the evidence in the portfolio. Very efficient for staff time. Also seems highly relevant for students as very akin to a job interview after completing an application form. In other words we develop and assess student skills at the same time as assessing their content knowledge.
Another key take home point was about immersion in learning and teaching and giving students the chance to experiment without it counting against them if it goes wrong.
Helping to students to be successful in assessments – Mantz Yorke
The guru of first year experience research gave the final keynote. He talked about five areas: language of assessment, expectations, standards, feedback & grades.
In HE we often talk about graduateness but what is this? Mantz partly defined it as moving from acquiescence to autonomy or from authority dependence to self-authorship.
acquiescence → autonomy
“What do you mean?”
Do we explain our regulations clearly to students? Are they sensible guidance or legal documents? Do we include examples to illustrate expectations in our assessments? Do we explain what the difference is between analysis and evaluation? The rules at university are different to school. I think one of the big ones is a deadline being a deadline. At school you tend to set interim deadlines because some won’t hand in on time and it gives everyone time to redo the assessment if it is not quite right.
Mantz made some interesting comment son standards. Saying that subject benchmarks are difficult to define and not written in the clearest manner. That Learning Outcomes are a proxy and inherently ‘fuzzy’. He then went on to look at the problem with criteria (e.g. assessment criteria etc.) Can we conceptualise criteria? What are the grade boundaries? (ie are they rather arbitrary?) How do we deal with multiple learning outcomes? Are some more important for others and therefore we get into weighting of outcomes and rules for combining bits.
What is feedback for?
We have evidence that formative feedback works (see Black & Wiliam (1998) Inside the Black Box). However, we don’t always do it well and maybe miss out aspects of formative feedback.
Typology of feedback
We tend to focus on the top left (formal teacher feedback) and should move towards the bottom right.
Mantz talked about the quantity and timing of feedback and asked “Why write feedback if it is going to be binned by students?” Feedback should be present early on and tail off. In other words, “should we give feedback on summative assessment?” We should also encourage students to establish their own internal standards so they can accurately evaluate their own work. We should also look at creative ways of giving feedback. Such as peer feedback and giving choice over how students receive feedback.
What does a grade really say?
This is one of my favourite topics. Many universities and courses still cling onto percentage marking for assessment such as essays and the like where such fine distinctions are impossible to make. What is the difference between a 65% and a 67% essay? Do we really mark from 0 to 100% or is it actually 30-80%? We also assume that the distribution of such grades tends towards the middle in a bell curve, although with learning outcomes this tends to shift to the right. We know that different subjects vary greatly in their distribution of grades. Even different modules in the same subject can have wildly different grades. If you didn’t know already, when an 18 year old asks which subject gives them the best chance of a first, tell them to study maths. Although you might want to neglect to tell them maths also gives them the best chance of failing too. Why? because in maths they use the full spectrum of percentages.
- What about programme level assessment?
- Being more intelligent about assessment, don’t assess a learning outcome more then once.
- Make assessment relevant, e.g. by including contemporary events.
- Learning leads to performance but do performance goals lead to learning? In other words, focus on the learning and a good grade will come but if you focus on getting a good grade you don’t always learn…and how do we convince the student of this?
- Are students strategic or just tactical?
The panel discussion consisted of a number of notable national teaching fellows. Here are some of the quotes that caught my ear.
“Far too much written assessment, we are now writing for a degree not thinking / performing for a degree.” “What is in people’s heads is not always what they can put on paper.” Phil Race
“Students love audio feedback.” “Give group feedback on general features of all assignments.” Bob Rotherham.
“How often is too often and how much is enough [feedback].” “Feedback should be more frequent in the 1st year.” Sally Brown
“Staff don’t always know what students need in the 1st year.” “Our system does not recognise the support 1st years need.” “Our best teachers should teach 1st years.” Ruth Pickford