Some student-professor pairings lead to "unusually effective teaching" (and it's possible to predict which ones)

Video trailers can be used to predict which
lecturers are the best teachers, and which
students they are especially suited to.

In the near future, students could be presented with a series of video trailers of different professors at their university. Based on their ratings of these videos, the students will be paired with the professors who provide the best fit. The outcome will be superior learning, and greater student satisfaction.

That's the promise of a new study that asked 145 psychology undergrads to rate 6-minute teaching videos of 10 different professors, and then to rate their experience of an actual 40-minute live lecture with those same professors taken several weeks later. The students were also quizzed on the content of those lectures to see how well they'd learned.

Jennifer Gross and her colleagues explain that student evaluations of professors are made up of three key factors: each professor's actual ability (this component tends to correlate across ratings given by different students); each student's rating bias (this component correlates across the ratings given by the same student to different professors – for example, some students are more lenient in their ratings than others); and relationship effects.

This last component that is one of the key points of interest in the new study. It pertains to the specific fit, or not, between a professor and a student. When there is a good fit, this leads to unusually high ratings by that student for the professor, above and beyond what you'd expect given the student's usual rating bias, and given the level of ratings the professor usually attracts.

To zoom in on these relationship effects simply requires factoring out each student's rating bias, and each professor's average rating across students.

The exciting finding is that the researchers were able to use the students' ratings of, and their mood during, the 6-minute trailers to forecast how they later rated the actual lectures, including predicting which professors got the highest average ratings after the lectures, and predicting relationship effects.

This result is important, the researchers explained, because the students' memory for material taught in a given lecture was independently related both to that lecturer's average ratings (some lecturers are better than others), but also to the specific relationship effects (i.e. whether the student in question had given that lecturer unusually high ratings – the sign of a good professor/student fit).

"These findings support the possibility of developing online systems that would provide personalised recommendations that specific students take courses from specific professors," the researchers said.

However, they acknowledged that their results need to be replicated, and they also outlined some limitations of the study. This included the fact they'd carefully compiled the 6-minute trailers to showcase each professor's teaching style (a time-consuming endeavour); that the live evaluations involved just one lecture rather than an entire course; and that the professors' teaching skills were confounded with the topics they taught.
_________________________________

  Gross, J., Lakey, B., Lucas, J., LaCross, R., R. Plotkowski, A., & Winegard, B. (2015). Forecasting the student-professor matches that result in unusually effective teaching British Journal of Educational Psychology, 85 (1), 19-32 DOI: 10.1111/bjep.12049

--further reading--
Engaging lecturers can breed overconfidence
Is it time to rethink the way university lectures are delivered?

Post written by Christian Jarrett (@psych_writer) for the BPS Research Digest.

Read More

Threat of punishment makes us better judges of our own knowledge

People show better understanding of
their own knowledge when threatened
with large penalties for wrong answers. 

There are some walks of life where trying to be right as often as possible is not enough. Just as important is having insight into the likely accuracy of your own knowledge.

Think of doctors and surgeons making diagnostic decisions. They can't be right all the time, and neither can they be completely certain over their judgments. What becomes important then, is that they have an accurate sense of the reliability of their own knowledge. Psychologists call this "metacognitive accuracy".

Michelle Arnold and her colleagues have performed the first ever test of whether people's insight into the accuracy of their own knowledge is affected by whether they are tested under conditions of high reward for correct answers, or high penalties for wrong answers.

Across two studies, dozens of undergrads attempted to score as many points as possible on a series of multiple-choice general knowledge questions. Crucially, for each question, they were compelled to state their favoured answer, and then whether they wanted to submit it or not for scoring (with points gained for correct answers, and points lost for wrong answers). This two-stage process gave the researchers insight into the students' accuracy of their own knowledge. Withholding answers that would have been incorrect and submitting correct answers would be a sign of high metacognitive accuracy.

The key finding was that the students showed more metacognitive accuracy when they were tested under high punishment conditions in which wrong answers led to a loss of four points, whereas correct answers gained just one point. This is compared to a baseline condition in which correct answers earned one point, and wrong answers lost one point; and compared to a high reward condition in which correct answers earned four points, and wrong answers lost just one point.

Under threat of high punishment, it makes sense that the students were more cautious in the answers they chose to submit. But critically, their submission of correct answers was only moderately reduced, whereas their withholding of wrong answers was significantly increased. In other words, the threat of high penalty for wrong answers had the effect of improving the apparent insight the students had into the accuracy of their own knowledge.

This is a new area of study and the effect needs to be tested in other contexts and over longer time periods. However the real-life implications are exciting. As the researchers concluded: "the positive impact of punishment on strategic regulation may be ripe for widespread application, especially in areas where training effective metacognitive discrimination is vital - such as medicine and business." It will be particularly interesting to see whether repeated testing under high penalty conditions - imagine medical students taking diagnostic exams in this format - leads to lasting, beneficial changes in people's insight into their own knowledge.  

_________________________________

Arnold, M., Chisholm, L., & Prike, T. (2014). No pain no gain: The positive impact of punishment on the strategic regulation of accuracy Memory, 1-8 DOI: 10.1080/09658211.2014.990982

--further reading--
The Mastermind effect - psychologists boost students' knowledge using priming placebo
Unleash the crowd within
When are two heads better than one?

Post written by Christian Jarrett (@psych_writer) for the BPS Research Digest. 

Read More