'It’s no longer one strike is out for students'
The pass rate at Chemical Engineering & Chemistry shoots from 30% to over 70% with digital interim testing.
How do you get students to nearly breeze through a course with a high failure rate? By offering the material in parts via video lectures and pencasts and by testing that knowledge digitally with interim tests that students can schedule themselves when they are ready. Thanks to the BOOST program for digital education innovations, Director of Education John van der Schaaf of Chemical Engineering & Chemistry was able to redevelop a second-year bachelor’s course. “This approach gives students confidence and motivation.”
The eyes of John van der Schaaf, Director of Education at the Department of Chemical Engineering & Chemistry, light up as he recounts the reaction that he received from a student at the end of his Basic Chemical Reactor Engineering course: “She said, ‘This is the first time I’ve taken a course and finished it with the feeling that I really understand it. And I’m in the fourth quartile of the second year.’” That was a compliment but, at the same time, a worrying comment for me as a director of education.”
“Many students prepare for exams through cramming and repetition, but that doesn’t mean that they really get it. Sometimes, they memorize solutions to standard questions without a real understanding. And I suspect that this doens't just happen on this course.”
Stumbling block
When Van der Schaaf became responsible for the second-year bachelor’s course, he took over a course with a 30% pass rate. “Many students stumbled on this course, resulting in study delays and stress. My colleague said that he always started the first lecture by saying: ‘Look at your colleagues to your left and to your right. Only one of you three is going to pass this course.’ He meant that as a humorous incentive to get the students to do their best, but it backfired,” Van der Schaaf says.
I don’t believe that students are lazy and unmotivated. You have to reward them to keep studying.
John van der Schaaf, Director of Education, Department of Chemical Engineering & Chemistry
He decided to change tack, refusing to believe that the cause of the low pass rate lay with the students. “I know the prejudices: students are lazy, not motivated, can’t plan. It’s all down to the student and never the teacher because we’re the experts. I don’t agree with that. On the contrary, I believe that you have to reward them to keep studying, especially when it comes to the courses on which many stumble. You have to help them through the difficult parts.”
The main message that the director of education wants to give his students is that if you put in the time, you can pass the course. And with him, they do that in parts and mostly digitally. “It’s no longer one strike is out; you have multiple moments to test whether you understand and can apply the material. And we are there to help you.”
How does it work?
Video lectures and online exercises
After analyzing the course, Van der Schaaf arrived at five core elements that he wanted to impart on his students in separate modules. The theory is offered digitally through Canvas with short video lectures and pencasts, a type of video lecture in which students can see what the teacher writes down and explains. Online exercises are also available. There are no more live lectures.
Feedback by peers and discussion forum
On Canvas, a discussion forum for questions and feedback is managed by PhD students and students can sign up for one-on-one contact with the teacher. “They mainly use WhatsApp to ask questions to their fellow students but delegations from groups sometimes come to me if they can’t work things out among themselves.”
Digital tests on your own laptop
Each module concludes with a digital quiz as an interim test. Students bring their own laptops, which they boot up with a USB stick so that all programs and the internet are turned off. They can only access the test software and a calculator.
Direct feedback and improvement
During the digital test, they get immediate feedback on whether their answer is right and are allowed to change it. “They don’t see what’s wrong, only that it’s wrong. If they make an error in the math or their thought process, they can correct themselves.”
Feedback teacher
“At scheduled hours, students can come to the instruction room, where I cover topics on demand or discuss tests with them. What was your approach? Where did you go wrong? I give them tips, like watching a video lecture again. Or I give them one-on-one instruction. I find that to be a very valuable interaction, but it’s unfortunately not used very much.”
Highest grade counts
“Students can test themselves twice a week on their knowledge. If they are not satisfied with the grade, they can take it again. They get a total of 18 attempts and the highest grade counts,” Van der Schaaf explains. “That’s the nice thing about these digital interim tests, with no extra burden on the teacher.” Each of the five tests counts for 20%.
BOOST
Through the education innovation program BOOST, Van der Schaaf received funding to implement his plan. There had to be a secure, fraud-proof website for the digital interim tests, which students take on their own laptops using a USB stick. A studio was used to record the online lectures and lightboard sessions. “That takes the most time but, during the course, I can focus much more on the learning process and on improving the course.”
Van der Schaaf sees many advantages: “Digital testing runs smoothly and offers students a lot of flexibility. In addition, this method is easier to scale up to large groups of students.”
Most students are enthusiastic because they get clear feedback and can complete sections quickly. But there are some who would prefer one final test rather than five interim tests. “They don’t like the fact that they are forced to study regularly,” says Van der Schaaf.
It’s better to be able to fix your mistakes right away while you’re still in the material.
John van der Schaaf
Van der Schaaf sees opportunities to use his method for more engineering courses and calculus within his department, which will take place next academic year. Nine courses within CE&C will be tested digitally, such as Calculus for CE&C and Advanced Calculus for CE&C, but also more CE&C-related courses such as Introduction to Thermodynamics and Chemical Bonding and Introduction to Molecules and Processes will work with digital tests. “Then you’ll be able to do calculus in parts, for example. I hope that the learning experience of students will become more positive as they get better feedback and can correct wrong answers.”
“Under the current system, students have the option to retake, but that can only be done after the quarter and it’s an extra course that you then have to study. That creates more work pressure and stress for students because the other courses are running in the meantime and these build on what you’ve already done and should know. It’s better to be able to fix your mistakes right away while you’re still in the material. With the digital tests, you can do that immediately. The teacher is then also available in that quartile and your fellow students are also working on the subject; it’s all better coordinated.”
Warm up
Van der Schaaf is trying to warm up other departments and teachers to his method. “It’s open software; all you need is a server to run the website and the USB sticks. This could also be a fantastic method for high schools to let students test themselves to see if they understand the material well.”
What do students thinks?
“I was personally a fan of this method; it made me stop putting everything off and it gave me insight into which subjects I had not yet mastered,” says fourth-year CE&C student Sander van den Brink of the new method.
“You know that if you put in enough energy, you’re going to pass this course. That’s reassuring,” notes fourth-year student Niels Beelen. “It leaves me feeling less stressed at the end of the quarter.”
One’s own tempo
Third-year student Jochem van Ierssel likes that you can go through the material at your own tempo. “Normally, everything comes together in one exam week. I finished this course before then. If you put in the time, it works out fine. If you leave it, you’re still stressing in that last period.”
Sander feels that he had a better command of the material “because we were tested on all subjects. If you only have one exam on all of the material, not all elements come back in it.”
Points of improvement
The luck factor plays a role in the tests because they are individually generated for everyone from a database of questions, so everyone receives different questions. “I passed a few tests because I was given easy questions that I recognized, not because I had prepared particularly well,” Jochem confesses.
Sander thinks that the feedback during the tests could be better. “You immediately get a notification if your answer is wrong, but you don’t know why. You can then start guessing, but that doesn’t mean that you’ve mastered the material.”
Live lecture preferred
This problem is overcome in the system of tutor hours in which you can discuss the material with the teacher. “But it was often busy and everyone has different questions, so you have to wait a long time,” says Sander.
He is not a big fan of video lectures. “These were fine during corona, but I now prefer live lectures where I can ask questions. This helps me to focus. Looking at a screen all the time is not chill for me,” says Sander.
“Both should be offered,” adds Niels.
All courses
“Next year, this method will be incorporated into more courses within our department,” says Sander, who is also an education officer at study association Jan Pieter Minckelers.
“The idea strikes quite a chord among students,” suggests Jochem, provided that the aforementioned minus points are addressed.
Students are a bit apprehensive about more test moments per week when more courses are set up using this method. “Currently, we have test moments on Wednesdays and Fridays. If you do this for three courses, you can have one of these tests every day for the whole quarter, although you don’t have to use all of the moments, of course. But then you’re bouncing from test to test,” says Jochem.
“And then every week feels like exam stress,” concludes Sander.
Luck factor
“Students sometimes feel that the random selection of questions is unfair,” Van der Schaaf confirms. “That they were given a difficult question while someone else had an easy question. I do understand that perception, but if you apply the systematic approach to each question, the answers should be clear. And it’s the approach that is being tested.”
He understands the need for a live lecture. “We have a whole cohort of students who are each on their own path to the finish line. Not everyone is equally far in the material, which makes live lectures less efficient. The disadvantage of a lecture is that you are teaching to the middle ground: a large section does not yet understand the material and another has already passed it. With online lectures, everyone can absorb the theory at the right time, with less stress and better results. The pass rate of the course is now over 70%.”
From our strategy: about BOOST
Thanks to the BOOST program, teachers with innovative ideas that improve education at our university can apply for funding to bring their idea to life. 84 projects have already been implemented in recent years. The BOOST program will continue until the end of 2024. BOOST falls within our Strategy 2030 under the theme of Talent.
Written by
More on our strategy
