This spring semester, we experimented in the Robotics and Autonomous Vehicles Research Group with what happens when a student’s practical assignment is reviewed by an AI assessment assistant before the lecturer. The goal was simple: faster feedback for the student and optimization of the lecturer’s time for what truly matters, a meaningful discussion. Below is an overview of what we did, what worked, and where we stumbled.
The experiment involved two third-year courses in the Product Development and Robotics (EARB) curriculum: “Internet of Things” and “Autonomous Vehicles”, with a total of 67 students. Both courses are built on the principle of continuous assessment and focus largely on interconnected practical assignments. The final grade is determined by the total score of practical work reports: one course has four reports, the other three; some are completed as group work, but most are individual.
Until now, assessment has followed the traditional route. The student uploads a report to Moodle (hopefully on time), the lecturer assesses it, usually within a couple of weeks, and provides feedback; the student improves the work, and the lecturer assesses it again. We have allowed this to be repeated for as long as the student wishes to improve their score. For the lecturer, this means a considerable workload, especially in the case of low-quality work that clearly does not meet the requirements, and after making improvements the student often has to wait weeks for feedback.
The solution tested this semester was designed precisely to alleviate these bottlenecks. We created a rapid prototype that allows a student to submit their work to an AI assessment assistant for evaluation before sending it to the lecturer. In fact, from the beginning of the semester this was a rule: every assignment first had to pass the AI assessor and receive a positive result before the lecturer would even start reviewing it. Our threshold has always been the same: the work must achieve at least 51% of the maximum score in order to be considered in the assessment. We prepared very specific assessment criteria and a systematic prompt for the AI assessor that covered both course-wide and report-wide rules. The student could submit their revised report to the AI as many times as they wished until they were satisfied with the score and it exceeded the threshold.
The student then arranged a time with the lecturer to defend the assignment. In previous years, we did not require oral defenses of reports; everything was based on written work. Since the lecturer no longer has to perform the labor-intensive initial assessment, we added an oral defense instead: the student and lecturer meet for a short discussion during which the lecturer can quickly determine what the student has actually learned from the course. This both accelerated the student’s initial feedback and freed up the lecturer’s time for face-to-face defenses.
In general, the pilot was successful: both lecturers and students considered this combined model the most preferable for the future. Overall, 90% of respondents evaluated the solution positively and as supportive of meaningful learning. Nevertheless, there were shortcomings. The biggest of these was the variability of AI assessment; similar assignments could receive different scores. Lecturers pointed out that AI is unable to meaningfully analyse graphical material or connect it with text, for example linking system models with program code. As an important nuance, it was also noted that students may be tempted to start optimizing their reports for the AI assessor.
The inclusion of AI in assessment is probably inevitable, especially as we also encourage students to use AI in their studies: in report preparation, idea generation, and other activities that support the learning process. It is entirely natural that we do the same to make the lecturer’s work more efficient and convenient. The most critical aspect, however, is that the substantive learning process, the acquisition of knowledge and skills, does not suffer but instead grows.
In poorly designed AI-supported education, there is a real risk that the student lets AI produce the entire report, the lecturer’s AI assessor evaluates it, and two AIs communicate with each other while the lecturer and the student remain spectators. To avoid this, the learning process in the AI era must be carefully designed so that AI is an integrated tool on both sides, while direct interaction between lecturer and student and meaningful learning actually increase. This is not easy. Returning to pencil and paper and purely oral examinations will not save the university or the course. It may work temporarily, but it will not take us very far.
Instead, the role of the lecturer must be reshaped: less mechanical dissemination of information and control of its reproduction, and more development of attitudes, critical thinking, and learning ability. To achieve this, it is necessary to experiment and shape the attitudes of students, lecturers, and leadership alike in order to bring the best practices into real teaching.
The development of the experimental software was supported by the Engineering Academy EARB programme.