eSTEeM

Centre for Scholarship and Innovation

Evaluating Generative AI as an Academic and Emotional Aid

    Project leader(s):  Hayley Ryder

  • Theme:  Technologies for STEM learning
  • Project faculties:  STEM
  • Status:  Current
  • Date:  to

The MSc in mathematics has about 400 student registrations each year. All modules except M840 are assessed via formative TMAs and a 3-hour exam.  The capstone module M840 is assessed via a 9000-word dissertation.  About 70 students start each M840 presentation but a third passively withdraw and fail.

Many mathematics students have little experience of extended academic writing or project planning. In M840, they must learn new mathematics while developing writing skills and delivering their first substantial project, which many find overwhelming and isolating.

GenAI use is permitted on M840, and well-scaffolded use can support academic writing [N, 2024] and provide emotional benefits [Z, 2024; L, 2025]. Most tutors in a recent survey felt that GenAI is helpful when studying mathematics [S, 2025].

We propose a “study-buddy” intervention consisting of: 

  1. Resources teaching mathematics dissertation relevant iterative GenAI use, using a prompt–critique–revise cycle.
  2. Guidance on appropriately acknowledging iterative GenAI use in a 9,000-word project.
  3. Worked examples illustrating good attribution.
  4. Audiovisual informal, online lectures demonstrating the approach in practice [R, 2025].

The key intervention is Resource 4: a series of short, pre-recorded sessions illustrating how an example student uses GenAI to support dissertation planning and writing, as well as to provide emotional support.

Our goal is to improve retention by supporting structured, iterative GenAI use to reduce loneliness, provide a felt sense of support, and strengthen writing. This will produce a transferable model that promotes iterative practice over standalone prompts, and addresses gaps in current GenAI guidance by tailoring it to a longer project. 

We will evaluate using a mixed-methods design. Mid- and post-module surveys will track perceived isolation, GenAI companionship, and writing progress, alongside submission and retention comparisons with prior cohorts. End-of-module semi-structured interviews will explore study-buddy experiences and whether subject specific self-efficacy [ICL] increased by making the dissertation feel achievable. 

[L, 2025] Li, J., Li, Y., Hu, Y., Ma, D.C.F., Mei, X., Chan, E.A. and Yorke, J., 2025. Chatbot‐Delivered Interventions for Improving Mental Health Among Young People: A Systematic Review and Meta‐Analysis. Worldviews on Evidence‐Based Nursing, 22(4), p.e70059.

[N, 2024] Nyaaba, M., Shi, L., Nabang, M., Zhai, X., Kyeremeh, P., Ayoberd, S.A. and Akanzire, B.N., 2024. Generative AI as a Learning Buddy and Teaching Assistant: Pre-service Teachers' Uses and Attitudes. arXiv preprint arXiv:2407.11983.

[R, 2025] Ryder, H, O’Neil T, 2025 Use of OULive recordings of `live mathematics’ and discussion forums on a level 3 Pure mathematics module in order to enable students to move to a growth mindset in maths and to add a social dimension to learning mathematics (eSTEeM) https://university.open.ac.uk/scholarship-and-innovation/esteem/projects/themes/use-oulive-recordings-live-mathematics-discussion-forums-level-3-pure-mathematics (last accessed 10/11/2025)

[S, 2025] personal communication containing early results relating to: Steele, E, Calvert, C, Bromley, A, Identifying tailored delivery routes for ALs to guide the use of Generative AI as a study support tool (eSTEeM) https://university.open.ac.uk/scholarship-and-innovation/esteem/projects/themes/technologies-stem-learning/identifying-tailored-delivery-routes-als-guide-use (last accesses 12/11/25)

[Z, 2024] Zhong, W., Luo, J. and Zhang, H., 2024. The therapeutic effectiveness of artificial intelligence-based chatbots in alleviation of depressive and anxiety symptoms in short-course treatments: a systematic review and meta-analysis. Journal of affective disorders, 356, pp.459-469.