How will academic writing be learned, taught and examined at universities in the future? How can students be made fit for the skills of tomorrow?
These questions are answered by the MWK-funded research and e-learning project KI@helpBW, which is being carried out at the writing lab together with the KIT library. It is developing an online course (OER, CC BY-NC 3.0) that introduces students to the critical approach to writing with text-generating AI in a comprehensive manner and in compliance with good scientific practice. The aim of this e-learning course is to conduct interdisciplinary research into how students use AI when writing and (can) act ethically in doing so. On the other hand, the aim is to provide teachers with examination forms that are suitable for evaluating texts under the conditions of the availability of AI.
The KI@helpBW network works across all higher education institutions with the involvement of local libraries so that all students and teachers in Baden-Württemberg can benefit from it.
Funding period: 01.03.24 – 28.02.27
Network partners: All locations of the Baden-Württemberg Cooperative State University, Mannheim University of Applied Sciences, Stuttgart Media University, Freiburg University of Education, Heidelberg University of Education, Karlsruhe University of Education
Online course Researching and writing with text-generating AI
Please note that you must be registered at https://opencourses.kit.edu/ in order to access the course.
Sub-module 1.1 Why they don’t think and we talk to them anyway – How LLMs work
- Why do LLMs generate good answers?
- How do LLMs get a good sentence?
- How do LLMs get information?
- Why do LLMs sometimes have to give wrong answers?
- How we talk to LLM
- Why LLM is not actually artificial intelligence
Sub-module 1.2 Why I would never use AI, but you should still learn about it in this course
- Doctor AI
- AI at the border
- ‘Moral outsourcing’
- Torture for minimum wage
- The real data garbage
- Why people die of thirst from AI
- Not every AI user is racist, white and heterosexual
- AI makes people poor and unemployed
- Why populists use
Introduction
- Why you should not use AI in all phases of research
- creative, bibliographic and full-text work
Sub-module 2.1 Preparation and implementation
- Sensible use of AI tools in preparation (creative work: search term table, search strings)
- Application examples
- Problematization of the use of AI tools in preparation
- Sensible use of AI tools in execution (bibliographic work: finding sources)
- Application examples
- Problematization of the use of AI tools in execution
Sub-module 2.2 Evaluation and administration
- Sensible use of AI tools in evaluation (full-text work: screening, selecting sources)
- Application examples
- Problematization of the use of AI tools in evaluation
- Sensible use of AI tools in administration (managing sources)
- Application examples
- Problematization of the use of AI tools in administration
Sub-module 3.1 How students use AI in academic writing
- Have a scientific text written
- Have notes and keywords formulated
- Have outlines created
- Have parts of the text edited
- Have stylistic weaknesses edited
- Have spelling and grammar corrected
- Have non-scientific components of a scientific text written
- Have AI as a writing process aid
- Have AI as a discussion partner
Sub-module 3.2 Prompting
- Learning a language for a system that speaks?
- How does the AI do what I want it to do?
- What elements can a prompt consist of?
- Application example: Formulating an abstract with AI
- Specifying the answer length
- Creating a specialist context
Assigning a role - Saving, revising and passing on prompts
Sub-module 3.3 Text editing with AI
- Science is not complicated
- Active formulations read better
- Writing confidently and clearly
- Why scientists don’t always write in the present tense
- Repetition is boring
- Highlighting models and lines of research
- Formulating criticism of research
- Building on research
- Stylistic finalization of headings
Sub-module 4.1 Communication in the fog
- Introduction to the topic (based on: Communication model according to Shannon & Weaver supplemented by text-generating AI)
- Interview with experts
- Reflection tasks
Sub-module 4.2 Rhetoric trainer or bore?
- Introduction to the topic: Tasks involved in preparing a rhetorical speech
- Interview with experts
- Reflection tasks
Sub-module 4.3 Can AI coach my writer’s block?
- Introduction to the topic: Writing types in times of text-generating AI
- Interview with experts
- Reflection tasks
Forms of examination
Independently of the development and research of the online course, the KI@helpBW project provides teachers with academic examination forms that enable the assessment of texts independently of the use of AI by students.
In addition to the actual writing task, each of these examination forms includes instructions for students and a didactic handout for teachers.
Essay writing
We understand an essay to be ‘a short text in which a consideration or argument is developed in a generally comprehensible form.’
The point of reference is the Anglo-Saxon tradition following Francis Bacon, which emphasises ‘the explanatory and argumentative structure of the essay’. (Frank, A., Haacke, S., & Lahm, S. (2013). Schlüsselkompetenzen: Schreiben in Studium und Beruf (2nd, updated and expanded edition). Verlag J.B. Metzler, p. 175.)
Formalities:
- Length: Two pages on a specific specialised topic.
- Inclusion of sources, figures and tables (depending on the subject area)
Our working hypotheses:
- Lack of originality, sprawling text, lack of nuance and vague language suggest that the present text originates from an AI. (Semrl, N., Feigl, S., Taumberger, N., Bracic, T., Fluhr, H., Blockeel, C., & Kollmann, M. (2023). AI language models in human reproduction research: Exploring ChatGPT’s potential to assist academic writing. Human Reproduction, 38 (12), 2281-2288, here 2286.)
- The relationship between an independent voice and an argumentative point of view is a central evaluation criterion.
- A convincing essay can only be created through a comprehensive process of reflection.
The essays are assessed according to the following criteria:
- Depth of argumentation, structure and composition
- Degree of complexity
- Conciseness
- Balance of subjectivity and objectivity
- Originality and element of surprise
Input:
- What is an essay?
- Introduction to academic argumentation
- Supporting prompt strategies
- Instructions: Have essays written during your studies.
- (Linguistic) assessment catalogue based on the researched assessment criteria
- Orientation aids for identifying argumentation and subjectivity markers
KI-Writing Diary
- AI writing diary as a table in which students enter prompts entered, prompt output and a reasoned evaluation of the AI use, among other things
- AI writing diary also shows whether and how AI output was checked
- AI writing diary structures the student’s documentation process
How can the use of text-generating AI be documented in such a way that it becomes a form of examination?
What forms of student use of text-generating AI are there?
- Students document their forms of utilisation when using AI
- Students are free in the way they document their utilisation
- Handout provides guidance on how the documentation process can be managed by students
- Handout shows what should be documented
- Flexible customisation options for the writing task
Nachgelagerte Textüberprüfung
- Task: ‘Revise the form and style of your own text, written without the help of AI, using AI.’
- No specifications regarding length, level, form, etc.
- Students submit both the original text and the revised text and receive feedback on their revision.
Can the use of AI lead to a more intensive examination of one’s own text and to an improvement in text form (editorial competence)?
- Essence of the content from sub-module 3.3:
Introduction to editorial work with AI
Structure prompts
Style prompts
Paraphrase prompts - Input on problematising the separation of form and content when dealing with AI
- Assessment grid for evaluating the formal and stylistic revision
- The grid is based on the honest revision options with AI presented in the instructions for students:
Text structure: organisation of text sections, argumentation, division into meaningful sections, chapter headings
Style: sentence structure, active and passive voice, modal verbs, subjunctive, tense, nominal style, general language level (academic language, technical language), text flow
Citation: completeness of references, correct presentation of research content, functional integration into the text
Prüfungsgespräch
- Oral defence of an academic text written with the help of AI
- Examination discussion is observed and documented using a guideline
Is it possible to determine from an examination discussion whether students have independently developed and understood the content of a scientific text written with AI?
- Guide for examiners adapted to students’ needs
- Insight into what is relevant in the examination interview
- Basic principles of argumentation
- Strategy and discussion prompts to prepare for the examination interview with AI
- Guidelines on how to conduct an examination interview, incl. range of questions:
Content (technical terms, context, …)
Argumentation
Use of relevant technical terms (use of language)
Sources - Reflection sheet:
Points of the guidelines and additionally
Degree of interactivity
Irritation
Process
Team
