Reshaping AI for education
How Cambridge is putting people at the heart of the generative AI revolution
When ChatGPT 3 was released a year ago, the large language model (LLM)-trained bot sparked excitement, fear and hype among educators and researchers.
Would it unleash a wave of cheating? Would teachers and academics lose their jobs? Is an education revolution coming?
Within months, OpenAI’s tool became the fastest-growing consumer software application in history, gaining over 100 million users, with tech giants like Google, Meta, Microsoft and Apple scrambling to launch or adapt their own LLMs.
What does generative AI mean for education and research? How is Cambridge responding?
AI ready
As the world’s attention turned to generative AI, Cambridge University Press & Assessment had some early advantages.
"We don't have all the answers on AI and education, but we're well-placed to help move the needle."
The organisation has deep links with University of Cambridge researchers, while drawing on a breadth of education expertise and experience that touches 100 million learners worldwide. Cambridge people have been engaging with AI, machine learning and data science for years - as have researchers around the world.
“At Cambridge, we weren’t starting from scratch.”
"Our networks of teachers, students, researchers and technologists bring important insights as we all work to make AI more effective," says Helena Renfrew Knight, Director of Strategy and Integration.
For a group of Cambridge linguists, computer scientists and education experts, the arrival of ChatGPT felt more like a vindication of the years they had already invested in developing AI, language models and data science to help English language learners.
The Cambridge University Institute for Automated Language Teaching and Assessment (ALTA), launched a decade ago, brings together teams from computing, engineering, linguistics and language assessment. Their multidisciplinary academic, teaching and assessment expertise in machine learning, text and speech processing has been improving how English is taught around the world for years.
The fruits of their work are not just in the R&D or experimental stage. By the time ChatGPT launched, their innovations were already in use in products reaching millions of students around the world – and had been for years.
Helena Renfrew Knight, Director of Strategy and Integration
Helena Renfrew Knight, Director of Strategy and Integration
Human in the loop
Linguaskill epitomises such advances. The quick and convenient AI-powered online test helps organisations check the English speaking, reading, writing and listening levels of candidates. Linguaskill may use AI, but it puts people first.
As Dr Evelina Galaczi, Director of Research for English, explains: “One of our AI-powered innovations in Cambridge is the way we mark our Linguaskill online English test. The test uses hi-tech features but keeps a human in the loop."
"AI will transform English language learning and assessment. We will make sure the technology is applied in the right way, supporting teachers and examiners."
“The Speaking module, for example, uses AI based auto-marking as part of a hybrid marking approach," adds Dr Galaczi. "An auto-marker is used in live assessment, but with the expertise of a trained human examiner who steps in where the automarker is not completely confident it has given the correct mark. This advanced marking system leverages the unique strengths of tech and humans, enabling reliable results to be awarded within 48-hours.”
Teachers and technologists
“Teaching and learning are all about people, and AI can't replace the social and emotional aspect of learning.”
One-third of UK teachers use AI tools to help with their work, a figure that has doubled within five months, according to a Teacher Tapp poll of 9,000 teachers and leaders.
Language teachers around the world appear to be particularly engaged with AI tools. A separate Cambridge study, in collaboration with the University of Bedfordshire, found 71 percent of surveyed teachers are using GenAI tools weekly or monthly.
In spite of formidable access to technologists, teachers are only becoming more central to Cambridge’s engagement with generative AI.
The International Education group is running focus groups with 80 teachers working on the challenges around GenAI, to get a better picture of their hopes, fears, ideas and solutions.
Rod Smith, Group Managing Director for International Education, sees this as a top priority: “We are working with our 10,000-strong community of schools worldwide as they engage with this rapidly evolving technology. That will mean embedding the knowledge, skills and conceptual understandings of AI into an adaptable and cutting-edge curriculum that supports a fairer future.”
Jill Duffy, Group Managing Director for UK Education, agrees: “We have to harness generative AI, data science and online learning in a more human-centred way. That means pinpointing where technology can allow teachers and students to achieve more, rather than simply displacing some of their work.”
Historic head-start
At the end of a corridor in the Triangle building, is a bright, clean, secure warren of temperature-controlled rooms that host an extraordinary resource: the archives of Cambridge examination and assessment materials, dating back to 1858.
The assessment archives, in partnership with Cambridge University Library, house exam papers including the first Cambridge-delivered overseas exams (in Trinidad in 1864), the first sat by women (from an 1865 “initial trial period”), and the first GCSEs and A levels.
The assessment archives, in partnership with Cambridge University Library, house exam papers including the first Cambridge-delivered overseas exams (in Trinidad in 1864), the first sat by women (from an 1865 “initial trial period”), and the first GCSEs and A levels.
Alongside vital paper originals, they have digital copies of exams dating back to 1990.
Since 2007, every exam paper and anonymised student answer has been digitised.
This Digital Script Repository is proving fundamental to Cambridge’s rapid experimentation with generative AI in the race to equip students and teachers with better tools.
Signals and noise
Nick Raikes and his team of data scientists have seized the opportunity to use AI and machine learning (ML) to spot patterns in how students respond to exams and in the marks and feedback which examiners and teachers give.
“In our initial research into automatic marking of short, free text answers to science questions, we found we needed several thousand marked answers for training. We are now seeing how the latest large language models will let us mark even more accurately with fewer answers for training.
"We are exploring with teachers how we can best use AI to provide them with insights on the common misconceptions and mistakes which students make in their exams," says Nick Raikes.
With Cambridge’s large data sets, common errors, misunderstandings and fallacies can shift from unevenly understood anecdotes into clear patterns. That can be turned into dynamic and personalised advice to students, teachers and markers.
“It’s a way to better see what students can do, rather than what they cannot,” says Helena Renfrew Knight, Director of Strategy and Integration.
Even relatively simple LLMs offer tantalising possibilities, says Nick Raikes: “We may be some way from a general-purpose AI that could mark questions it hasn’t been trained on, but we can get quite accurate marking for specific questions by training a machine learning model on a sample of free text answers that have been marked by human examiners.”
This puts us on the cusp of AI co-pilots, empowering examiners to mark papers more rapidly, accurately and without some of the drudgery of one- or two-mark questions or straightforward answers that are clearly right or wrong, enabling teachers and examiners to refocus on more nuanced challenges, using their expert judgment.
Big data
2.5 million
Number of Cambridge results in GCSEs, IGCSEs, A levels and vocational qualifications delivered to students around the world in August 2023 alone.
82 million
Images scanned and uploaded for that August 2023 exam series.
4.6 million
Number of completed exam papers, or ‘scripts’, marked for August 2023.
3.1 million
Multiple-choice answer sheets for August 2023.
"Millions of real student answers are perfect for training ML."
The Cambridge English Corpus, comprises billions of words of written and spoken British and American English taken from news, literature, emails, textbooks, the web, conversations, broadcast media, presentations, speeches, lectures, film and television. It's big data in action.
The Corpus helps keep the language used in Cambridge products, services and technologies relevant, natural and up to date.
Several Cambridge teams are exploring ways to use AI and LLMs to accelerate this work, and even more effectively draw-on these precious human-generated resources.
In a culture where hackathons are encouraged, Cambridge colleagues from Manila to Madrid, are playfully experimenting and innovating with such resources.
Hallucinations and confabulations
The Cambridge Corpus informs the Cambridge Dictionary, the world’s most popular English learners’ dictionary.
The Corpus’ relevance and speed, drawing on human insight alongside machine learning, enabled Cambridge Dictionary lexicographers to spot how generative AI is changing how we speak and write in English.
That influenced this year’s choice for Cambridge Dictionary Word of the Year: hallucinate.
AI tools, especially those using large language models (LLMs), have proven capable of generating plausible prose, but they often do so using false, misleading or made-up ‘facts’. They ‘hallucinate’ in a confident and sometimes believable manner.
Wendalyn Nichols, Cambridge Dictionary’s Publishing Manager, said: “The fact that AIs can ‘hallucinate’ reminds us that humans still need to bring their critical thinking skills to the use of these tools.
“At their best, large language models can only be as reliable as their training data. Human expertise is arguably more important – and sought after – than ever, to create the authoritative and up-to-date information that LLMs can be trained on.”
Hallucinations matter, argues Dr Henry Shevlin, an AI ethicist at the University of Cambridge.
Dr Henry Shevlin said: “Hallucinations involve factual errors and are usually recognizable, like misstating London as France's capital. Yet as we depend more on AI for decisions, we face murkier challenges in areas like politics and ethics, where right answers aren't so obvious. Seeking AI advice on what to study or whom to vote for delves into what philosophers call normative territory, beyond simple scientific facts.”
"Human expertise is arguably more important - and sought after - than ever."
Guidance and ethics
Ethical concerns came to the fore in early 2023. Academics entered uncharted territory as they began to use new AI tools in their research.
"Guardrails are important - particularly with GenAI."
"We need to be sure that what we are trying is safe, ethical and that we protect the rights of authors and creators", says Helena Renfrew Knight. "It's important that researchers and others exploring the potential of GenAI can access support and guidance."
In early 2023, some academic publishers allowed scholars to publish research papers that listed ‘ChatGPT’ as a co-author.
Amid this, Cambridge University Press brought clarity through new guidelines.
This really mattered. 400 peer-reviewed Cambridge journals and 1,500 research monographs, reference works and higher education textbooks were relying on the guidance. As were many readers, given the 114 million downloads of Cambridge scholarly materials last year.
The Cambridge principles for generative AI in research publishing set out how AI could be used ethically in scholarly works.
AI tools would be banned from acting as co-authors on Cambridge publications. After all, it’s harder to hold a chatbot to account than it as a human. But Cambridge also allowed scholars to use generative AI, updating enduring and long-established standards on authorship, plagiarism, transparency and accountability.
Caltech political scientist Professor R. Michael Alvarez, who uses AI to detect online harassment, trolling and abusive behaviour, said: "I appreciate the leadership Cambridge University Press is taking to outline guidelines and policies for how we can use these new tools in our research and writing. I anticipate that we will be having this conversation about the opportunities and pitfalls presented by generative AI for academic publishing for many years to come."
Enduring values
Cambridge has always used technology for education and research. From the printing press to online learning and digital assessment.
“Each generation of new technology empowers educators in a different way and Cambridge has been at the forefront of this relationship for centuries,” says Peter Phillips, Chief Executive of Cambridge University Press & Assessment.
Peter Phillips, Chief Executive of Cambridge University Press & Assessment
Peter Phillips, Chief Executive of Cambridge University Press & Assessment
Talk to the organisation’s experts and leaders and, even as they are steeped in rapid technological development, enduring principles come through.
“Quality and integrity will come first, whatever technology we use.”
“It is our engagement and confidence across the full spectrum of technology and education, rigorously applied, while adeptly managing the limits and risks, that marks what Cambridge can bring,” adds Peter Phillips.
In some areas, perhaps paradoxically, technology has only reinforced long-established methods.
Writing in the Independent, OCR Chief Executive Jill Duffy observed: “We now know what it’s like to sit exams in the age of generative AI. As it turns out, it isn’t so different from the pre-ChatGPT era. You cannot summon an AI chatbot to an exam hall to take an exam for you. Students are still equipped only with a pen and their own wits.
“If anything, the rise of AI over the past few years has reinforced the value of in-person and human-supervised exams. They are still the best protection against cheating. They put students on a level playing field. They allow students to show what they know and what they can do.”
But that should not slow down efforts to adapt technologies to deliver better assessment. We have “a huge opportunity to get the balance right: to take the best of tech, including on-screen exams and generative AI learning materials, while strengthening tried and tested methods.”
So where does an exam board head see this going?
“About half of parents and students would prefer a mix of on-screen assessment and pen and paper, according to an Ofqual poll,” notes Duffy. “This is the hybrid future for exams that we can, and should, move towards. It will involve some generative AI to support teaching and revision, digital exams alongside pen and paper ones, and people being ever more central to assessment: students, teachers and examiners."
Putting people first
“In everything we do with AI, we’re putting people first, and ensuring everything is checked by humans. This technology can, and must, serve researchers, readers, students and teachers.”
“We know where technology cannot match what people offer. AI cannot replace the teacher skilled in questioning to develop individual students’ understanding; who can respond to pupils’ well-being needs; or who can tell when a student is producing their own work using clues that no large language model will be trained to recognise.”
Generative AI tools are only as good as the content that feeds into their large language models. And it’s human-created rather than synthetic data that matters most.
That puts Cambridge at a unique advantage, says Mandy Hill, Managing Director for Academic: “It’s an exciting time, but also one that reminds us why human generated content and ideas will endure."
"In an age of AI hallucinations or confabulations, there’s arguably even more value in original human work.”
Mandy Hill, Managing Director of Cambridge University Press
Mandy Hill, Managing Director of Cambridge University Press
Experimentation and focus
As Cambridge teams embark on new pilots in generative AI, they can draw on the extraordinary reach and expertise that comes from being part of one of the world’s great universities and most influential education organisations.
In this environment each team is, in the words of Peter Phillips, “remaining completely focused on supporting students, teachers and researchers.”
As the AI revolution progresses, that focus may be Cambridge’s greatest advantage of all.
By Andrew Scheuber, Sophie White, Alana Walden and Annie Zhang.
