Inside the university AI cheating crisis(theguardian.com)
theguardian.com
Inside the university AI cheating crisis
https://www.theguardian.com/technology/2024/dec/15/i-received-a-first-but-it-felt-tainted-and-undeserved-inside-the-university-ai-cheating-crisis
112 comments
The one student who got caught for plagiarism when I was a TA was busted by turnitin. Maybe the tool isn't perfect but it caught this student who had painstakingly replaced every single word in a copied paper with a synonym. I'm talking "perhaps the utility is not flawless, but the use of this utility allowed for the apprehension of a pupil who..." etc.
I was surprised at the level of effort
I was surprised at the level of effort
So it either only caught a single cheater among some unknown number, or only a single student cheated? Neither case seems to justify the expense and hassle of subjecting everybody to this process, not to mention everything else wrong with TurnItIn.
I'm always shocked at how much students will put in effort simply not to do an assignment given. I guess this is just part of what you will get when you attach cultural and economic worth to a set of hoops to jump through [even if the intent is to have institutions of knowledge, sadly I think many students just see it as a way to get economic stability].
Unrelated to the cheating part, if you don't see American colleges primarily as a path to economic stability, I think you are probably either rich that it doesn't matter or haven't seen the bottom 25% outcomes of college graduates who didn't treat it as a financial opportunity/investment and are struggling.
More likely that they used something like quillbot or grammarly for automatic paraphrasing.
This was about 15 years ago
grammarly is pretty old, Not sure when it became available, but it was already pretty well established as a high quality writing assistant by ~2012.
It had various tools and heuristicsv to suggest different phrasings and better words that were pretty incredible if compared to anything else back then
It had various tools and heuristicsv to suggest different phrasings and better words that were pretty incredible if compared to anything else back then
I never once submitted an assignment on a course with TurnitIn turned on, it makes you agree to their terms of service, which I disagree with and I'd just sed my work to the teacher through email, explaining what was my issue with TurnitIn, and most times they'd just turn it off, which I think speaks loads about that "tool".
They're 92% accurate. Sounds good until you realize that's a false plagiarism accusation in every 20 person class.
As a current academic who uses Turnitin regularly, I find it works fairly well. There are a lot of false positives, but it also regularly identifies actual plagiarism for me.
It is a tool like anything else. From student perspective I perfectly understood reasoning for it and how useful it would be when person doing review actually cares. Being able to easily find most obvious things and then cross-reference is very useful.
Problem comes when users do not care and blindly just believe in percentage value and some threshold
Problem comes when users do not care and blindly just believe in percentage value and some threshold
"lots of false positives"
Maybe this is okay when we're talking about plagiarism and you can review manually, but humans can't reliably identify LLM written text
Maybe this is okay when we're talking about plagiarism and you can review manually, but humans can't reliably identify LLM written text
Totally agreed. I'm personally only relying on it to identify cases of material which was directly (with minimal changes) copied from a third party source without attribution.
For what’s worth (for you) the results would be the same if it identified all of them as plagiarism
No, it would definitely not. There are a number of false positives, but significantly less than all submissions. It also specifically highlights areas of concerns along with a cross-reference for me to review. I would not be able to do all this manually.
My wife is an English prof at a local University. She is tired of being put in the position of being an AI cop. She is tired of 10% students whose work was flagged
wasting 50% of her office hours trying to convince her that their work is real.
For her literature classes - where the class grade is composed of a participation grade and a written work grade - she is strongly considering getting rid of all written homework. Instead all writing work will be hand written in class.
For her literature classes - where the class grade is composed of a participation grade and a written work grade - she is strongly considering getting rid of all written homework. Instead all writing work will be hand written in class.
I've been saying for a couple of years now that academic essays are going to have to go back to being handwritten. It seems like possibly the only solution.
The job of essays isn't to produce a Word document of text that can get a grade above 70 - it's to demonstrate and improve the student's writing and thinking. The computer seems to have now completely messed this up.
The job of essays isn't to produce a Word document of text that can get a grade above 70 - it's to demonstrate and improve the student's writing and thinking. The computer seems to have now completely messed this up.
If the goal is to improve something don't make it worth 50% of your grade.
When you attach a literal degree to something being done "right" and not "demonstrated considerable improvement" you're going to get people that game the system. When you then make me pay $30,000 or more for a piece of paper you will get people that seek an edge. I did my fair share of underhanded tricks in school because I didnt want to pay another $1200 because I had to maintain a certain GPA to get where I wanted. This was over 15 years ago. Focus in the classes you care about, game the classes you don't. Its a tale as old as education itself.
Blame the OP's wife and the faculty and most universities for not only supporting but encouraging this behavior. If universities are for learning and improving the weight of grades shouldn't be career-determining and the cost should be commensurate to the expectation - both from the professor and the students.
When you attach a literal degree to something being done "right" and not "demonstrated considerable improvement" you're going to get people that game the system. When you then make me pay $30,000 or more for a piece of paper you will get people that seek an edge. I did my fair share of underhanded tricks in school because I didnt want to pay another $1200 because I had to maintain a certain GPA to get where I wanted. This was over 15 years ago. Focus in the classes you care about, game the classes you don't. Its a tale as old as education itself.
Blame the OP's wife and the faculty and most universities for not only supporting but encouraging this behavior. If universities are for learning and improving the weight of grades shouldn't be career-determining and the cost should be commensurate to the expectation - both from the professor and the students.
No one said anything about things 'being done "right" and not "demonstrated considerable improvement"'.
Improvement and meeting the rubric is all that is expected in my wife's class. My wife has significantly dropped expectations over the years she has taught. And some students still think they can just whine, cheat, and play victim to get whatever grade they want rather than just working hard.
I would add a twist to it, which is to give each student ten printed articles from which to accumulate material to handwrite the essay in class. Five or the ten would be totally unrelated, two would be strongly related, two would have a medium relation, and one would be mildly related. It's the student's job to figure out the useful sources and write the essay, all within 120 minutes. In this way, the students don't have to hallucinate content for the essay.
this is an overkill... the universities need to simply do what is done in many other countries - oral final exams. let them study all semester (as is done in many other countries) and then come to the oral exam and defend your knowledge in person.
I think we're looking at a symptom not a cause.
Essays are only ever used in an academic context. No-one outside of academia writes an essay. This use-case has been replaced by Powerpoint presentations.
Essays are obviously not the only method of "demonstrating and improving the student's writing and thinking".
If we were to write an essay, we would use an LLM to do the actual writing, and feed it a set of bullet points for the points we want to make. Same as our parents moved from handwriting to word processing, we're moving to LLMs as a writing tool.
We're also seeing (that the article didn't mention) that LLMs are being used by staff to grade student essays. Getting to that ridiculous point of an LLM writing content that only an LLM will ever read.
So why not just abandon the essay as a teaching tool? I realise that academia is slow to change, but this might force them to.
Replace it with student presentations, or whatever the actual industry the students are heading for uses to communicate.
As for academia moving away from publishing papers (which are also increasingly being written by LLMs) into journals... that will need to happen too. What replaces that is going to be interesting.
Essays are only ever used in an academic context. No-one outside of academia writes an essay. This use-case has been replaced by Powerpoint presentations.
Essays are obviously not the only method of "demonstrating and improving the student's writing and thinking".
If we were to write an essay, we would use an LLM to do the actual writing, and feed it a set of bullet points for the points we want to make. Same as our parents moved from handwriting to word processing, we're moving to LLMs as a writing tool.
We're also seeing (that the article didn't mention) that LLMs are being used by staff to grade student essays. Getting to that ridiculous point of an LLM writing content that only an LLM will ever read.
So why not just abandon the essay as a teaching tool? I realise that academia is slow to change, but this might force them to.
Replace it with student presentations, or whatever the actual industry the students are heading for uses to communicate.
As for academia moving away from publishing papers (which are also increasingly being written by LLMs) into journals... that will need to happen too. What replaces that is going to be interesting.
You just wrote an essay
>Essays are obviously not the only method of "demonstrating and improving the student's writing and thinking".
This is patently false, with many counter examples to count. Editorials, blogs, substacks entries, long posts on twitter are all essays.
This is patently false, with many counter examples to count. Editorials, blogs, substacks entries, long posts on twitter are all essays.
I don't consider a video to be an essay, and yet it is perfectly capable of improving someone's writing and thinking.
I quoted the wrong part. I intended to respond to this claim
"Essays are only ever used in an academic context. No-one outside of academia writes an essay. This use-case has been replaced by Powerpoint presentations."
"Essays are only ever used in an academic context. No-one outside of academia writes an essay. This use-case has been replaced by Powerpoint presentations."
> Essays are only ever used in an academic context. No-one outside of academia writes an essay.
ooh-a for the new way to write outrageous essays so to engage people.
ooh-a for the new way to write outrageous essays so to engage people.
youll have to take away their phones/computers in this case too. ChatGPT can write an excellent essay in less than a minute
[deleted]
the authors of this paper[0] were able to use AI to figure out if a block of text was written by AI, so hopefully this becomes a tool available to educators very soon.
[0] https://arxiv.org/html/2410.16107v1
[0] https://arxiv.org/html/2410.16107v1
AI ripped the cover off of a dirty secret of higher education:
The bulk of assignments are useless busywork using outdated formats, and educators and educational institutions haven't been motivated to change it.
Instead of realizing this and embracing the theories and models of newer education sciences, most institutions have become near ideologues with their inertia.
The bulk of assignments are useless busywork using outdated formats, and educators and educational institutions haven't been motivated to change it.
Instead of realizing this and embracing the theories and models of newer education sciences, most institutions have become near ideologues with their inertia.
I would say it is an even deeper issue of credentialism vs education.
True education doesn't need busywork but credentialism does. The busywork is the filter for the credential.
Also, if a person doesn't write essays in college how are we going to have all these authors that turn 20 pages of ideas into 300 page books? It would ruin the whole book industry. It takes a ton of practice to uncompress an idea into a coherent structure using 10X more words than needed so the book is the optimal length for sale when printed.
True education doesn't need busywork but credentialism does. The busywork is the filter for the credential.
Also, if a person doesn't write essays in college how are we going to have all these authors that turn 20 pages of ideas into 300 page books? It would ruin the whole book industry. It takes a ton of practice to uncompress an idea into a coherent structure using 10X more words than needed so the book is the optimal length for sale when printed.
Either that, or, you don't have respect for the fact that one TA has to correct the work of >30 students per class, for perhaps 10 different classes PLUS other work (like actual research), including checking for plagiarism and the like. This only works with certain formats that are characterized by a large asymmetry of work between student and teacher.
You could, of course, pay 30x more and get better service.
And then there's the total disrespect students have for just outright knowledge and practice. Being able to recite a math formularium from memory "is useless". It kind of is. But students who can do this work 5x faster than people who can't, and surely that's a good part of an education. The second thing that really matters for math is doing A LOT of exercises (like 10 intermediate to hard problems per day, ideally) over a prolonged period of time (months, minimum). That's every last exercise in a big calculus book. The exercises "are useless". Mathematica can do it, and in almost all cases, can do it better than a 40-year experience mathematician. But ...
The difference between students who've done both and students who haven't is night and day. Including the difference in using tools like Mathematica. Frankly, you can even see the difference clearly in other subjects.
You could, of course, pay 30x more and get better service.
And then there's the total disrespect students have for just outright knowledge and practice. Being able to recite a math formularium from memory "is useless". It kind of is. But students who can do this work 5x faster than people who can't, and surely that's a good part of an education. The second thing that really matters for math is doing A LOT of exercises (like 10 intermediate to hard problems per day, ideally) over a prolonged period of time (months, minimum). That's every last exercise in a big calculus book. The exercises "are useless". Mathematica can do it, and in almost all cases, can do it better than a 40-year experience mathematician. But ...
The difference between students who've done both and students who haven't is night and day. Including the difference in using tools like Mathematica. Frankly, you can even see the difference clearly in other subjects.
In an world with honorable human beings raised in Wisdom, we would understand that cheating only hurts ourselves in the long run, but that is not our world majority's ethical zeitgeist at the moment.
The lack of ethics, like all selfishness, propagates inefficiencies into the greater society, as exemplified here for our overworked and underpaid teachers.
Our son has become quite a good chess player over the past 6ish years, during which online chess cheating via engines has proliferated. I explained to him that he is completely responsible for being able to honestly say, "No." in answer to the question, "Have you ever cheated in online (or over-the-board) chess?".
Having a decision matrix that leads to ethical action confers a specific peace that accompanies knowing that one has done nothing wrong. In a world full of liars and cheats, being honest is its own very special and quite rare power, looks to me.
The lack of ethics, like all selfishness, propagates inefficiencies into the greater society, as exemplified here for our overworked and underpaid teachers.
Our son has become quite a good chess player over the past 6ish years, during which online chess cheating via engines has proliferated. I explained to him that he is completely responsible for being able to honestly say, "No." in answer to the question, "Have you ever cheated in online (or over-the-board) chess?".
Having a decision matrix that leads to ethical action confers a specific peace that accompanies knowing that one has done nothing wrong. In a world full of liars and cheats, being honest is its own very special and quite rare power, looks to me.
It could represent that the current zeitgeist doesn't believe in the system any longer and thus feels like cheating is a valid answer.
We live in an age when almost all our supposedly honest and self made heroes end up exposed as liars, grifters, cheaters, often with no shame and no consequences. The bad examples are coming from inside the house.
And their effects are multiplied because most people have zero connection with compassion, making them easily fooled by the most bald-faced of liars. We are filled with a world full of people "too stupid to know how stupid they are".
We must utilize and orient our moral compass consciously in accordance with universal compassion, or we are susceptible to the ravings of the powerful, who, as you accurately said, tend to be callously cruel liars who crave only money, power, and pleasure, at the cost of anyone and everyone.
Peace be with you.
"The Way goes in." --Rumi
We must utilize and orient our moral compass consciously in accordance with universal compassion, or we are susceptible to the ravings of the powerful, who, as you accurately said, tend to be callously cruel liars who crave only money, power, and pleasure, at the cost of anyone and everyone.
Peace be with you.
"The Way goes in." --Rumi
This is an easy to solve problem: oral exams / interview exams.
That's how grad degrees are done. But there's simply not enough time to do this at an undergrad level with potentially hundreds of students.
Unless... You have an AI listening and evaluating their oral exam.
I know it sounds like a farce. Because it is. But it might also be a proper solution.
I know it sounds like a farce. Because it is. But it might also be a proper solution.
Today’s first year undergraduate is roughly as literate as a sixth grader was a century ago. College was never meant to be a mass remedial education program and by attempting to be one it’s failing at its core mission.
All this nannyware of various sorts is just more evidence, proof even, of how few of those students should even be there.
All this nannyware of various sorts is just more evidence, proof even, of how few of those students should even be there.
Not really in my experience. That's how final PhD examinations are typically done. Outside of that, it's pretty rare. That said, PhD students are also regularly interacting with faculty members and discussing their work so it would be difficult to get by without actually knowing things.
Agreed, but you know someone will shout bias or whatever
[deleted]
Rough when you have 200 students.
Rough, but perfectly doable. It's how pretty much all my finals were done way back when. Classes of 200 and more.
Depends on the institution. If you have one faculty member for all those students and no TAs (yes, this happens) then it's going to be rather c hallenging.
Or just regular written exams.
Imagine an oral c++ algorithms and data structures exam
This isn't just essays, AI will happily output any known algorithm you ask for in a few seconds. CS coursework can be almost entirely automated in many cases
This isn't just essays, AI will happily output any known algorithm you ask for in a few seconds. CS coursework can be almost entirely automated in many cases
We need validation of GPT detection tool accuracy (false/true & negative/positive), and in what circumstances, or they are useless. Without independent information about whether something is GPT-written [1], there is no way to verify it, so they can't be verified on submitted student papers where the info is from people (students) with very strong incentives to lie (not wanting to be caught).
Here are a couple references in the article, but I would like to see a review of this question:
> Researchers at the University of Reading recently conducted a blind test in which ChatGPT-written answers were submitted through the university’s own examination system: 94% of the AI submissions went undetected and received higher scores than those submitted by the humans.
> One study at Stanford found that a number of AI detectors have a bias towards non-English speakers, flagging their work 61% of the time, as opposed to 5% of native English speakers (Turnitin was not part of this particular study).
[1] That needs to be defined: Every word? mostly? GPT-written then edited? What about ideas from the humans and writing by GPT? First draft by human and editing by GPT - what about using it as a grammar checker? For suggestions on clarifying some tricky passages? ?
Here are a couple references in the article, but I would like to see a review of this question:
> Researchers at the University of Reading recently conducted a blind test in which ChatGPT-written answers were submitted through the university’s own examination system: 94% of the AI submissions went undetected and received higher scores than those submitted by the humans.
> One study at Stanford found that a number of AI detectors have a bias towards non-English speakers, flagging their work 61% of the time, as opposed to 5% of native English speakers (Turnitin was not part of this particular study).
[1] That needs to be defined: Every word? mostly? GPT-written then edited? What about ideas from the humans and writing by GPT? First draft by human and editing by GPT - what about using it as a grammar checker? For suggestions on clarifying some tricky passages? ?
Is that even feasible? You can affect LLMs output by asking them to form response as some specific person with specific style.
Apart some hidden secret sauce by the LLM provider, detection is always going to be an arms race.
Apart some hidden secret sauce by the LLM provider, detection is always going to be an arms race.
> Is that even feasible?
If you mean, is it feasible to empirically test the accuracy of detection software when the LLM output is so varied? That's an interesting question.
Is there a way to create a representative sample of LLM output for testing purposes? That's an important question, not least because we'd also be able to define the distribution and range of outputs (unless there's a statistical way to create a sample without knowing those things?).
It goes to the question of how deterministic is the output? How predictable, even statistically?
Maybe we just work with a large collection of output. Or maybe we can create a sample of inputs (the prompts), and work from that.
If you mean, is it feasible to empirically test the accuracy of detection software when the LLM output is so varied? That's an interesting question.
Is there a way to create a representative sample of LLM output for testing purposes? That's an important question, not least because we'd also be able to define the distribution and range of outputs (unless there's a statistical way to create a sample without knowing those things?).
It goes to the question of how deterministic is the output? How predictable, even statistically?
Maybe we just work with a large collection of output. Or maybe we can create a sample of inputs (the prompts), and work from that.
Well it all depends on how hard you want to cheat. If you know nothing about the topic you basically hack the format of the essay.
If you know something you can prompt to insert some errors into logic or different perspective. Effectively making it somewhat your work too
I don't think most common LLMs are deterministic. Especially if you work on the prompt. That is not just ask Write a synopsis of Romeo and Juliet.
Out of curiosity I tried following
> Write a synopsis of Romeo and Juliet > Rewrite it in style of a teenager, insert some slight grammar errors > Write in as if teenager was trying to sound formal > more formal > make a different take, more edgy and judgemental
I have no idea how one would combat that.
> Maybe we just work with a large collection of output.
yes, But how do you get that large collection from a single student who uses non-default prompts?
Though I wonder, are we not fighting kids using calculators in a math class?
If you know something you can prompt to insert some errors into logic or different perspective. Effectively making it somewhat your work too
I don't think most common LLMs are deterministic. Especially if you work on the prompt. That is not just ask Write a synopsis of Romeo and Juliet.
Out of curiosity I tried following
> Write a synopsis of Romeo and Juliet > Rewrite it in style of a teenager, insert some slight grammar errors > Write in as if teenager was trying to sound formal > more formal > make a different take, more edgy and judgemental
I have no idea how one would combat that.
> Maybe we just work with a large collection of output.
yes, But how do you get that large collection from a single student who uses non-default prompts?
Though I wonder, are we not fighting kids using calculators in a math class?
Maybe this will finally liberate humanity from educational prestige dominating life outcomes.
What about education?
I’d never let school get in the way of my education!
I believe it: The too-clever kids in school focused on sharp quips and peer approval, which displaced real work on serious issues and actual learning.
There are a hundred lifetimes of quality education available online for free. If you learn it and can demonstrate its value or use it to create value then awesome. Not sure why we need to involve a certification process that costs $100k
We’ll be free of that as well.
I am enormously glad I went to college before these tools existed. I don't know how I would handle myself with access to them.
It depends on your goal: Do you just need that piece of paper with your name on it? Or are you going to school to really learn material?
What we do depends on much more than our goals. There's also stress, time pressure and workload, emotional problems, money issues, competitive pressure, fears (of failure, of disappointing important people, etc.), exhaustion, etc.
People don't do bad things because that's their goal but because of these issues. If you give humans an easy way to do bad things, many people under pressure will do it.
People don't do bad things because that's their goal but because of these issues. If you give humans an easy way to do bad things, many people under pressure will do it.
We all start out with good intentions. Do we all know if we would follow them to the end?
You also know that you probably shouldn't eat that extra cookie and yet...
I did undergrad before chatGPT and now I’m back in school for an MBA at a school where the school pays for all students to have access to GPT4o. Maybe because we’re all older the tools are less of a problem, but I would say they have been a net positive. GPT3 was ass and students could easily tell when a classmate used it for their group contributions. Profs seems to be about 6-12 months behind the curve in terms of the tech. All profs are aware and thinking deeply about how AI is changing how they teach.
Group papers are usually made into an outline at first and then we divvy up the responsibilities. If you use AI, most kids only care if your work sucks. AI consistently scores like an 80-85% (profs sometimes submit and blindly grade the responses), but almost always misses the core teaching points of classwork.
In my program, grades don’t matter (so employers can’t stack rank us). People who make extensive use of AI are really only cheating themselves. If you’re a big AI user, other kids generally know and try to avoid forming groups with you if they know in advance. You learn better when learning alongside others and if all someone does is dump AI slop in the google doc, you’re wasting my time in addition to yours.
I use AI to flesh out points, especially on assignments I don’t care about. It can help for idea generation and “connecting the dots” between ideas, but I always edit the output because the AI makes stylistic choices I don’t like. It’s definitely an accelerator for me when writing papers. I stand behind all the papers I’ve submitted, though some have sucked (regardless of AI usage or not).
In undergrad, when my priorities were less about learning and more towards dating/partying, I definitely would have abused this tool. At the end of the day, using the tool mainly cheats your own learning. I hope they transition to talking about using LLMs like one uses gambling - a little is fine here and there but if it’s all you do that’s a problem.
I’m not sure what to do about elementary age kids, because the AI easily writes “better” essays. At least in college I could do better than an AI if I applied myself. But in sixth grade? Good luck me. Cats out of the bag now and we should be really empathetic to the younger generations. Imagine getting slammed with TikTok->Pandemic->ChatGPT in the span of like 6-7 formative years. They are growing up differently and I certainly have no clue what we need to do to help them be successful.
Group papers are usually made into an outline at first and then we divvy up the responsibilities. If you use AI, most kids only care if your work sucks. AI consistently scores like an 80-85% (profs sometimes submit and blindly grade the responses), but almost always misses the core teaching points of classwork.
In my program, grades don’t matter (so employers can’t stack rank us). People who make extensive use of AI are really only cheating themselves. If you’re a big AI user, other kids generally know and try to avoid forming groups with you if they know in advance. You learn better when learning alongside others and if all someone does is dump AI slop in the google doc, you’re wasting my time in addition to yours.
I use AI to flesh out points, especially on assignments I don’t care about. It can help for idea generation and “connecting the dots” between ideas, but I always edit the output because the AI makes stylistic choices I don’t like. It’s definitely an accelerator for me when writing papers. I stand behind all the papers I’ve submitted, though some have sucked (regardless of AI usage or not).
In undergrad, when my priorities were less about learning and more towards dating/partying, I definitely would have abused this tool. At the end of the day, using the tool mainly cheats your own learning. I hope they transition to talking about using LLMs like one uses gambling - a little is fine here and there but if it’s all you do that’s a problem.
I’m not sure what to do about elementary age kids, because the AI easily writes “better” essays. At least in college I could do better than an AI if I applied myself. But in sixth grade? Good luck me. Cats out of the bag now and we should be really empathetic to the younger generations. Imagine getting slammed with TikTok->Pandemic->ChatGPT in the span of like 6-7 formative years. They are growing up differently and I certainly have no clue what we need to do to help them be successful.
> I hope they transition to talking about using LLMs like one uses gambling - a little is fine here and there but if it’s all you do that’s a problem.
If I hire someone to manage my money, I don't want them to do any gambling. Although given that investing is somewhat of a gamble, I at least want to set the terms and have them to disclose to me exactly how they're gambling.
If I hire someone to manage my money, I don't want them to do any gambling. Although given that investing is somewhat of a gamble, I at least want to set the terms and have them to disclose to me exactly how they're gambling.
> In my program, grades don’t matter (so employers can’t stack rank us).
Can you expand a bit on how that works? I have limited academic experience, so I’m fascinated. Does everyone end up with a 4.0 if they pass, or..?
Can you expand a bit on how that works? I have limited academic experience, so I’m fascinated. Does everyone end up with a 4.0 if they pass, or..?
Internally we get grades. fail, low pass, pass, good, excellent. I tend to get high marks (good+), but there are forced percentages of what grades are awarded. Maybe 15% get excellent? Nobody fails unless you are super truant. Kids hunting for investment banking jobs are advised to study for interviews and not go to class (by peer advisors not the school). They all still pass lol
Grades correspond to a GPA internally that may account for scholarship or whatnot. Mean raw score for some classes is like a 90+. Most grade differentiation comes from class participation. We can pull true GPA from the school if we dig, but we are explicitly told not to share GPA with employers. I’ll share my undergrad grades, but they hardly matter. As far as I can tell, no one shares their grades and employers are expected not to ask for them. This is a top 10 MBA program in the USA, I understand that other programs are very similar.
Grades correspond to a GPA internally that may account for scholarship or whatnot. Mean raw score for some classes is like a 90+. Most grade differentiation comes from class participation. We can pull true GPA from the school if we dig, but we are explicitly told not to share GPA with employers. I’ll share my undergrad grades, but they hardly matter. As far as I can tell, no one shares their grades and employers are expected not to ask for them. This is a top 10 MBA program in the USA, I understand that other programs are very similar.
If I could wave a wand to wish away all LLM's I would do it in a heartbeat. More harm than good.
I, and pretty much all the rest of the class was accused of AI-plagiarization due to TurnitIn flagging it. Since our given task was to write about some niche article quite specific to our profession, TurnitIn flagged any mentions of author names or article titles because some class in the UK had received a similar assignment, where they apparently had to write about the same article.
Therefore by doing as you were instructed and referencing to the article as instructed you would fail the task automatically.
Therefore by doing as you were instructed and referencing to the article as instructed you would fail the task automatically.
How do students that cheat with AI pass the exams?
A lot of university courses don’t have exams. Rather, the students are graded based on projects, papers, or presentations.
I’ve been teaching at a university for more than 20 years, and only a few times have I given students a final exam. I used to assign final papers, but I stopped doing that in the spring of 2023, when ChatGPT was becoming widely known and I couldn’t decide how to deal with its use by students.
The big issue, in my opinion, is that AI can be used productively and ethically in education; it can be used to cheat; and there’s a huge gray area between those two extremes, where there doesn’t seem to be any consensus yet.
For example, suppose a student uses Claude to brainstorm topics, then chooses one from among those topics and researches it in depth, then does some more brainstorming with ChatGPT based on what he or she found, and then jots down what he or she wants to say as bullet points. Finally, Gemini is used to write a paper that presents the information in those bullet points in a logical and well-formed manner, and the student checks the paper and makes revisions before turning it in. Is that okay?
When I’ve discussed this issue with humanities faculty, they’ve generally regarded that kind of AI-assisted writing to be cheating. Science faculty, however, have been more receptive to it, as they care more about the accuracy and originality of the ideas than about how the words are strung together. My sample was small, though, and I am sure that there are different opinions on both sides.
I retired from my full-time university post in March 2023. Now that I’m teaching only one class a semester, I have each student do a final one-on-one presentation and interview with me, which makes up a large part of their grade. I’m able to do that because I’m teaching only that one class. If I had a full teaching load with a hundred students or more total, there’s no way I could interview each one individually.
I’ve been teaching at a university for more than 20 years, and only a few times have I given students a final exam. I used to assign final papers, but I stopped doing that in the spring of 2023, when ChatGPT was becoming widely known and I couldn’t decide how to deal with its use by students.
The big issue, in my opinion, is that AI can be used productively and ethically in education; it can be used to cheat; and there’s a huge gray area between those two extremes, where there doesn’t seem to be any consensus yet.
For example, suppose a student uses Claude to brainstorm topics, then chooses one from among those topics and researches it in depth, then does some more brainstorming with ChatGPT based on what he or she found, and then jots down what he or she wants to say as bullet points. Finally, Gemini is used to write a paper that presents the information in those bullet points in a logical and well-formed manner, and the student checks the paper and makes revisions before turning it in. Is that okay?
When I’ve discussed this issue with humanities faculty, they’ve generally regarded that kind of AI-assisted writing to be cheating. Science faculty, however, have been more receptive to it, as they care more about the accuracy and originality of the ideas than about how the words are strung together. My sample was small, though, and I am sure that there are different opinions on both sides.
I retired from my full-time university post in March 2023. Now that I’m teaching only one class a semester, I have each student do a final one-on-one presentation and interview with me, which makes up a large part of their grade. I’m able to do that because I’m teaching only that one class. If I had a full teaching load with a hundred students or more total, there’s no way I could interview each one individually.
Outside of labs and a capstone project course, I don't think I had a single class across a dual-degree prorgam that didn't have a midterm and a final. There was a single class that I remember that had a take-home final, and a single class that was open notes for the exams. For most humanities we had in class essays. For foreign language classes there were written components and interviews with TAs. For engineering/math there were tons of math problems. Even a C programming class had a written exam (where we had to write correct programs by hand). This was ~10-15 years ago in a public university in the US.
I graduated before LLMs but after WolframAlpha and I essentially cheated my way through calculus I and II. Lenient grade weighting made 90s on the homework and 60s on the exams enough to slide through with a C. Funnily enough, now that it's over a decade later and I know more about myself and my neurodivergent patterns, I feel much more able and interested in actually learning calculus. I'm looking forward to seeing the pedagogical changes that result from LLMs enabling this sort of trickery for all subjects.
[deleted]
If you use AI exclusively I assume they won't. But I feel like most people who use it, use it to save some time - like they have the general idea of what they want to say, and just need a little help polishing it into a final form.
Like that joke about how you write a summary, use AI to expand it, then the recipient uses AI to get a summary. It's no longer a joke - it's really happening.
I've also seen students use AI to understand material - instead of reading the material, they feed it through AI to explain it to them. This is either a mediocre student who'll have a hard time if they ever need to do original research, or it's a bad teacher. Either way they'll do OK on the exams.
Like that joke about how you write a summary, use AI to expand it, then the recipient uses AI to get a summary. It's no longer a joke - it's really happening.
I've also seen students use AI to understand material - instead of reading the material, they feed it through AI to explain it to them. This is either a mediocre student who'll have a hard time if they ever need to do original research, or it's a bad teacher. Either way they'll do OK on the exams.
A few universities scrapped exams during Covid and didn’t bring them back, focusing on coursework instead
there is no crisis. stop teaching students to write trash, using ai or not. if the essay is good, it should get an A.
Just came to the same conclusion with my professor father in law. If AI can actually do a good job at writing some assignment then people shouldn't be bothering with that thing. If it can't it will not matter that it was written by AI it will just be bad.
Also our universities are failing us if the only thing we get at the end is a boolean piece of paper (or even one represented by a floating point number).
This is like servers at McDonald's complaining that the customers are cheating with trash cans because they aren't eating the meals they buy, just throwing them in the trash. Why would they do that?!?!? Because someone told them they need to go to McDonald's "for the experience"?
Also our universities are failing us if the only thing we get at the end is a boolean piece of paper (or even one represented by a floating point number).
This is like servers at McDonald's complaining that the customers are cheating with trash cans because they aren't eating the meals they buy, just throwing them in the trash. Why would they do that?!?!? Because someone told them they need to go to McDonald's "for the experience"?
Is content the only thing that matters? What if the whole class submits the exact same essay? What if the next class submits the exact same essay too?
As ex Oxford academic tutor in a college, I was never asked to assess for plagiarism, but this was a time of 1:3 (max) tutorials, very easy to identify students not putting the effort or struggling.
However, as scientific writer and conference/journal reviewer I once spotted a plagiarism using the crude detector (I believe it was Turnitin) embedded in one of the ACM portals we used to use for conferences. The “bad luck” of the submitter was they had plagiarised by copy paste 100% of one of my previous papers :’)
However, as scientific writer and conference/journal reviewer I once spotted a plagiarism using the crude detector (I believe it was Turnitin) embedded in one of the ACM portals we used to use for conferences. The “bad luck” of the submitter was they had plagiarised by copy paste 100% of one of my previous papers :’)
We’ve built a solution in conjunction with a university to this problem that is pretty low effort to implement, but very few professors can be bothered to even try it out (the apathy and red tape is unreal). Honestly, it has been disheartening that distribution is so tough, as the results have been great for those who are using it.
Could you share the product and some details about it? I'm curious.
Presumably it's: https://inktrail.co
As I understand it, it records when you're writing, including all edits and such, and verifies it's human based on that. Well, see the demo at https://inktrail.co/in-action
It will probably work well right now, but I don't know how easy that would be to fool once the hucksters build tools to circumvent it.
As I understand it, it records when you're writing, including all edits and such, and verifies it's human based on that. Well, see the demo at https://inktrail.co/in-action
It will probably work well right now, but I don't know how easy that would be to fool once the hucksters build tools to circumvent it.
> very few professors can be bothered to even try it out
Do you happen to know if there is some overlap between these professors and professors who also refuse AI-detectors? Apathy could be the reason but I wonder how much of it is driven by cynicism encouraged by the inefficacy of AI-detecting tools.
Do you happen to know if there is some overlap between these professors and professors who also refuse AI-detectors? Apathy could be the reason but I wonder how much of it is driven by cynicism encouraged by the inefficacy of AI-detecting tools.
Well that's terrible news. Currently building a product for the same market (completely different solution if the nephew comment is correct about your product). I'm already not thrilled to be in ed-tech selling to instructors with admin's money. I thought at least the instructors would have some enthusiasm to solve the problem. Shit.
As a professor would be interested in a solution to this problem, I'd be curious to see hear more details. FERPA issues can sometimes make it difficult to adopt solutions where student info of any kind needs to be sent to a third party.
Would you please share more details about this?
Prompts need to be integrated into grading/eval of where students are mentally. Like - cite/attach your prompts to the paper, code, whatever.
I think that there will be people that will only learn to copy LLM generated text, and they will be not able to look like people that learn.
Unnoted real evil -- most grading being a zero-sum game
Ergo, if you have some students who use AI tools (and accept that those tools produce better papers and/or produce them with less effort) then those students will be ranked higher than students who study equally hard, are equally smart, and don't use AI tools.
Which then collapses simply down to more $ = better grades and higher class rank.
That's bullshit.
There have always been tutors, study groups, etc. that offer varying degrees of academic aid, but the broad accessibility of AI writing aids is novel.
Ergo, if you have some students who use AI tools (and accept that those tools produce better papers and/or produce them with less effort) then those students will be ranked higher than students who study equally hard, are equally smart, and don't use AI tools.
Which then collapses simply down to more $ = better grades and higher class rank.
That's bullshit.
There have always been tutors, study groups, etc. that offer varying degrees of academic aid, but the broad accessibility of AI writing aids is novel.
Why cannot this be solved by written exams?
> “What if I’m doing an MA, or in a job, and everyone got there just by cheating…”
This is absolutely the problem here. The work is done, the answers are right, the report is the best anyone has seen, but oh no a fancy new tool was involved so its CHEATING! This is so friggin stupid. When i get my work done I am responsible that it comes out right. When its right, thats just it. - it's right. No matter how i got there it's fine. But wait, what if i used some freelance platform to farm out my job to another person then turn in their work? Well that's breach of NDA and my terms of employment. So what, are the student over here committing wage theft of some kind even though they are the ones paying for tuition? Make this make sense to me, you can't. School is a job you pay to do so you can learn to get good at a job you get paid to do. Somehow school ends up being a bad investment and we get mad at the students? Schools are broken and useless and at this point i would prefer my doctor was great at using chatgpt and checking the results carefully over being taught at an educational institution.
This is absolutely the problem here. The work is done, the answers are right, the report is the best anyone has seen, but oh no a fancy new tool was involved so its CHEATING! This is so friggin stupid. When i get my work done I am responsible that it comes out right. When its right, thats just it. - it's right. No matter how i got there it's fine. But wait, what if i used some freelance platform to farm out my job to another person then turn in their work? Well that's breach of NDA and my terms of employment. So what, are the student over here committing wage theft of some kind even though they are the ones paying for tuition? Make this make sense to me, you can't. School is a job you pay to do so you can learn to get good at a job you get paid to do. Somehow school ends up being a bad investment and we get mad at the students? Schools are broken and useless and at this point i would prefer my doctor was great at using chatgpt and checking the results carefully over being taught at an educational institution.
Why do we teach kids how to do basic arithmetic by hand? After all, calculators are basically ubiquitous at this point. Why waste the time teaching a skill that's not necessary as an adult?
The reason is because doing arithmetic by hand teaches kids how math actually works, while using a calculator just teaches them how to find the answer. This base-level understanding is crucial for understanding more advanced math. But it's also important for making mental estimations, or just having a basic grasp on how the world works.
This is why students using ChatGPT is a problem. If you never put in the actual work in the first place, you never gain any actual understanding. A doctor "great at using ChatGPT" won't know how to check the results carefully if they've relied on ChatGPT for their entire education. Having a baseline skill set is crucial for evaluating LLM output, and for handling situations that LLMs can't handle. And there will always be situations LLMs can't handle, regardless of how advanced genAI becomes.
The reason is because doing arithmetic by hand teaches kids how math actually works, while using a calculator just teaches them how to find the answer. This base-level understanding is crucial for understanding more advanced math. But it's also important for making mental estimations, or just having a basic grasp on how the world works.
This is why students using ChatGPT is a problem. If you never put in the actual work in the first place, you never gain any actual understanding. A doctor "great at using ChatGPT" won't know how to check the results carefully if they've relied on ChatGPT for their entire education. Having a baseline skill set is crucial for evaluating LLM output, and for handling situations that LLMs can't handle. And there will always be situations LLMs can't handle, regardless of how advanced genAI becomes.
> Why do we teach kids how to do basic arithmetic by hand?
I think kids waste way too much time doing hard arithmetic by hand. As a particularly egregious example, I visited a Montessori primary school where they demonstrated what is essentially a complex single player board game, for calculating square roots by hand. It's a process whereby a 5th grader would take ~10 minutes to calculate the root of a 3 digit number, by moving dozens of pins around. It looked so silly, and while other Montessori equipment offers geometrical intuition, this one seemed to offer none. Squaring with trial and error would have been so much easier.
Other more typical examples are of kids spending hundreds of hours practicing long multiplication and long division of e.g. 4 digits, and then getting punished and stressed out about having made silly mistakes. While in the real world, no one in their right mind would choose to perform these calculations by hand any more, and would instead use a tool incapable of making arithmetic mistakes.
I don't have a clear solution, but would tend towards an educational system that teaches only the very basics of performing each mathematical operation (probably up to the level the average literate adult would do mentally), and then focus the rest of the time on problems where the students are the ones posing the question, using a calculator/computer to solve it, and then check whether the answer makes sense in the original context.
EDIT: For those curious about it, I just found a blog post demonstrating the logic behind this square root peg board. The author of the blog argues that it does help build understanding, but from what I saw, I would politely disagree.
https://borisreitman.medium.com/square-roots-the-montessori-...
I think kids waste way too much time doing hard arithmetic by hand. As a particularly egregious example, I visited a Montessori primary school where they demonstrated what is essentially a complex single player board game, for calculating square roots by hand. It's a process whereby a 5th grader would take ~10 minutes to calculate the root of a 3 digit number, by moving dozens of pins around. It looked so silly, and while other Montessori equipment offers geometrical intuition, this one seemed to offer none. Squaring with trial and error would have been so much easier.
Other more typical examples are of kids spending hundreds of hours practicing long multiplication and long division of e.g. 4 digits, and then getting punished and stressed out about having made silly mistakes. While in the real world, no one in their right mind would choose to perform these calculations by hand any more, and would instead use a tool incapable of making arithmetic mistakes.
I don't have a clear solution, but would tend towards an educational system that teaches only the very basics of performing each mathematical operation (probably up to the level the average literate adult would do mentally), and then focus the rest of the time on problems where the students are the ones posing the question, using a calculator/computer to solve it, and then check whether the answer makes sense in the original context.
EDIT: For those curious about it, I just found a blog post demonstrating the logic behind this square root peg board. The author of the blog argues that it does help build understanding, but from what I saw, I would politely disagree.
https://borisreitman.medium.com/square-roots-the-montessori-...
Obviously there's a balance to be struck. There are certainly schools and teachers that teach arithmetic poorly, and there's a practical limit to the amount of hours spent doing equations before it becomes tedious busywork.
But I still think it's important that kids are taught how to do arithmetic by hand. And part of that will involve repetition, both to help reinforce and practice skills, and so educators can more accurately gauge a student's ability. You can't just bypass the need to sit down and do the work, even if the exact amount of work necessary is debatable.
But I still think it's important that kids are taught how to do arithmetic by hand. And part of that will involve repetition, both to help reinforce and practice skills, and so educators can more accurately gauge a student's ability. You can't just bypass the need to sit down and do the work, even if the exact amount of work necessary is debatable.
This argument can be extended downwards infinitely. You can argue against teaching addition and writing of any form with this mindset.
I also think this argument means that theoretically someone with a 6th grade education and a 12th grade education should have the same economic output, as they both know how to read and add/multiply numbers. In real life the reason someone drops out before/during high school is always extenuating life circumstances, so the outcomes are much worse for them. But in a world where that didn't happen, do you think the outcomes would be essentially identical?
I also think this argument means that theoretically someone with a 6th grade education and a 12th grade education should have the same economic output, as they both know how to read and add/multiply numbers. In real life the reason someone drops out before/during high school is always extenuating life circumstances, so the outcomes are much worse for them. But in a world where that didn't happen, do you think the outcomes would be essentially identical?
There are too many confounding variables. But yes, I think that in a future world where kids who grow up in a stable family can choose to easily move between schooling and the real world (including work experience, travel, alternative education and just the occasional slacking off), you'll see improved outcomes for those who do these other things.
> what is essentially a complex single player board game
Perhaps this sounds silly, but I think that’s actually deeply interesting. There is a profoundly deep relationship between calculation and games. This applies not just to ordinary arithmetic, but also things like game theoretic proofs to prove the soundness of logical inference rules. Exposing children to that relation, even with a toy case, sounds like it could be fruitful.
Perhaps this sounds silly, but I think that’s actually deeply interesting. There is a profoundly deep relationship between calculation and games. This applies not just to ordinary arithmetic, but also things like game theoretic proofs to prove the soundness of logical inference rules. Exposing children to that relation, even with a toy case, sounds like it could be fruitful.
This seems like it's replying to a misunderstanding of their comment.
> A doctor "great at using ChatGPT" won't know how to check the results carefully if they've relied on ChatGPT for their entire education.
They didn't rely on ChatGPT for their education; they relied on ChatGPT for finishing homework. If they go over the generated essay and make sure it doesn't have any inaccuracies and they consistently do so correctly, what's the problem?
> A doctor "great at using ChatGPT" won't know how to check the results carefully if they've relied on ChatGPT for their entire education.
They didn't rely on ChatGPT for their education; they relied on ChatGPT for finishing homework. If they go over the generated essay and make sure it doesn't have any inaccuracies and they consistently do so correctly, what's the problem?
The purpose of most educational productions is not to produce some correct thing but to undergo a process of learning required to produce the thing.
And now academia needs to adapt since they can't just assign out all the learning to the students. All this is doing is exposing how broken the entire concept of higher-ED learning as been for quite some time.
But this was always the case. When I was studying chemical engineering, it was an open secret that all the students were cheating. There was a giant google drive with all the solutions manuals, old tests, old programs, lab reports, whatever you wanted.
That's why pencil and paper tests were worth 80% of the grade.
That's why pencil and paper tests were worth 80% of the grade.
> Well that's breach of NDA and my terms of employment. So what, are the student over here committing wage theft of some kind even though they are the ones paying for tuition?
Students are typically agreeing to submit original work. It's a breach of the terms they agreed to, so I'm not sure it's so different.
> i would prefer my doctor was great at using chatgpt and checking the results carefully over being taught at an educational institution.
What is checking over the results carefully going to do if your doctor has never been taught medicine?
Students are typically agreeing to submit original work. It's a breach of the terms they agreed to, so I'm not sure it's so different.
> i would prefer my doctor was great at using chatgpt and checking the results carefully over being taught at an educational institution.
What is checking over the results carefully going to do if your doctor has never been taught medicine?
Even before the 'AI detection' tool (IMHO snake-oil) the features around copy-pasting/plagiarism were so bad I always ignored them. The tool would flag things like commonly used coverpage forms. That universities pay a fortune for those tools is an indictment of the industry.