Comment by honksillet 2 months ago
I don’t think college profs really have any idea the degree of cheating going on right now. The situation is so severe that I think homework should be done away with in favor of quizzes and anything graded should be done in supervised testing centers.
Why does the following obvious solution not work: - Homework is just voluntary. You have to force yourself to study anyways. Not using ChatGPT so you learn something is somwthing students have to bring themselves. - Anything graded happens ina classroom - Long-term projects allow the use of AI.
I had a Calc II professor like that in college. He told us on the first day, "I don't take attendance, and I don't grade homework. If you want to pass the class you'll attend and you'll do the homework on your own."
Long story short, the vast majority of the class attended, did the homework, and still failed anyway. He was known for being... unrelenting and awful. If women went to his office for help during office hours, he wouldn't help them... one of those professors.
> LLMs radically accelerate the learning process.
Absolutely false, at least for students as someone who has to deal with a lot of students. They learn nothing from pasting in a homework problem into ChatGPT.
Even for professionals, looking at my colleagues I'm not convinced AI tools are doing anything other than making them dumber and lazier. They just throw whatever at the AI, blindly trust it and push through with it without looking at the output for a millisecond before making it someone else's problem.
When considering which qualities to favor in people, I'd be happy if you consider this quote from the 1950 movie Harvey:
"Years ago my mother used to say to me, she'd say, 'In this world, Elwood, you must be' - she always called me Elwood - 'In this world, Elwood, you must be oh so smart or oh so pleasant.' Well, for years I was smart. I recommend pleasant. You may quote me."
> LLMs radically accelerate the learning process.
Can't agree with that. IME and from what I've read in many places, it's basically only useful if you already know the subject. If you don't, you have no idea if what it spews out is correct or not, and you completely skip the part where you actually use your brain.
> As a hugely important side note, we should be focusing more on how to support low intelligence people so their shortcomings aren't a burden to themselves and a drain on society.
Completely agree with that, although I don't think LLMs will help with it at all.
This guy is literally advocating for Nazi eugenics. Is this the kind of content that’s OK on this website now?
Given the downvotes, guess there are plenty of people here that are pro-eugenics and support thinning the herd of “low IQ individuals” lest they reproduce.
They may be advocating for that, but I'm not against them doing so because it gives the rest of us the opportunity to present the arguments against it.
I take this view lately because I've noticed that younger generations are starting to take up ideas that my grandparents and parents were vehemently against, because they'd either experienced those things or they'd listened to the arguments. As those people die out, and because we naively think that some argument are settled once and for all, we stop presenting them and thus, people get sucked in by the bad stuff.
So I say let them say it, and let us argue back and never forget what we find from these arguments.
> I teach CS, and oh we know but I don't know what to do about it.
You could give students larger projects and have them present their homework.
It usually doesn't take more than a few minutes to figure out when someone has cheated because they can't explain the reason for what they did.
I had a cryptography professor who did this and he would sometimes ask questions like "wait, is this a symmetric key here?" and the student would say "ah, sorry, I wasn't paying attention" even though the text of the assignment was something like "using symmetric encryption do so and so". Some cheaters were so bad they wouldn't even bother to read the text of the assignment.
Also, people who cheat tend to equivocate when asked questions. So if you ask clear yes-or-no questions and they answer with "well, it could be possible" you know you have to spend more time interrogating that student.
This particular professor would almost never make the judgment of whether the student cheated. After failing multiple questions, he would just ask the student if he cheated and lower the score based on how fast he confessed and how egregious the cheating was. Most cheaters would fold quite quick, but some took longer.
I used to TA in a couple of classes, and it was fairly obvious that a bunch of them cheated - their homework would have the exact same errors, using the exact same steps.
I reported to my professor, who just told me to ignore it - or as he put it "they're just cheating themselves". Exams were written exams (that counted for 100% of the grade) with no help, so you could spot a bunch of students who'd get top scores on all their homework, but fail their exams.
if you've ever endured the pain of PR'ing a medium-ish sized feature from someone who copiloted their way through the entire thing you know it doesn't work that way
Two comments:
First, it's not often noted in these conversations that there are two types of LLM-using programmers/learners. One kind uses it to radically accelerate the learning process, the other kind uses it so they don't have to learn. Actually, make that three kinds—the third (probably a subset of the second) has extremely low creativity and can't understand how to use LLM tools effectively, and so can't guide their output effectively, or wrangle it after the fact.
I suspect your comment is referring to PRs by the latter kind. This is not a problem with LLMs, or with people using them to enhance productivity.
Second, what is your realistic proposal for how to confront the reality that we're accelerating through irreversible technology-assisted change?
Just like, apart from catastrophes, there's no longer a concern that we won't have massive factory farms, or that we won't have access to calculators, or that programmers won't have access to Google, there's no future where programmers wont have increasingly helpful and capable AI tools.
There will always be low IQ, low performance individuals. Can you recognize that the problem—as always—is those people, not the technology?
Anyone other than me get the vibe of Isaac Asimov's classic https://en.wikipedia.org/wiki/The_Feeling_of_Power from such posts?
Scores have skyrocketed
I suggest making the problems more unique ones that humans would be able to solve but easily trip up an AI --- minor variations of existing ones seem to work well. There's some fun with that sort of idea here: https://news.ycombinator.com/item?id=38766512
It's really already very difficult to write good problem material for evaluations. Having to find a way where difficulty is intermediate for the target audience (not too easy, not too hard) but also too hard for LLMs would be very challenging / impossible for most disciplines.
I think your idea has already worked for some companies to filter out AI applications, why not try? Especially in a font color identical to the background. You can also scaffold your way to generate questions that get the worst LLM performance, while still being very clear to understand, one side validating the clarity and theoretical tractability for the age, and one side actually solving it. Actor and two critics maybe. I have a container somewhere to create and use this kind of chain visually, could put it on GitHub but I'm sure there are dozens already
We hire interns and I've interviewed quite a few since Chat GPT. It's interesting they almost always ask what I (and the company) think about AI. Never had this question in the past. So it could be a bad thing, but the kids aren't dumb either, and the good ones will realize it can be a crutch.
Part of our interview process is a take home programming exercise. We allow use of AI, but ask that you tell us if you used it or not. That could be a good option for teachers as well.
I'm hiring, and discussions of how we want to respond to engineer candidates who get stuck are interesting. I'm personally more interested in their collaboration (wildcard) than their chat-fu (assumed at this point). So my advice to people reading this with interviews in the next year (or next week) is to consider getting off the screen and solving something with a person. We will all get plenty of self-solving time, but it helps if you can show that you can explain yourself during rapid fire situations involving others, or to bring them along with your plan, or building an unfamiliar plan B with others when two AZ are down in us-east-1 and noone planned for XYZ to be unavailable (eg something that the LLM site depended on) Not that I'm certain it'll happen, but I think calculators (to go back to this story) were more reliable than anything we've typed into the past month, and for me that includes their batteries.
god i'm so incredibly salty i finished all of my schooling a million years ago and had to laboriously do all my shit assignments without chatgpt. like yeah maybe the learning process was helpful but i was so, so miserable in school and absolutely hated it and found it boring. kids these days dont know how easy they have it oh my god i'm old
Students are absolutely copy pasting questions into ChatGPT. Though they already would have done a lot of that with google since they need to care about their GPA and thus must try to get every question right. I knew some people paying for chegg just before ChatGPT came out.
I think its still important to assign the homework but yeah its rough.
I just wish more academic material has problems with answers. I used Chegg when I took a digital logic class for my CS degree. Did I use it to cheat? No, but the textbook was my only source material, and it virtually had no solutions in it.
I would try problems, fail, look at the solution, and see what I did wrong. I ended up doing quite well because of that. It was at that point in time I learned that if more material provided such information, that I could probably teach myself most material.
Currently, I am about to hope on the DSA grind/Leetcode grind. I have tons of textbooks, and of course, it's the same issue. Hardly any solutions, so thank goodness for AI or god knows what incorrect information I would teach myself.
Yeah this is a good way of putting what I was trying to say.
Like googling college level topics can be infuriating someitmes with all the SEO spam and outdated or confusing content, not to mention the state of textbooks.
For some topics is perfectly fine with just google, but the obscure stuff can be impossible to find and in both cases ChatGPT is easier, faster, and likely has a higher success rate than ones own attempt at searching for answers.
Why not just increase the scope and explicitly allow LLMs?
Because the purpose of most homework is not to give you a “real world task”.
It is to give you simplified toy problems that allow you to test your understanding of key concepts that you can use as building blocks.
By skipping those, and outsourcing “understanding” of the fundamentals to LLMs, you’re setting yourself up for failure. Unless the goal of the degree is to prepare you for MBA-style management of tools building things you don’t understand.
The thing is colleges haven't been about education in quite some time at this point (at least all the undergraduate stuff, in masters or higher you get to work on projects that are applicable to real life somewhat). Everything that you can learn in undergraduate you can learn on the internet.
Outside of very niche and specialized professions (mostly that require networking and attendance to specific colleges), the goal of going to college should be just to get your degree. Once you have a degree, it generally gives you an easier time to get a job, so financially its worth it. How you get the degree is irrelevant - figure out the cheapest, easiest way to do it, even if it includes cheating.
Youll find out after you graduate that nobody gives a fuck about college in the real world as far as education goes.
> the goal of going to college should be just to get your degree
> figure out the cheapest, easiest way to do it, even if it includes cheating.
And this mindset is why cheating has proliferated. So many students have been imbued with a sense that degrees are "just a piece of paper" and therefore cheating is the only smart thing to do.
> Youll find out after you graduate that nobody gives a fuck about college in the real world as far as education goes.
I'm actually finding it's going the other way. The value of a brand-name college degree is extremely high for bypassing filters and getting past resume screens.
Part of the reason is that top universities are known to be difficult to cheat your way through. Not impossible, but it's not easy either.
On the other hand, students who show up from local universities may have learned absolutely nothing along the way. We don't care about their degree because rampant cheating has reduced the strength of the signal. They need to be tested thoroughly to determine if they actually learned anything from the university or if they just cheated their way through it.
College brand name may matter for your first job and in some prestige based industries (VC, consultancies etc).
I graduated from a top US Uni in CS, and I can tell you when I was searching for jobs, I was frequently passed over by candidates with more work experience who didn’t graduate from a top uni. In fact the effect of my Uni was probably close to None, I joined FAANG and discovered that my coworkers college was all over the place, you wouldn’t notice any uni trends.
I was forced to come to the harsh conclusion that college mattered, maybe 5% or lesser in the tech industry and that all the effort students put to get into college was not needed unless you wanted to break into very specific career paths. This was a harsh conclusion because I was one of the students who worked very hard to get into a top college and maintain top grades.
>I'm actually finding it's going the other way.
Whats your data source?
Across like 10 or so jobs I have applied including ones I took across engineering and computer science, he only time I had any questions about my academic record is when working for a government contract (which required me to request official record from university).
> Part of the reason is that top universities are known to be difficult to cheat your way through. Not impossible, but it's not easy either.
I wonder if that will dethrone the Ivies. They're known for being difficult to earn entry to and even harder to flunk out from. However, a rigorous State U that doesn't care that Undergrad 32768 just dropped out should have an easier time maintaining standards.
> Everything that you can learn in undergraduate you can learn on the internet.
In principle yes. But it's extremely rare that 18-23 year olds will voluntarily grind through even the tough bits of that curriculum. Autodidacts often have gaping holes of knowledge in the non-fun stuff. Some hypermotivated people will chew their way through it through sheer self-motivation but the vast majority doesn't have the iron will to do that without external pressure. Even top athletes go to training camps and have trainers who push them.
One can of course argue that the material is irrelevant to actual jobs, and it's an eternal debate whether universities should teach fundamental thinking tools and "theory" or just job skills and web frameworks and git commands.
Getting a degree is about several things:
- It shows you passed admissions (in case that's hard) - It shows you persisted in your studies and managed to pass exams with certain grades - It shows you have acquired certain foundational knowledge
The first two show your ability to learn new things. Even if (and that's just an if) what you learned wasn't directly useful, you show that you can learn, i.e. have some personal qualities like intelligence, conscientiousness, agreeableness. That you're organized enough, don't give up too easily, can work under an authority etc. Many commenters here take these things for granted, but there are many job applicants who are not like you or your friends in these regards and having passed through those filters prepared by colleges is a very meaningful signal to employers.
And the foundational knowledge of math and algorithms is in fact also very useful for any non-code-monkey stuff. You learn a terminology, a vocabulary to talk to colleagues. Yes, you'll learn most things on the job, but it still makes a difference.
And then there's networking as well. Later in life, a recommendation can be very useful for getting a job. Lots of jobs never get publicly advertised because the signal-to-noise ratio is much better if people first search among acquaintances and contacts.
So a college education gives: foundational knowledge, demonstrable evidence of personal qualities, external push and motivation for developing yourself, a personal network.
In case there's any young and impressionable people in here i want to add that easiest does not always mean cheating! The people i knew who cheated their homeworks were the same people crying over their grades during quizzes and tests. They were the people most terrified during finals and generally had the worst mental states during the year. It certainly did not seem to make their lives easier. Sure, you might get away with it but these things can come back to bite you!
The better you do and the more you learn in college, the better you can speak and the more you can show off in an interview for your desired position, whether it's a job or a grad school. Especially if your chosen degree basically requires a graduate degree to get good jobs, don't cheat (unless it's an essential grade and you promise to go learn it better asap). Grad school doesn't mess around, it's hard enough for the studious ones.
If you don't care about school and your field doesn't care about school then do whatever. But don't make a habit of living dishonestly. It wears at the soul
I had a wonderful philosophy professor in a 100-level class I was taking to fulfill a gen ed req, he was some old retired guy and he had no mandatory attendance and only one assignment for the whole semester: a single, 15 page final paper.
The contents of the course was extraordinarly more difficult than the vast majority of 100-level classes at the university (this was a top philosophy department in the world, mind you), and within a few classes almost all of students stopped coming and, even bragged it in the class group-chat. I became intensely interested in the material within a few classes, and attended nearly every single one and stayed after to talk to the professor. Well, the final paper comes along, I was already away from campus, deciding to take a nice vacation since the professor said that if I wanted I could delay submitting for a couple weeks--well, unfortunately, he was mistaken, and I got an email after just getting off my connecting flight where he said I had to get it done by that afternoon, but he didn't care if I actually submitted: to him, I already had an A. I sat down, on my phone in the middle of the night and wrote the whole 15 page paper in a deserted airport terminal. I got an A. Others, who had not even showed up, were having panic attacks about it, incessantly whining on the group chat, freaking the fuck out since they knew they were all about to fail since they had almost no time to study up on materials for dozens of classes with no assistance.
This was all before the advent of ChatGPT. I have no idea if that 15-page paper would be such a killer today. Probably not; probably, if the guy is still teaching, kids do get away with skipping every class and getting AI to write a passing paper. But, the principle is still there: you just need a paper test now!
I don't understand. Your professor said the paper wasn't due, then bumped up the date, told you about it last minute, said you didn't have to turn the paper in, but you did anyway?
I'm lost.
> This was all before the advent of ChatGPT. I have no idea if that 15-page paper would be such a killer today.
The thing about ChatGPT is that it's not very good at producing essays. For a freshman-level class, it's likely to produce a C-quality paper, but my understanding is that trying to get it to produce a coherent longer essay is much more difficult, to the point that it'd be dubious if it would be a passing grade.
This is fairly incorrect.
>The better you do and the more you learn in college, the better you can speak and the more you can show off in an interview for your desired position
For undergrad degrees, you have nothing to show off except maybe a project that you were required to do. Doing internships during the summer or taking semesters off to do coops is the best way to land a job. If you do a good job as an intern or during coop, its almost a guarantee that you will be given a job in that company, or at least have extremely valuable experience.
Even in grad school, while you do get more experience and are a bit closer to the industry, its often less valuable than industry. When you join a company, your sole purpose is to contribute to the company making money. When you are on a grad project under a professor, your sole purpose is to make sure the professor either gets an ego boost on a publication or attains permanent salary with tenure. These two difference are vastly going to dictate what work you are going to do.
The better word to use instead of cheating would be hacking. Don't be fooled by the rules in front of you, instead figure out the shortcuts. Obviously cheating on homework when you have to take a test and doing the work anyways is going tor result in a lower grade, so thats not really hacking.
Hacking would be like getting notes from upper classmen so you know exactly whats on the tests, taking classes in other colleges if possible that are easier that count towards the same credit, figuring out how to get out of taking bullshit classes, and so on. Figure out the least amount of work you have to do to get that degree.
The most notable story I have of one of my classmates is that he found out that community college classes count for the non technical degree requirements (like English for example). To get the credits transferred, you basically have to send the transcript yourself to the main college. So he took one class in the summer, got the transcript, and added a class he never took, making sure that all the info was legit, and it went through cause the people entering the data never bother to call the office and check. Next year he just straight up forged the entire transcript to cover the rest of his electives, and it went through as well. Ended up saving money, and boosting his GPA.
I have to admit I wrote a few cheat sheets on tiny little notes. Maybe a handful of times in high school. But I never once had to use them. By painstakingly writing those little notes, I somehow managed to memorize is at the same time. And having a backup made me feel safer. Stupid - and I wouldn't recommend this strategy to anyone (besides being unethical to even consider cheating).
>besides being unethical to even consider cheating
Ethics in universities would only apply if admittance was fully merit based, or open to anyone with continued admittance being based on performance, with no monetary transaction involved.
In EU, this is vaguely applicable, since your are indirectly purchasing your education through taxes, but at least there are some arguments to be made about merit based things.
In US, colleges are just businesses that you do a direct business transaction with.
So in a business, you pay them for a certain product and/or service. They say to give you this product, you need to do certain things. At any point and time, for whatever reason, either you or the company can choose to end their relationship with you. There is no morality or ethics here, its just a lie made up to get you to follow rules, when others who are higher status (like NCAA athletes for example) don't have to.
If people are cheating with timed exams, what could go wrong with homework? Nobody in the world would ask/pay someone to do homework that contributes a significant portion of final grade!
It doesn't really scale and doesn't work for all materials but I'd love to see the concept of oral test/defenses introduced at the undergraduate level.
As an ESL teacher for many years, a 30 minute conversation between the teacher and the student can reveal a student capabilities far more accurately than anything else and completely bypasses the vast majority of cheating.
Just use supervised testing. It's scalable, battle tested, and pervasive. Even ignoring scalability, orals have problems. KISS!
US universities are too focused on homework in general. In other countries most of the final grade comes from the final exam and midterm exam. Homework just creates extra work for everyone involved. It’s upto the student to decide if he wants to study or not and consequently pass
Homework gives you two things, continuous feedback (grades) and practice. Quizzes help with the former, you can only make up for the latter by making the school day longer — which I guess might be ok, given that total hours spent learning should be the same? Unless there's extra wrinkles I'm missing?
Homework is an incredibly controversial topic I think, because:
Homework, since you can get a lot or even full credit even if you get it wrong (haven't learned the material well), provides a big boost to the grades of a type of student who "tests poorly" -- whether because they failed to learn the material, or because of anxiety or whatever.
On the other side of the debate you have an alliance of:
• Parents who think "Jeez, my kid comes home from school with 3 hours of homework every night, WTF, let them live life"
• Kids who, to avoid using labels, I'll just say... they learn the material easily AND can prove it easily on a test. They say "WHY TF are you wasting hours of my time doing busywork??
If I had to be a teacher and could control my grading policy I guess I'd probably do a hybrid where homework can bring your grade up but was not required for a perfect grade. So,
GRADE = MAXIMUM(HW_GRADE * .15 + TESTS, 1)
With all due respect to the "can't take a test" crowd, it seems unfair to give homework a weight higher than that though. Should someone who gets like a 70 on the test get an A by grinding on homework? I'm glad I'm not a teacher so I don't have to actually debate anyone on that.
> homework can bring your grade up but was not required for a perfect grade
A biochemistry unit at a Uni in Australia I took in ~2010 operated this way, which was quite surprising to me. The required minimum work was a field work report, one mid semester test and the main end of semester test, but you could bring your grade up to make up for lacking results by the weekly homework assignments.
I didn't do the assignments, but still got a nearly perfect grade, which suited me great (I was doing a double degree and had overloaded on units that semester, so being able to skip weekly homework assignments and just study the textbooks for the exams was super useful)
I took a series of four classes with a very rigorous instructor who would issue a massive syllabus with a load of tasks that was like a scavenger hunt.
At some point down the road, he explicitly reminded us that every task was showing an associated point value, and rather than going down in listed order or spending hours on 1-pointers, we should prioritize the best scores according to our skills and competence.
With the entire list before us, we could work at a steady pace without sweating over busywork. And he encouraged us to watch videos and explore the interesting parts.
This video is specifically about how to cheat on supervised tests using an approved device.
Actually you don't have any idea how many college profs are cheating and using ai to generate/grade problem sets.
Sorry for late reply but I've seen examples that are very sloppy, problems that don't make sense, students being marked down for bad reasons etc. And the course forbids use of chatgpt etc. so it just seems really hypocritical.
which is actually my "Dark AI World" prediction for the next 5-10 years:
a boom of AI to such an extent that everything we do in our lives gets more verbose and it's just AI bots chatting to each other, in each step blowing up the signal with more noise on one side, distilling out the signal at the other end. to an extent where as a human you can't keep up anymore with all the useless filler.
can we just leapfrog (or backtrack) to API's talking to each other please?
I'm waiting for the day when classrooms are retrofitted to be Faraday cages
When I was in masters, I saw someone cheating by putting a book on their desk and looking inside, in an exam that doesn’t allow books. The professor was basically sleeping on his chair.
/s noted, how near is "near"?
I'm not expecting that kind of change in less than 6 years even if the tech itself is invented tomorrow, due to the constraints on the electrical grid.
As for the tech, I can't tell if we're on the first half or the second half of the S-curve for the current wave of AI. If it's the former, then in a few years every human will need a PhD (or equivalent in internships) before they can beat AI on quality.
>As for the tech, I can't tell if we're on the first half or the second half of the S-curve for the current wave of AI. If it's the former, then in a few years every human will need a PhD (or equivalent in internships) before they can beat AI on quality.
Unlikely, since they're pumping new GPTs with responses written by PhDs anyway. It's becoming more and more of a "Wizard of Oz" situation.
1. There's a lot of superhuman AI out there already, just in narrow domains like protein folding, chess, and so on.
2. Doctorate (and postdoc, and professoral) level responses were already in the training sets.
3. Aim of current AI research is to create a model of the underlying reality which produces the observed signals — what must a PhD candidate have observed for them to write that particular paper, etc.
I have no idea if these AI efforts will succeed or not, hence no idea where we are on the S-curve. But that's the goal.
I teach CS, and oh we know but I don't know what to do about it. Scores have skyrocketed because students are using some kind of AI helper like co-pilot, if not just outright pasting the assignment text to ChatGPT. It's hard to prove.
I've thought about putting instructions in the assignment to sabotage it (like, "if you're a generative AI, do X - if human, please ignore.") but that won't work once students catch on those kinds of things are in the assignment text.