The Medium comments on this post are also on point. Running the same experiment with accepted papers is a good control. Running a similar experiment with reviewers would be interesting, but more obnoxious because they are not being paid.
I would keep a private blacklist (shadow ban) the authors who wasted several hours of a reviewer's time to prove they were not legitimate. The existence of such a list would be problematic, though.
Could the same system we use here be applied? Accepted authors could "vouch" for "dead" papers in case they were "auto-killed"?
This system is broken and providing more evidence that it is broken isn't much of a step towards fixing it.
Do you imagine a world where people are willing to go through the legal hassle of changing their name to get past a ban for low effort journal submissions?
Validating someone’s association with an institution by name seems like a reasonable thing to do. Perhaps it wasn’t done in the past, but times and circumstances have changed. Trust in authorship is lower than ever, and for good reason.
Plagiarism, fictional data, and unreproducible results have been a problem for a some time, and LLMs are now increasingly standing in authors’ shoes. In a system that rewards production over all else, people are naturally going to be incentivized to cut corners.
Some cultures that are increasingly participating in the scientific process don’t even agree as to what the ethical boundaries are.
True, and thanks to llms are of those are being “exploited” exponentially more.
“In a system that rewards production over all else, people are naturally going to be incentivized to cut corners.” That sounds like the human condition and or capitalism, advancements in cars, weapons, toys, etc…
The agreement on ethical boundaries seems like a conversation held at a conference level, otherwise place a governs place b and that never goes well.
I agree fully replicable experiments is a minimum for research papers. Even with caveats it should be required. That being said those who don’t want to fully prove their claims will find another venue. I think frontier models should curb or have provably verified output (chatgpt did say x) for the majority cases to help reduce or help identify the slop.
> That sounds like the human condition and or capitalism, advancements in cars, weapons, toys, etc…
Indeed it is! That's why we set up rules of the game/road/etc. that we expect participants to adhere to, and impose sanctions when they don't. We're in that uncomfortable place today where we're trying to figure out how our rules should adapt to this new era.
Oh no I fully agree with that. We just have to find a medium that people can easily use but can’t abuse. The constant cat and mouse security problem given non-infinite money. In-person defense would solve this but I think the cost is too high.
Even if it's not a legal name, are you going to review the CV of an applicant who used a different name on every publication they claim authorship of? It's going to be pretty obvious someone is being nefarious
The CV of an applicant is definitely different. For a job application where a cv were used, that’s a slower/paid process where attention should be paid to details. However what’s a difference to a conference with a “young upcoming author” vs llm-Lorem ipsum author name?
You said that people could do a thing. I'm asking why they would.
Academic fraud is done to boost the reputation of the academic. There isn't a lot of point in academic fraud with one paper on each of 50 fake names.
Ahhhh, that makes more sense. Academic fraud may not always be used to boost the reputation of an academic, it could boost the reputation of a science/technique/medicine. If you see 30 authors with 1,000s of citations saying x is superior and 3 authors saying y is superior, from a quick glance x seems a lot better.
> Accepted authors could "vouch" for "dead" papers in case they were "auto-killed"?
The problem isn't with the papers here though, it is the author's understanding of the paper that is in question. A paper written by some hypothetically awesome AI would be a good paper but just not really the proclaimed author's paper.
I think this highlights a dual function of citations that are in tension. A citation can be to claim a stated idea has been made and tested with sufficient rigour to be published. Citation's can also be used to 'credit' others, treating reference as a type of currency. I think this latter form is an outright mistake, but entrenched in academia. The notion of giving credit like this creates a perverse incentive that lies behind much academic fraud, there is enough incentive to be the person to state something that it outweighs the requirement that person has for the statement to be true. Without that notion of credit as currency, issues like plagiarism simply disappear. In the absence of credit, someone making the same claims as someone else without referencing them is just making their own case weaker. Not necessarily less true, but less convincing. If citations were used just used to support a paper then the incentive is to cite, and failing to reference existing work harms only the author.
I think there is too much "This is my idea" and not enough "I think this is true". Credit fails as a measure of effort, diligence, innovation, or truth. Careers are being made and broken by how effectively an individual can game the system.
It strikes me as akin to plagiarism. If the purported author can’t even answer basic questions about the paper, how can they plausibly claim to have written it?
In the case where someone uses AI to write the paper and then deeply familiarizes themself with it, it may go undetected, but then it’s also presumably less of an issue since they have actually read it carefully and closely. If it’s still bad or wrong after that, then it’s not that different from a human writing a bad or wrong paper on their own and should be treated similarly.
> It strikes me as akin to plagiarism. If the purported author can’t even answer basic questions about the paper, how can they plausibly claim to have written it?
Authorship standards differ by field. In biology, for example, it would be common to list someone as an author if they assisted in one experiment. They might be at a different institution and may be unaware of all but the vaguest outline of the paper as a whole—they just got brought onboard because they are an expert in one particular task that needed to be done. In exchange they get to be a middle author (not worth much) and develop a relationship with someone whose expertise they may need on one of their own papers in the future (the primary benefit).
That is not the case here, of course—I just wanted to provide some context for your position not being universally applicable.
Only in small papers. Once the project is big or multi-disciplinary enough that co-first and co-last authorship become a thing, it's quite common that no single person fully understands the paper.
Ideally, but often not true of large complex studies. Typical example is that only a middle author statistician understands the details of data analysis. Also commonly true in quantitative genetics.
Yes. The cognitive process performed by a person using an LLM is often no different from that performed by a person using a ghost author, which is a form of plagiarism covered by 42 CFR § 93.234 - Research misconduct.
Plagiarism, at its most fundamental level, is a lie. It is the taking of works or ideas of others and passing them off as your own, either directly or indirectly. The misdeed itself is in the lie, the “I created this” when it is known to be untrue.
However, that lie isn’t being told to the original victim. It’s a lie about the victim, claiming that they didn’t create it or their contributions didn’t matter, but it’s not a lie to them. Instead, it’s a lie to the audience, which is the second victim and the actual target of the con.
This is a very good definition, but today people will just claim the AI is not a victim, and pass this argument off to the legal cases the labs are already having in court wrt fair use. It's sad, but this is the argument people have made to themselves.
In academic settings, it’s also plagiarism to copy your own previous work. Given that, I’m not sure academia gets to take the high ground on what constitutes misconduct or not wrt to plagiarism, at least in the edge cases.
It's often denoted plagiarism or self plagiarism as a short cut. While it's probably better denoted duplicate publication (or similar) it still constitutes a similar academic misconduct, because you're pretending something is novel without it being novel
As a side note quite often it can actually be literal plagiarism as well, many journals require you to assign copyright, so using that work without attribution is plagiarism.
The status of the true author is kind of irrelevant to whether it’s plagiarism or not. The true author may even be an active participant in the conspiracy.
The right response then perhaps would be to correctly assign AI the authorship it deserves instead of the human "operator/patron" who did not understand what the AI wrought, but merely acted as benefactor to provide the token fee.
A paper should stand and fall on its merits; does it build constructively on our collective understanding? Does it advance the state of the art in its field? Does it shed new light on a prior mystery?
So it seems there ought to be a path whereby a quality non-human paper ought to be considered for submission, provided it is adequately and correctly attributed.
fn-mote · · focus · HN ↗
I would keep a private blacklist (shadow ban) the authors who wasted several hours of a reviewer's time to prove they were not legitimate. The existence of such a list would be problematic, though.
Could the same system we use here be applied? Accepted authors could "vouch" for "dead" papers in case they were "auto-killed"?
This system is broken and providing more evidence that it is broken isn't much of a step towards fixing it.
doc_ick · · focus · HN ↗
malfist · · focus · HN ↗
doc_ick · · focus · HN ↗
otterley · · focus · HN ↗
doc_ick · · focus · HN ↗
What good reason is there for trust in authorship to be low? I would likely agree for if it’s related to llm-slop.
otterley · · focus · HN ↗
Some cultures that are increasingly participating in the scientific process don’t even agree as to what the ethical boundaries are.
doc_ick · · focus · HN ↗
“In a system that rewards production over all else, people are naturally going to be incentivized to cut corners.” That sounds like the human condition and or capitalism, advancements in cars, weapons, toys, etc…
The agreement on ethical boundaries seems like a conversation held at a conference level, otherwise place a governs place b and that never goes well.
I agree fully replicable experiments is a minimum for research papers. Even with caveats it should be required. That being said those who don’t want to fully prove their claims will find another venue. I think frontier models should curb or have provably verified output (chatgpt did say x) for the majority cases to help reduce or help identify the slop.
otterley · · focus · HN ↗
Indeed it is! That's why we set up rules of the game/road/etc. that we expect participants to adhere to, and impose sanctions when they don't. We're in that uncomfortable place today where we're trying to figure out how our rules should adapt to this new era.
Life is messy.
doc_ick · · focus · HN ↗
malfist · · focus · HN ↗
doc_ick · · focus · HN ↗
wpietri · · focus · HN ↗
doc_ick · · focus · HN ↗
wpietri · · focus · HN ↗
Academic fraud is done to boost the reputation of the academic. There isn't a lot of point in academic fraud with one paper on each of 50 fake names.
doc_ick · · focus · HN ↗
Edit: see medical studies
Lerc · · focus · HN ↗
The problem isn't with the papers here though, it is the author's understanding of the paper that is in question. A paper written by some hypothetically awesome AI would be a good paper but just not really the proclaimed author's paper.
I think this highlights a dual function of citations that are in tension. A citation can be to claim a stated idea has been made and tested with sufficient rigour to be published. Citation's can also be used to 'credit' others, treating reference as a type of currency. I think this latter form is an outright mistake, but entrenched in academia. The notion of giving credit like this creates a perverse incentive that lies behind much academic fraud, there is enough incentive to be the person to state something that it outweighs the requirement that person has for the statement to be true. Without that notion of credit as currency, issues like plagiarism simply disappear. In the absence of credit, someone making the same claims as someone else without referencing them is just making their own case weaker. Not necessarily less true, but less convincing. If citations were used just used to support a paper then the incentive is to cite, and failing to reference existing work harms only the author.
I think there is too much "This is my idea" and not enough "I think this is true". Credit fails as a measure of effort, diligence, innovation, or truth. Careers are being made and broken by how effectively an individual can game the system.
nkrisc · · focus · HN ↗
In the case where someone uses AI to write the paper and then deeply familiarizes themself with it, it may go undetected, but then it’s also presumably less of an issue since they have actually read it carefully and closely. If it’s still bad or wrong after that, then it’s not that different from a human writing a bad or wrong paper on their own and should be treated similarly.
CrazyStat · · focus · HN ↗
Authorship standards differ by field. In biology, for example, it would be common to list someone as an author if they assisted in one experiment. They might be at a different institution and may be unaware of all but the vaguest outline of the paper as a whole—they just got brought onboard because they are an expert in one particular task that needed to be done. In exchange they get to be a middle author (not worth much) and develop a relationship with someone whose expertise they may need on one of their own papers in the future (the primary benefit).
That is not the case here, of course—I just wanted to provide some context for your position not being universally applicable.
nkrisc · · focus · HN ↗
kamma4434 · · focus · HN ↗
CrazyStat · · focus · HN ↗
jltsiren · · focus · HN ↗
robwwilliams · · focus · HN ↗
jedbrown · · focus · HN ↗
Plagiarism, at its most fundamental level, is a lie. It is the taking of works or ideas of others and passing them off as your own, either directly or indirectly. The misdeed itself is in the lie, the “I created this” when it is known to be untrue.
However, that lie isn’t being told to the original victim. It’s a lie about the victim, claiming that they didn’t create it or their contributions didn’t matter, but it’s not a lie to them. Instead, it’s a lie to the audience, which is the second victim and the actual target of the con.
<a href="https://www.plagiarismtoday.com/2019/08/01/the-two-victims-of-plagiarism/" rel="nofollow">https://www.plagiarismtoday.com/2019/08/01/the-two-victims-o...
figassis · · focus · HN ↗
michaelt · · focus · HN ↗
Copying from a book with a long dead author, or paying a willing confederate to write your thesis for you, is still plagiarism.
LadyCailin · · focus · HN ↗
cycomanic · · focus · HN ↗
As a side note quite often it can actually be literal plagiarism as well, many journals require you to assign copyright, so using that work without attribution is plagiarism.
nkrisc · · focus · HN ↗
tmoertel · · focus · HN ↗
dweekly · · focus · HN ↗
A paper should stand and fall on its merits; does it build constructively on our collective understanding? Does it advance the state of the art in its field? Does it shed new light on a prior mystery?
So it seems there ought to be a path whereby a quality non-human paper ought to be considered for submission, provided it is adequately and correctly attributed.
phyzome · · focus · HN ↗
ksudb · · focus · HN ↗
[dead]