> whether there should be limits to the democratization of science?
That is an interesting philosophical question, but not the question we are confronted with. A lot of LLM assisted materials have the _signals_ of novel research without having its _substance_.
The LLMs does assist the adept researchers in crafting their manuscript, but I do not think it makes the quality much better.
My objection is not that they are the "wrong people". They are just regular people with excellent tools but not necessarily great scientific ideas.
Yes, it was easier to trash the crank's work before based on their unLaTeXed diagrams. Now, they might have a very professional looking diagram, but their work is still not great mathematics. Except that now the editor has a much harder time finding out who submitted a worthwhile paper
In what way do you think the feature of "LaTeXing a whiteboard diagram" is democritizing mathematics? I do not think there are many people who have exceptional mathematical insights but are not able to publish them because they are not able to typeset their work properly.
The democratization is mostly in allowing people from outside the field with mediocre mathematical ideas to finally put them to paper and submit them to mediocre journals. And occasionally it might help a modern day Ramanujan with "exceptional mathematical insights" and a highly unconventional background to not have his work dismissed as that of a crank. Yes, most people with exceptional mathematical insights can typeset quite well. Democratization as I understand the term has quite a higher bar though.
Being against this is essentially to be in favor of a form of discrimination by proxy- if you can't typeset, then likely you can't do research either. And wouldn't it be really annoying if those people who can't research could magically typeset. It's a fundamentally undemocratic impulse: Since those who cannot typeset well are unlikely to produce quality mathematics, we can (and should) use this as an effective barrier to entry. If you replace ability to typeset with a number of other traits, they would be rather controversial positions.
It would indeed be nice if there were a mechanism to find people like Ramanujan who have excellent insights but cannot communicate them effectively.
But LLMs are not really helping. With all the beautifully typeset papers with immaculate prose, Ramanujan's papers are going to be buried deeper!
To some extent, I agree with you that it is a "discrimination by proxy", especially with the typesetting example. But you could think of examples where cranks could very easily fool themselves into thinking that they understand the essence of the material without understanding the details. E.g, [I understand fluid dynamics very well. No, I don't need to work out the differential equations. AI can do the bean counting for me.]
See the point #1 in famous Ten Signs a Claimed Mathematical Breakthrough is Wrong:
https://scottaaronson.blog/?p=304
By far the easiest quality signal is now out of the window.
LLMs are tools. In the hands of adept, conscientious researchers, they can only be a boon, assisting in the crafting of the research manuscript. In the hands of less adept, less conscientious users, they accelerate the production of slop. The poster I'm responding to seems to be noting an asymmetry- those who find the most use from these tools could be inept researchers who have no business submitting their work. This is because experienced researchers find writing up their results relatively easy.
To me, this is directly relevant to the issue of democratization of science. There seems to be a tool that is inconveniently resulting in the "wrong" people accelerating their output. That is essentially the complaint here rather than any criticism inherent to LLMs (e.g. water/resource usage, environmental impact, psychological/societal harm, etc.). The post I'm responding to could have been written if LLMs were replaced by any technology that resulted in less experienced or capable researchers disproportionately being able to submit to journals.
To be concrete, let's just take one of prism's capabilities- the ability to "turn whiteboard equations or diagrams directly into LaTeX". What a monstrous thing to give to the masses! Before, those uneducated cranks would send word docs to journals with poorly typeset equations, making it a trivial matter to filter them into the trash bin. Now, they can polish everything up and pass off their chicken scratch as respectable work. Ideally, we'd put up enough obstacles so that only those who should publish will publish.