Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

You can't rely on journals because they have bad incentives. Their customers are the universities they're supposed to monitor for fraud, and their prestige comes from the same place news/magazine prestige does - publishing incredible claims.

Catching fraud further down the chain is easy. Other professions know how to do it. Just adopt the same framework as used in financial controls. External auditors, written standards, randomized spot checks, lots and lots of paperwork to provide audit trails back to real experiments with hard-to-forge evidence. And severe punishments for those found to have engaged in intentional fraud.



That sounds expensive and difficult. Science is not a single field with a single set of practices. Regulating it is more like trying to draft a single set of regulations for finance, agriculture, pharma, and software.

You can get a glimpse of that when applying for grants or publishing in more prestigious journals. There are often endless checklists and questions about regulatory and ethical compliance, data availability, reproducibility, and whatever else. And more often than not, many of the questions are category errors or otherwise make no sense in the context of your work. Even the people who are supposed to regulate science lack the ability to do so in a way applicable to the entire breadth of science.


I don't think the differences between fields in (academic) science are as big as the differences between agriculture and software. They all take grants to publish papers, in journals, and they are measured via similar metrics like citations, papers published, impact factors and so on. Academia is highly homogenous across institutions too.

Yes, for sure a lot of controls implemented via audit aren't perfect fits for what is being regulated, that's the nature of trying to fix bad incentives via regulation. It's better to just have aligned incentives to start with, but that can't be done in academia, it's regulation or nothing.

Also, my experience has been that sometimes academics like to claim their field is special and shouldn't be expected to play by the same rules as other fields. But that's often not true. They should in fact be held to the same standards as everyone else is and their justifications for why they're an exception don't hold water.


The academia is far from being homogeneous. Institutions in the same country are usually similar, but things vary a lot between countries.

Science is fundamentally an activity, not a field. You can do scientific research in most fields. Agriculture and software are nice examples, as academic scientists are doing both. And in many cases, the outputs are used by the general public.

Quantitative metrics such as citations, paper counts, and impact factors are mostly used by bureaucrats who don't know what they are doing. Actual evaluations in reputable institutions depend heavily on the subjective judgment of other people in the field.

The role of grants varies a lot between fields and insitutions. In some cases, a professor is expected to become a manager, win grants, and hire other people to do their research. In other cases, grants are nice but optional. Such professors can do research with limited external support, both personally and with students funded from other sources.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: