I go into many reasons why big-5 (and personality psychology in general) might be a pseudoscience in a nibling comment. Here I would like to add that what you might have seen big-5 proponents do to grit (a trait from positive psychology), other fields of psychology (such as cognitive psychology, neuroscience, social psychology, behavioral psychology, etc.) does to personality psychology (and psychometrics in general).

I’m not aware of any debunking claims in particular, but skepticism is plenty, and the usual accusations range from pseudoscience to poor science.

In my opinion the worst critics go into an alternative theory of personality traits, e.g. 7 axis instead of 5 or neurotism is actually something else etc. These are attempts to debunk big-5 and in my opinion they always fall short. It doesn’t take long for proponents of big-5 to debunk these critics as the statistics behind the theory of big 5 is solid. Personality tests are robust and they consistently yield 5 disjoint axis in cluster analysis.

A better criticism goes into the foundation of the theory, attacks the fact that personality (and the personality traits) are operationally defined with little evidence they exist outside of the models. The behavior that personality tests claim to predict often have better correlation to non-personality traits, such as your brain dopamine levels, class, education or economic status, anxiety or depression, prior stimuli and response etc. Behavior is better predicted by looking at the person not how the person responds on a piece of paper. Their tests might be robust, but their science is lacking.

I’m not a believer in personality traits in general, neither “grit” nor “agreeableness”. To me the findings of any psychometrician is unremarkable at best and racist at worst (See Stephen J. Gould (1996) “The Mismeasure of Man”). In a given situation how you respond is never dictated by your personality traits. If it explains any variability behavior the effect size is hardly ever significant, and if it is there are probably other factors—not measurable with questionnaire—will explain it better.

Great post and thank you for going in-depth. What types of behavior were you thinking of when you typed "The behavior that personality tests claim to predict often have better correlation to non-personality traits, such as your brain dopamine levels, class, education or economic status..."? I'm just curious about what has been studied.

It’s been awhile since I was doing psychology, but following is a list of the top of my head (note that I’m likely to misremember or misrepresent some of these):

* One good study demonstrated that a common drug used to treat Parkinson deices has an unfortunate side-effect of limiting dopamine levels in your frontal cortex. People who used that drug are way more likely to engage in anti-social behavior after they start treatment then before. It seems like dopamine levels in your pre-frontal cortex can severely affect your personality.

* Social scientists often create amazing studies where they are easily able to manipulate the environment in such a way that they get participants to e.g. lie, cheat, etc. Off course there is variability within these studies and you can argue that personality trades explains this variability, however I don’t know if that has been done, and if it has, I wouldn’t be surprised if you would find that other factors such as religiousness, socio-economic status, gender, age etc. explains a bigger part of the variability in those social science studies.

* If you are looking for a better general theory of behavior then cognitive science has a number of constructs which don’t have this problem of operational definition. For example research has shown that you can prime people with a stimuli to increase the chances the participants will respond to similar stimuli in a short timespan afterwards.

* Behavioralism is more like a philosophy then science and posits that you can explain behavior by looking at the reinforcement history of an individual. That theory has been somewhat debunked by cognitive science, however behavioralists are still doing some impressive research which shows how you can modify behavior by offering some reinforcement contingencies. The same variability applies though as with social the social science studies.

I tried to sway away from specific studies because it has been a minute since I was studying psychology. But I hope this list is still relevant and accurate, and that it inspires you to go look for the individual studies which backs these claims.

Do note that what I’m doing here is a bit unfair. I’m making grand claims backed by evidence I think exists, but pushing the responsibility of looking for these evidence onto the reader. This is not how science communication is supposed to work, and my only excuse is that I’m lazy.