> Tons of tests got added, but some tests that mattered the most (maestro e2e tests that validated the app still works) were forgotten.
I've seen many LLM proponents often cite the number of tests as a positive signal.
This smells, to me, like people who tout lines of code.
When you are counting tests in the thousands I think its a negative signal.
You should be writing property based tests rather than 'assert x=1', 'assert x=2', 'assert x=-1' and on and on.
If LLMs are incapable of acknowledging that then add it to the long list of 'failure modes'.
> Tons of tests got added, but some tests that mattered the most (maestro e2e tests that validated the app still works) were forgotten.
I've seen many LLM proponents often cite the number of tests as a positive signal.
This smells, to me, like people who tout lines of code.
When you are counting tests in the thousands I think its a negative signal.
You should be writing property based tests rather than 'assert x=1', 'assert x=2', 'assert x=-1' and on and on.
If LLMs are incapable of acknowledging that then add it to the long list of 'failure modes'.