> but I found that LLMs do a pretty good job of generating Typst code.
Interestingly, I've had the opposite experience. ChatGPT and Claude repeatedly gave me errors, apologized profusely, and then said, "ah, I had the wrong keyword. It's actually <blahblah>"--and that would simply give me another error and a subsequent apology.
At least Gemini had the good taste of telling me that it didn't know how to do what I wanted with typst.
It's certainly possible that I was trying to do something a little too unusual (who knows), but I chalked it up to the LLMs not having a large enough corpus of training text.
On the bright side, the typst documentation is quite good and it was just a matter of adjusting example code that got me on track.
Well, that just goes to show that these tools are wildly unpredictable. I've had bad experiences generating Go, whereas I've read many experiences of the opposite.
> I chalked it up to the LLMs not having a large enough corpus of training text.
I'm inclined to believe the opposite, actually. It's not so much about the size of the training data, but the quality of it. Garbage in, garbage out. Typst is still very young, and there's not much bad code in the wild.
And the language itself plays a large role. A simple language with less esoteric syntax and features should be easier to train on and generate than something more complex. This is why I think LLMs are notoriously bad at generating Rust code. There's plenty of it to train on, but Rust is a deep pit of complexity and unusual syntax. Though it helps when the language is strict and statically typed, so that the compiler can catch issues early. I would dread relying on generated Python code, despite of how popular and simple it is on the surface.
Interestingly, I've had the opposite experience. ChatGPT and Claude repeatedly gave me errors, apologized profusely, and then said, "ah, I had the wrong keyword. It's actually <blahblah>"--and that would simply give me another error and a subsequent apology.
At least Gemini had the good taste of telling me that it didn't know how to do what I wanted with typst.
It's certainly possible that I was trying to do something a little too unusual (who knows), but I chalked it up to the LLMs not having a large enough corpus of training text.
On the bright side, the typst documentation is quite good and it was just a matter of adjusting example code that got me on track.