Our internal thought process is an emergent property of a bunch of thoughtless chemical reactions. You're making a big assumption that LLM will never produce similar internal thought process as an emergent property.