It's not a "minuscule number" because visionOS is controlled by looking at things and pinching your fingers. If people subconsciously stare at the edges of a square button, can you imagine how often they'll misclick? (mispinch?)
Other eyetracking software solves this and all related problems by simply showing you a pip of some sort where it senses you "looking". It's stupidly intuitive, low intrusiveness (and can go away when it senses you don't need to "look" at anything like when you are looking at a movie screen with no controls on it) and works just fine. Only Apple decides the solution to their broken UI is to break more of the UI.