This isn't a problem at all for modern type systems. OCaml, Haskell, and even C++ have Turing complete type systems and this is one of the last concerns for developers.
I don't understand why that would make a fixed length of road be so much more expensive, unless commute time for blue collar workers is compensated better than lawyers. Shipping a truckload of asphalt plus the required machinery can't be that expensive.
The reinforcement learning framework is perfect for representing cause and effect. An agent could learn that in a state of no fire, taking an action of rubbing sticks together would transition into a state of having fire. This concept is formalized as learning the dynamics function.
Neural networks do exactly what you are describing as "symbolic reasoning". It seems to be a common thing recently to dismiss modern ML techniques as curve fitting, but these fundamental models are extremely powerful.
Neural networks are capable of approximating any system to arbitrary precision.
This is theoretically true, but it's like saying "computers can compute any function, given enough time and resources".
There is a need to construct logically-deduced models which impose an inductive bias so that your regression methods are efficient. That's where reasoning comes in, and where automated reasoning methods should be useful.
The "self-model" you're talking about is the agent in the Reinforcement Learning framework. It moves between states in an environment and learns from reward it earns from each action.
Yes, although I wonder if, or how well these self-models develop in practice compared to the world-models. Say, if a 2D-agent has a rectangular body shape, it probably won't develop a high-level representation of that fact unless its actions allow it to perceive it accurately. Purely figuring that out from collisions produced by basic actions (rotate left, rotate right, move forward etc.) seems to be practically infeasible. It has neither sight (observing self-movements) nor self-touch (which would allow it to observe its boundaries and relate it to what it has seen).
