By observing modern day collective behaviour I have come up with a hypotheses. I suspect that as small furry rodent like mammals, which all mammals have been early on in their evolution, that we spent a lot of time in amongst rocks, caves, and eventually burrows, like guinea pigs or rock rats, for the same reasons that the modern rodent lives there, protection from predator and element alike. I think we spent more time in our evolutionary path amongst rocks then trees for nesting purposes.
Firstly, observing how we prefer our infrastructure and social arrangements, you will see on a physical level, we gather together in communities, such as cities, where the environment, probably by the collective subconscious, appears to have evolved to look like piles of rock, cliffs and caves (the physical structures). Gardens are desired, but considered optional by most. They are less distressed by the loss of a garden then a house generally. If you have a breeding pair of humans, they seek an adequately secure building (cave, rock) before a garden that is big enough "to play in", even if that garden represents food. If you have a person who feels they are in danger, they will run for the cover of a rock like building before they climb a tree for security. Our classic image of security is a house, not a tree. This leads me to believe that we evolved oppossable thumbs through rock climbing, burrow digging, and moving stones, sticks and tools. Also, food gathering would have played a part, hence plant contributions, and could have lead us to the next, tree based lifestyle. We could have learnt to stand in part from climbing up rocks to evaluate the terrain for dangers and opportunities.