While in the desert, Moses' periods of seclusion on the mountain were key to his progress as a leader and the transmission of The Torah. Like any introvert, only by entering into a safe, individual space could this have happened.
Remember that there is a higher reality in the world. The material here and now is not all there is. Nobody can ever be perfect in the material world, but we darn well can live life with purpose and with meaning.