Here is a simple recipe I have learned for efficient software design:
Less planning, more prototyping!
Planning is typically a long and expensive process. Some “experts” justify it by claiming that one week of planning saves ten weeks of programming. In practice, this payoff rarely comes. Why? Because as you make progress, the project itself changes and the planning becomes obsolete.
I am not saying you should not manage and plan your projects. However, all planning should be short-term. The only question you must answer is:
What is the most efficient use of my time today?
In short, I advocate you follow a greedy algorithm to manage your time and your projects. Given that your problems are ill-defined and constantly changing, maximizing your short-time efficiency is the only sane thing to do.
Stop investing your time today in the hope that you will be efficiency tomorrow. Be efficient now!
Of course, if you live in some huge slow-moving corporation and work on problems that will remain the same for decades, then please do plan ahead!
Update: What if you are asked to provide engineering-like schedules and deadline? Make them up.