Nope. Sometimes it's the right choice to leave something in that your heuristic would lead you to rip out.

Can we stop with these simplistic maxims? Engineering is about tradeoffs and dealing with complexity. Trying to reduce the decision-making process to a single sentence is silly.