My caveman brain agrees too. We have evolved to have ideas about what our fellow man is experiencing. This says nothing about the philosophical validity of this empathy.
Rather than paint the world as less happy, I am proposing that we can say such a world is unjust, or disutilitarian (my conception).
What you mean is that “all plans are equal in aggregate good”. No plan can ever be evaluated for this parameter becuase it does not exist. Aggregate ‘good’ is a linguistic invention, nothing more. It only holds our attention because of our evolutionary instincts that I explained about.
Thus my Utilitarianism abandons trying to maximize ‘overall good’ or whatever. The only good that can be advanced is subjective and individual. You cannot make interpersonal comparisons of this property. The only way to promote this kind of good is to preserve people’s freedom to pursue their goals.
This approach frustrates any attempt to sacrifice one for the many, because it must rely on some conception of aggregate good, which is meaningless.