It is not the case that property rights are merely granted to people by society. The most you can say about property rights and society, is that a society might respect property rights, yet it does not grant them.
The only way that morality could be constructed is by a system of property rights, hence its a pretty simple calculus that is not utilitarian so much as perhaps trancendental, or hypothetical. If one adopts a moralistic viewpoint, or wishes to construct ones behaviours and advocations in line with morality, then one must respect and abide by natural law property rights theory.
Granted, property rights theory might have no purchase with amoralists, but lots of people who profess to be so slip into moralist habits very frequently (in my experience), which gives me cause to doubt their honesty in this regard. Any ciriticisms such people might make about natural rights property theorists will be that their preferences differ from their own, and will not be critcisms which have any moral force.
Even amoralists may well concede that respecting property rights is ‘prudent’ since we know (by theory and empricial evidence) that this way of structuring human society confers material benefits (leaving aside the question of how much the material produced and distributed is valued) and the effect of greater individual freedom which some amoralists may appreciate.