This is a good excerpt by Rothbard from The Ethics of Liberty. But the basic idea is reciprocity: “life for life, eye for eye, tooth for tooth…” I wrote a little bit about my thoughts on it here. Most people find restitution to be preferable (and if true restitution/restoration/undoing could be done, I think almost all people except for the really sick and twisted would prefer that to retribution), but retributive justice is the basis for restitution. If you beat me up, and whether or not I want restitution, I may not beat you up more than what you did to me, else I would be in the wrong.
Not all libertarians agree on this, but it’s a fairly standard viewpoint.
The way I see it, it all depends upong what the aggressor is doing. If some guy pulls a gun on you, he has certainly threatened you with deadly force. You are within your rights to use deadly force too, as the aggressor is the one who first raised the conflict to that level. But if the guy just grabbed your wallet out of your hands (maybe you were paying at the store), you do not have a right to just shoot him. He has certainly wronged you, but not to the same degree that you would be acting against him. If you just shot him, you would be in the wrong.
Someone might try to pose a scenario where the mugger’s gun isn’t loaded and you don’t know it. But it’s beside the point. The point is that the mugger has raised the level of conflict to deadly force. He has not only threatened your life, but he intends for it to be a credible threat. Anyway, I’m a little tired*, so this post is probably a little out of order, but Rothbard’s chapter is probably far more illuminating (I can’t remember if I agree with everything he says in that chapter, but I certainly agree with the general gist).
*Not that I’m going to go to sleep right now.