Laws in a libertarian, or rather anarchist, society would not be designed rationally but come to be through legal precedent. Read that David Friedman stuff about conflict resolution agencies. Basically, everything would be private, including the judge, and if you sue someone that decision will become precedent for the next time someone has a similar disagreement. The laws would be shaped informally by what the judges decide. It would be a market in the sense that the laws are made in a evolutionary process, depending on who wins the cases. Nobody designs it.
And Eugene, the market is not a type of democracy. A democracy is a system where one group of people can force another group of people to do what they want. In a market everyone can do what they want themselves.