Tuesday lunchtime in Old Harry’s Bar was packed to the rafters with delegates promised something better than food: a list. Don’t we just love lists? Not this time the 10 most influential fluffy bloggers, but a ranking of how liberal the 63 Liberal Democrat MPs are on the basis of Parliamentary votes and Early Day Motion (EDM) support, on issues relating to personal liberty – i.e. drinking and smoking, rather than tax and CCTV. Nobody really believes the methodology behind the list to be sound, but, hey, it’s just a bit of fun, isn’t it?

I was quite pleased to observe that the most liberal candidate in each of the last two leadership elections, won that election. Ming scoring 40%, Nick Clegg 38.9%, Chris Huhne 36.1% and Simon Hughes 35.0%. Vince Cable came third from last with 22.5%. And would Lembit have followed this pattern, coming first with 55%? I was less pleased when I remembered that party leaders are not allowed to sign EDMs, so this may just be an artefact of flawed methodology.

Anyway, the meeting had nothing to do with this statistical gratification, which we had to pick up at the end. Instead we were treated to a series of more or less frighteningly libertarian speeches, and some sanity from Malcolm Bruce MP (35%).

First up was some guy from the Institute of Economic Affairs who argued that we are missing out on many benefits by banning too many things, as detailed in some book he was plugging. So, for example, the many lives would be saved if we permitted the sale of organs, and that laws against drugs and prostitution cost more in terms of promoting criminality and so on, than they are worth.

He also pointed out the extent to which casinos lobbied against the supercasinos. And this a fair point. You can lose your money as fast as you want on the Internet. The size of a casino does not limit how quickly it can take all your money, and it is just protectionism to allow these casinos and not those.

The sale of organs point raised some discussion. Wouldn’t this lead to gross exploitation of the poorest in society? No, the retort came, you could still choose whether to sell a spare kidney. Personally I think the incentive point is guesswork, and incentives can sometimes lead to reductions in supply as willing donors may walk away.

But it may still be correct, in which case the sale of organs would seem to be justified on consequentialist (utilitarian) grounds. Yes, it involves exploitation, but only because dire poverty exists and is so terrible that it can be worse than selling an organ. We are used to the poverty and not to the organ trade, and this seems to bias our instinctive reaction.

The second speaker was former Lib Dem staffer Mark Littlewood, who, if I remember correctly, once had a big rant on Newsnight once about how terrible the Lib Dems are for wanting to raise taxes, at a time when our policy was to not raise taxes. Today he admitted being such a big drinker, smoker and gambler, that in arguing for greater acceptance of these vices, he had been accused of special pleading.

Would the same point have been made if he was talking about being gay, he asked, provoking more debate. Isn’t this debate over whether something is a choice or is intrinsic to your nature, missing the point? There are no grounds for prohibiting it either way.

Next was Malcolm Bruce, explaining his position on gun controls, and generally arguing against the absolutism of what is usually called libertarianism, and that there are conflicting rights in areas such as smoking in company. He was right, of course: “libertarians” seek to resolve any possible conflicts of rights, by elevating the sanctity of property above all else. There was even much talk from other panellists of “self-ownership”, which is libertarian-speak for the assertion that property is sacred because it is like your relationship with your own body, rather than being something much weaker and rightly defined by laws passed by democratic legislatures. (And if I merely owned myself, would I lose my organs on bankruptcy?)

Finally we had Gavin Webb, complete with a magnificent neatly trimmed beard, that put every other beard at conference to scruffy shame. He recounted the issues behind his expulsion from the party over his extreme views on drugs, prostitution and drunken driving.

The discussion covered much ground, but struggled it seems to engage with these libertarian arguments without reverting to a left-right analysis: aren’t you just terribly right wing, thus wanting the vulnerable to curl up and die, rather than bother us? This is a fair question to some “libertarians”, particularly among the Tories, but I think a better and clearer enagement can be found if we go back to the “harm principle” of JS Mill. The harm principle states that the only valid grounds for banning something is that it causes harm to others; if it is bad for the person committing the act, that is not grounds for a ban.

The problem is that in the real world (unlike the atomised “libertarian” vision in we are each free of our neighbours in our sniper position on our house in the middle of the ranch, enforcing our laws on our own land), almost every action has an effect on somebody else. Tea or coffee? Some farmer’s livelihood depends on your answer. Start a business and you may drive a competitor to poverty, desperation and suicide. So the “harm principle” could be used to justify total tyrannical micromanagement of our lives.

Except that the harm principle doesn’t say that – it says that harm to others is necessary before somthing is banned, not that it is sufficient; and it is not sufficient as my examples show. Libertarianism could be seen as the bare minimum application of the harm principle, and communism as its maximum application. But Mill was a utilitarian, and the harm principle should be seen in this context. Utilitarians weigh up cost and benefits. Banning anything is a cost, and harm is a cost, and we must weigh them.

Juding from the spin with which their report on the Tory/Lib Dem battleground was launched, Liberal Vision is a organisation that is hostile to the Liberal Democrats. That is a shame, even if they are frighteningly libertarian to the point of being hostile towards democracy, because they do make some arguments that deserve engagement. It is too easy when you are trying to make policy for a government, to say that the state should simply do whatever it is that is required. Understanding how things get done organically, without government dictat, is always more difficult, but it is a challenge that we must rise to, to deserve the name liberal.

* Joe Otten is a Liberal Democrat member in Sheffield.