Back in the mists of time when I worked in the House of Commons there were plenty of MPs who were at best on the fringes of their parties. They were widely regarded by their colleagues as idiosyncratic or eccentric; in some cases they had tipped over into monomania. But with no chance of them ever needing to exercise power, of becoming Ministers or Shadows, they were essentially seen as harmless, and often indulged and treated with some affection.

On the Labour side there was an ‘awkward squad’ which included people like Bob Cryer, Alan Simpson, Paul Flynn – and Jeremy Corbyn. They had a fine time doing things like objecting to private legislation (a particular bugbear for someone working in that area, as I was) and refusing to follow the Labour whip. For many of them this rebelliousness reached its peak when Tony Blair was in power: during the 2001-05 Parliament Mr Corbyn defied his party’s whip 148 times. The Conservatives had people like Bill Cash, Tony Marlow and Sir Teddy Taylor, and others who regularly abstained or voted against their party, for example over the Maastricht Treaty. And quite a few of the Liberal Democrat MPs of the time could reasonably be described as eccentric.

Everyone who worked in Parliament had a story to tell about their dealings with these folk, using told with a smile and a roll of the eyes. I once sought out Mr Cash, as he was then, for a discussion about a completely innocuous Statutory Instrument Committee he was due to chair and on which I was the clerk, only to find myself listening to a lengthy monologue about a dastardly – but entirely unrelated – act on the part of the European Union. But again, it didn’t really matter. I listened politely and the Committee itself went off without a hitch.

All of these people were a long way from power, so whether or not they were competent didn’t really matter. But then came 2015, and the strange combination of events that led to Jeremy Corbyn becoming leader of the Labour Party. Affection, or maybe the sense that he was a harmless outsider, was part of the reason he ended up in the top job: more than one of those who nominated him had no intention of supporting him. But as he got closer and closer to wielding real power many ended up regretting their decision. Bringing someone in from the fringes with no track record of competent delivery began to look like a big mistake.

Once in post those fears were realised. Mr Corbyn filled many senior positions in the party with fellow travellers from his part of the party, and for a while it looked like they had captured Labour forever. But it turned out that competence matters to voters. Keir Starmer realised immediately on becoming leader that the most important trait he had to demonstrate was that he knew what he was doing; he has made progress but the road back has already been a long one and he has much further to go. How he must regret the misplaced indulgence shown by far too many people who should have known better.

For a long time the Conservative Party looked on at the Labour Party’s travails with smug complacency. What happened with Corbyn could never happen to us Tories, has been the refrain; we would never allow our party to be captured by the lunatic fringe. And despite the party’s divisions over Europe for a long time it seemed that it would at least continue to promote competent leaders: Theresa May was not exciting but she appeared to be a solid senior manager. But once she went the slide into bringing in fringe figures of questionable competence accelerated.

It is all too easy to question the abilities of Boris Johnson. In any other circumstances would a man who was sacked by The Times and by Michael Howard and who was widely derided for his performance as Foreign Secretary (and who is now clinging on by his fingertips in No10) have been trusted with the top job? Of course not: he is the Conservatives’ Corbyn, but he made the right call, electorally, on Brexit, and he is significantly better on television. He does have some competent Ministers in his Cabinet: Rishi Sunak, Michael Gove and Therese Coffey are amongst those who seem to know what they are doing. But the list of people in important roles who in normal circumstances would have been obscure figures far from the levers of power does not begin and end with Jacob Rees Mogg. Whatever happens next there is an urgent job to do to root out those brought in from the fringes and promoted far beyond their abilities.

Because in terms of having a competent frontbench the Labour Party is much further down the road than the Conservatives: at least they have realised there is a problem. As I’ve said, Keir Starmer has made a good start, and in Yvette Cooper, Rachel Reeves and Wes Streeting, amongst others, he has put together a team that is starting to exude competence and also confidence. If they carry on this way, and the Conservatives don’t raise their game, then Labour will have a chance at the next election just on the basis of whether they look and act the part. Time for whoever succeeds Mr Johnson to return some fringe folk back to the margins.