As western military intervention ends, it is worth asking what are the causes of this humiliating debacle. Why are so many Taliban willing to die for their cause, while government soldiers take flight or surrender? Why is the Afghan government in Kabul so corrupt and dysfunctional? What happened to the $2.3 trillion spent by the US, trying, and failing, to win a war in a country that remains miserably poor?

More generally, why did what was presented as a decisive victory by US-backed anti-Taliban forces twenty years ago turn into the present rout?

One answer is that Afghanistan – like Lebanon, Syria and Iraq – is not a country where the word “decisive” should ever be used about any military victory or defeat. Winners and losers do not emerge, because there are too many players, inside and outside the country, who cannot afford to lose, or to see an enemy win.

Simple-minded analogies with Vietnam in 1975 are misleading. The Taliban does not have anything like the military might of the North Vietnamese army. Moreover, Afghanistan is a mosaic of ethnic communities, tribes and regions, which the Taliban will struggle to rule whatever happens to the Kabul government.

The disintegration of the Afghan army and security forces has accelerated the Taliban’s attack, which has often faced little resistance, and has enabled it to make spectacular territorial gains. Such rapid changes of fortune on the battlefield in Afghanistan are traditionally fuelled by individuals and communities swiftly changing to the winning side. Families send their young men to fight for both the government and the Taliban as a form of insurance. Swift surrenders by cities and districts avoid retribution, while over-long resistance leads to massacre.

There was a similar pattern in 2001. While Washington and its local allies in the Northern Alliance were trumpeting their easy victory over the Taliban, the latter’s fighters were returning unscathed to their villages, or slipping across the border into Pakistan to wait for better days. These came four or five years later, when the Afghan government had done enough to discredit itself.

The great strength of the Taliban is that the movement has always had the support of Pakistan, a nuclear-armed state with a powerful army, a population of 216 million, and a 1,616-mile-long border with Afghanistan. The US and UK never sufficiently took on board that unless they were prepared to confront Pakistan, they could not win the war.

Other strengths of the Taliban are a core of fanatical, experienced commanders and fighters who are rooted in the Pashtun community, which makes up 40 per cent of the Afghan population. A Pakistani colonel commanding irregular Pashtun troops just across the border from Afghanistan once queried me about American and British efforts “to win hearts and minds” in heavily Pashtun southern Afghanistan. He thought their chances of success were low, since, he said, experience had taught him that a central feature of Pashtun culture was that “they really hate foreigners”.

Propaganda about “nation-building” by foreign occupiers in Afghanistan and Iraq was always patronising and unrealistic. National self-determination is not something that can be fostered by foreign forces, whatever their supposed good intentions. They invariably consult their own interests first and last, and the Afghan government’s reliance on them delegitimised it in the eyes of Afghans, depriving it of roots within Afghan society.

The vast sums of money available because of US expenditure produced a kleptocratic elite. The US has spent $144bn on development and reconstruction, but some 54 per cent of Afghans live below the poverty line, with earnings of less than $1.90 a day.

An Afghan friend who had once worked for United States Agency for International Development (USAID) explained to me some of the mechanics of how corruption was able to flourish. He said that American aid officials in Kabul thought it too dangerous for them personally to visit projects they were financing. Instead, they stayed in their heavily defended offices and relied on photographs and videos to show them the progress of the projects they were paying for.

On occasion, they would send an Afghan employee like my friend to see for himself what was happening on the ground. On a visit to Kandahar to monitor the building of a vegetable-packing plant, he discovered that a local company akin to a film studio would, for a fee, take convincing pictures of work in progress. Using extras and a suitable backdrop, they were able to show employees in a shed busily sorting carrots and potatoes, though no such facility existed.

On another occasion, the Afghan aid official had discovered evidence of a fraud, though this time there had been little attempt to conceal it. After looking in vain for a well-funded but non-existent chicken farm near Jalalabad, he met with its proprietors, who pointed out to him that it was a long road back to Kabul. Interpreting this as a threat to murder him if he exposed them, he kept quiet and resigned from his job soon afterwards.

Foreign aid has built real schools and clinics, but corruption has corroded every government institution. On the military frontline, it means “ghost” soldiers, and the garrisons of threatened outposts left without enough food and ammunition.

None of this is new. Visiting Kabul and other cities over the years, I came to feel that the Taliban had limited support, but that everybody viewed government officials as parasites to be circumvented or bribed. In Kabul, a prosperous real-estate dealer – not a business normally sympathetic to radical change – told me that it was impossible for a system so saturated by corruption “to continue without a revolution”.

Instead, government failure has enabled the Taliban to believe that they can return to power within a year. Such a prospect frightens many people. What, for instance, will be the response of the 4 million Hazara minority, who are Shia by religion and are close to Iran? Earlier this year, bombs in Kabul killed 85 Hazara girls and teachers as they left their school. As in 2001, the forever war in Afghanistan is far from over.