The White House is revising its Afghanistan strategy to embrace the idea of negotiating with senior members of the Taliban through third parties – a policy to which it had previously been lukewarm.
Negotiating with the Taliban has long been advocated by Hamid Karzai, the Afghan president, and the British and Pakistani governments, but resisted by Washington.
The Guardian has learned that while the American government is still officially resistant to the idea of talks with Taliban leaders, behind the scenes a shift is under way and Washington is encouraging Karzai to take a lead in such negotiations.
"There is a change of mindset in DC," a senior official in Washington said. "There is no military solution. That means you have to find something else. There was something missing."
That missing element was talks with the Taliban leadership, the official added.
The American rethink comes in the aftermath of the departure last month of General Stanley McChrystal, the top US commander in Afghanistan.
Barack Obama, apparently frustrated at the way the war is going, has reminded his national security advisers that while he was on the election campaign trail in 2008, he had advocated talking to America's enemies.
America is reviewing its Afghanistan policy which is due for completion in December, but officials in Washington, Kabul and Islamabad with knowledge of internal discussions said feelers had been put out to the Taliban. Negotiations would be conducted largely in secret, through a web of contacts, possibly involving Pakistan and Saudi Arabia or organisations with back-channel links to the Taliban.
"It will be messy and could take years," said a diplomatic source.
The change of heart by the US comes as Afghanistan hosts the biggest international gathering in its capital for 40 years, with representatives from 60 countries including Hillary Clinton, the US secretary of state, and Ban Ki-moon, the UN secretary-general.
The dominant theme of the Kabul conference is "reintegration", which involves reaching out to low-level insurgents to encourage them to lay down their arms.
Earlier this year Richard Holbrooke, Obama's special envoy to Afghanistan and Pakistan, distinguished between "reintegration", which the US supported, and "reconciliation" or negotiating with senior Taliban. Holbrooke said: "Let me be clear. There is no American involvement in any reconciliation process."
There is growing disenchantment in the US with the war in Afghanistan and members of the Senate's foreign relations committee last week questioned Holbrooke over what they described as a lack of clarity on an exit strategy.
The US has no agreed position on who among the leaders of the insurgency should be wooed and who would be beyond the pale. The Taliban leader, Mullah Omar, would be a problem as he provided Osama bin Laden with bases before the 9/11 attacks.
The US would also find it problematic to deal with the Pakistan-based insurgents led by Sirajuddin Haqqani, whose group pioneered suicide attacks in Afghanistan. The third main element in the insurgency is Gulbuddin Hekmatyar, who has hinted he is ready to break ranks.
A source with knowledge of the process said: "There is no agreed US position, but there is agreement that Karzai should lead on this. They would expect the Pakistanis to deliver the Haqqani network in any internal settlement."
The US has laid down basic conditions for any group seeking negotiations. They are: end all ties to al-Qaida, end violence, and accept the Afghan constitution.
A senior Pakistani diplomat said: "The US needs to be negotiating with the Taliban; those Taliban with no links to al-Qaida. We need a power-sharing agreement in Afghanistan, and it will have to be negotiated with all the parties.
"The Afghan government is already talking to all the shareholders‚ the Taliban, the Haqqani network, Gulbuddin Hekmatyar, and Mullah Omar. The Americans have been setting ridiculous preconditions for talks. You can't lay down such preconditions when you are losing."
Some Afghan policy specialists are sceptical about whether negotiations would succeed. Peter Bergen, a specialist on Afghanistan and al-Qaida, told a US Institute of Peace seminar in Washington last week that there were a host of problems with such a strategy, not least why the Taliban should enter negotiations "when they think they are winning".
Audrey Kurth Cronin, a member of the US National War College faculty in Washington, and the author of How Terrorism Ends, said talks with Mullah Omar and the Haqqani network were pointless because there would be no negotiable terms.
She said there could be talks with Hekmatyar, but these would be conducted through back channels, potentially by a third party. Given his support for jihad, she said, "it would be unreasonable to expect the US and the UK to do so".
Asked how Obama's Afghan strategy was progressing, a senior former US government official familiar with the latest Pentagon thinking said: "In a word, poorly. We seriously need to be developing a revised plan of action that will allow us a chance to achieve sufficient security in a more sustainable manner."
Officials have mentioned possible roles in negotiation for the UN and figures such as the veteran UN negotiator, the Algerian Lakhdar Brahimi, who heads, along with the retired US ambassador Thomas Pickering, a New York-based international panel which is looking at such a reconciliation.
Another name mentioned is Michael Semple, an Irishman based in Boston at Harvard's Kennedy School who has extensive contacts with the Taliban.
