Lenore Taylor – A photo album of the past 30 years of Australian/Indonesian relations would contain some memorable snaps. Look closely and a lot of them would show the same faces. Here's Prime Minister Keating and President Soeharto in the Presidential Palace, beaming as their foreign ministers, Gareth Evans and Ali Alatas, sign a security treaty in 1995. Here's ex-prime minister Keating at the president's home in 1998, offering advice and consolation as the Asian crisis closes in on Soeharto's ailing regime.

There's Evans and Alatas again, joking and arm-wrestling after signing the Timor Gap Treaty in an aeroplane above the Timor Sea.

There's a youthful Richard Woolcott, then Australia's Ambassador to Indonesia, at the Jakarta funeral of the five Australian journalists killed when Indonesian forces invaded East Timor in 1975. There he is again, ushering Australian editors into a meeting with Soeharto in 1996.

It's not only the same faces that reappear throughout the album. Put the images in context and it becomes clear that for all that time Australia has viewed Indonesia through the same frame.

For 30 years, ever-closer political, military, business, academic and media ties with our northern neighbour have been based on a foreign policy assessment that became an article of faith, an orthodoxy with a total stranglehold in the corridors of power.

There were dissenters – an array of exiled Timorese, activists, academics and members of the Catholic Church. But to the powerful elite they rated as little more than an annoyance.

The orthodox view among the elite was that Australia's security and economic interests were best served by Indonesia remaining a united and stable nation, that the reign of Soeharto was delivering that stability and economic growth, and therefore deserved Australia's support and friendship. Human rights blemishes had to be seen against that backdrop and should on no account overshadow the bigger, friendly picture.

It was a view actively promoted by the "Indonesia lobby", powerful politicians and opinion-makers who based their policy decisions, intellectual reputations, careers and often financial interests and personal friendships upon it.

It was embraced by the Department of Foreign Affairs and Trade (under successive secretaries, including Michael Costello – now chief of staff to Opposition Leader Kim Beazley – and present secretary Ashton Calvert, a former foreign affairs adviser to Paul Keating) in advice to prime ministers from Whitlam to Howard and through DFAT-funded organisations like the Australia Indonesia Institute.

It had bipartisan political support. It was espoused by academics, particularly Australian National University scholars like Jamie Mackie, Harold Crouch, Heinz Arndt and Hal Hill, some of whom received funding or consultancies from DFAT or agencies in Indonesia. It was advanced by journalists like Paul Kelly (who is also on the Australia Indonesia Institute's board and the Government's foreign affairs advisory council), and Greg Sheridan, both writing for The Australian.

It was argued in newspaper columns by former departmental secretaries like Richard Woolcott (founding chairman of the Australia-Indonesia Institute, one-time consultant to BHP on Indonesia-related issues, director of Bonlac, which exports to Indonesia), former ambassadors to Indonesia like Rawdon Dalrymple (also chairman of the consulting and investment business, the Asean focus group).

Former politicians like Keating, Allan Griffiths, Gerry Hand and John Button used their Indonesian contacts to further post-political careers.

The Australia Indonesia Business Council and its sister organisation, the Indonesia Australia Business Council, forged closer business ties. Educational links grew strongly.

The Australian and Indonesian militaries conducted joint exercises. Australia trained Indonesia's special forces troops, Kopassus, who were used for many years to maintain control in Timor, Aceh and Irian Jaya. Indonesian military figures invited top Australian brass to their daughters' weddings.

The political, academic, business and military networks were being woven ever tighter, based on the view that, under Soeharto or his successors, change towards democracy or greater respect for human rights would be at best incremental.

"At Government level," wrote Woolcott in 1995, "both Australia and Indonesia have attempted to construct a framework of institutional and high-level contacts, which would be strong enough to withstand occasional shocks."

Problems in the relationship boiled down to differences about human rights violations which had to be seen as "aberrations in a turbulent society's struggle for unity and stability".

Indonesians were aggrieved by anti-Timorese demonstrations in Australia, he wrote, and more and more Australians were also resentful of "the misuse by political refugees of the hospitality of this country".

But from 1997 the Indonesia lobby's world turned upside down. The Asian economic crisis and the political instability in Indonesia stunned them, but still they could not conceive of a fundamental change to the status quo in which they had invested so much.

Woolcott wrote in January 1998 that in Indonesia "there will be no people-power movement comparable to that in the Philippines in 1986", A month later Paul Kelly wrote that "Indonesia in the 1990s is not a re-run of the Philippines of the '80s. There is no political reform movement." By May, Soeharto had been forced from office.

And then this year the unthinkable happened. The Howard Government broke ranks. The Prime Minister declared that East Timor marked a turning point in Australia's relations with Indonesia and Asia, that he was abandoning the policy of "please at all costs" and that he would "defend the values we hold as Australians". According to the Indonesia lobby, Howard has trashed in a few months what had taken decades to build.

According to its critics, the Indonesia lobby should be eating large helpings of humble pie, with its views proved fundamentally wrong.

Keating, under whose prime ministership the relationship with Indonesia was at its closest, launched a passionate defence of his government's stance, saying it had been impossible to do more for Timor under Soeharto and that governments had to steer a course between competing objectives. "The human rights absolutists might not like it, but no government policy can base itself on absolute morality," he said.

Woolcott, who successfully nominated Alatas for an AO in 1995, who prepared Alexander Downer for his first meeting with Soeharto for a $13,000 fee, who has dedicated much of his working life to building relations with Indonesia, is in shock.

"Forty years of bipartisan effort to build up a relationship with Indonesia has been seriously eroded by recent events," he says, choosing his on-the-record words carefully.

"It is a matter of great regret that our policy towards Indonesia has been driven by our policy towards East Timor ... the Government has been excessively responsive to the East Timor lobby and pursued short-term populist policies rather than long-term visionary policies."

Dalrymple is incredulous. "The relationship has been destroyed," he says. "Indonesians feel betrayed by Australia, who they had come to see as a friend. There almost isn't a relationship any more." But according to Scott Burchill, lecturer in international relations at Deakin University, it is the Indonesia lobby's long-term strangehold on foreign policy that is the matter of great regret. "By any criteria they are totally discredited," he says. "There is nothing to show for their appeasement of Jakarta over several decades.

"They were complicit in denying basic human rights to the people of East Timor ... They were falling over themselves to develop a relationship with Soeharto, who I believe has a war crimes record comparable to Pol Pot ... They boasted of access to [the former head of Indonesian defence forces] Benny Murdani, who I regard as an unindicted war criminal."

Michael Backman, former Liberal Party staffer and author of Asian Eclipse: Exposing the Dark Side of Business in Asia, says the Indonesia lobby "ought to be very embarrassed".

"Their whole raison d'etre has been cut from under them. For years they have been telling the Government what to do and now it is no longer listening," he says. "Instead of trying to construct a way forward, they are all scrambling around trying to justify their past positions.

"The taxpayers have for a long time subsidised their activities, through DFAT and through academic funding. I think it's fair to ask what we have got for our money."

Another long-term critic of the orthodox view is Brian Toohey, former editor of The National Times and a columnist for this newspaper.

"The prevailing view was that it was smart to get closer and closer to the Soeharto regime as it became more corrupt, more violent and more offside with the Indonesian people," Toohey says.

"The core problem was that Australian policy actively supported the anti-democratic forces in Indonesia when our long-term interests were better served by a more democratic state. That's not a moralist argument, it's a realist argument. We were acting against our own interests."

But the Indonesia lobby remains convinced its stance was correct – although after the massacres in Timor some concede the close ties with the military may have been misjudged.

According to Sheridan, achievements like the Cambodian peace process and the establishment of APEC would have been impossible without sharing a close relationship with Indonesia.

"The Left say we should have destroyed our relationship with Indonesia in order to demonstrate our bona fides on human rights," he says.

"I disagree, and they are not entirely consistent because China and Vietnam also have far from perfect human rights records and I don't see the Left arguing we should cut off relations with those countries."

Dalrymple says it is "absurd to suggest that Australia should have put the well-being of some Timorese people above its overall national interest". Crouch, senior fellow at the ANU's research school of Pacific and Asian studies, says that while "quiet diplomacy" may not have prevented the present breakdown in relations, the "megaphone" diplomacy advocated by the Timor lobby would also certainly have failed.

But all three concede policy mistakes with regard to the military. "Training the Indonesian military did not lead to their substantial reform," says Sheridan.

"Some politicians and officials believed for too long the Indonesian military were redeemable. There were people who basked in the idea that they could be friends with a famous killer like General Murdani," says Dalrymple.

"The major flaw in policy towards the military was the belief that they were a defence force, like ours, but actually they are an internal security apparatus. That was always a mistake, and I said so at the time," says Crouch.

One of the Indonesia lobby's aims was to educate the Australian media about Indonesian culture, customs and sensitivities. The Australia Indonesia Institute organised regular meetings between senior Australian and Indonesian editors and political leaders, including in 1996 with Soeharto, and earlier this year with President B.J. Habibie and Timorese leader Xanana Gusmao.

Some are now blaming the Australian media for inflaming the present situation. Peter Church, founding board member of the institute and managing director of the Asean focus group, says: "The Australian media has a higher duty to think globally, to think about the incredible emotive impact it has in Indonesia when they publish something like that photograph of an Australian soldier standing with a gun behind an Indonesian with their hands on their head."

So where does the new reality of the Australian/Indonesian relationship leave the Indonesia lobby?

Economic links so painstakingly built have certainly suffered because of the economic crisis, and the present wave of anti-Australian feeling in Indonesia could make things worse.

"The business relationship has always weathered political ups and downs," says Church. "The new government will be more nationalistic and if it uses Australia as a whipping boy in the same way as [Malaysian Prime Minister] Mahathir has done, then business could be very affected."

Most of the former politicians have already abandoned or scaled back their business interests in Indonesia.

Button quit working as an adviser to the then Research and Technology Minister Habibie after a year because he "found it difficult working in an environment of what seemed to be chronic corruption".

Griffiths is also understood to have scaled back his business involvement in Indonesia. Hand worked as an adviser to Robbie Sumampow, who owned the Christmas Island casino until it was placed in the hands of an administrator in 1997.

Hand says he has no ongoing active business involvements with Sumampow. Keating, now an international business consultant, has had several clients with interests in Indonesia, including Macquarie Bank and Aus-pac Aluminium, proponent of a $3 billion aluminium smelter at Lithgow.

Aus-pac is run by Keating's business partner Chris Coudounaris and John Benson, the businessman who helped find an Indonesian buyer for Keating's share in that now-famous piggery.

Besides detrimental effects on economic links, there is the uncomfortable fact that many senior bureaucrats have held, and many Government advisers and people in senior institutional positions continue to hold, different views to the Government about the appropriate weight to be given to human rights in the setting of foreign policy. The policy and opinion-making apparatus is out of step with the policy.

When the time comes to start building from scratch a relationship with the new Indonesian regime, both sides are likely to tread far more cautiously. The passionate embrace of the past 30 years is unlikely to be rekindled. Dalrymple echoes the sentiment of the Indonesia lobby when he says of the Soeharto years, with some regret, "I fear we shall not see the like of him again."

Burchill echoes the sentiment of the Timor lobby when he responds that the Soeharto years "may have been diplomatically convenient for Australia, but won't be missed by the people who lived under repression".