Nearly 23 years after five Australian-based journalists were killed in East Timor, the International Commission of Jurists has found that doubts remain that Canberra is telling all it knows about the incident. In a special investigation, Foreign Editor Hamish McDonald uncovers some of the facts surrounding their deaths and traces a cover-up that continues to this day.

One morning towards mid-October in 1975, a white Holden bearing the CD-18 numberplates that marked it as an Australian Embassy vehicle pulled off from the bustle of food and motor-tyre repair stalls on Jalan Tanah Abang IV into the driveway of a discreetly modern three-storey building.

In Jakarta's steamy heat two crisply dressed diplomats stepped from the air-conditioned car to the coolness of the lobby, and after a smiling welcome from the reception headed with a familiar step for the lift.

Upstairs in an office decorated with carved teak and elaborate Balinese paintings, the embassy's number-two official, Malcolm Dan, and its political section head, counsellor Allan Taylor, sat in armchairs facing two Indonesians in batik shirts, Harry Tjan Silalahi and Yusuf Wanandi.

The two Australians were to be given inside information of a detail, importance and timeliness that, had it come from a spy or decoded signal, would have been regarded as the intelligence coup of a lifetime. But what followed was one of the great bungles of Australian diplomatic history.

The building in which these four men met housed a private academic institution, the Centre for Strategic and International Studies, and the two Indonesians held no official position with the Indonesian Government or military.

But as with many aspects of the New Order political system then solidifying under President Soeharto, the outfit known as CSIS or "Tanah Abang" had much more clout than it pretended. Under the patronage of top intelligence chiefs, the CSIS had become a key player in Indonesia's second track of diplomacy.

As the four men sat around the coffee table in Tjan's office, Dan and Taylor were told that three days later, on the morning of October 16, 1975, some 3,200 Indonesian soldiers, mostly commandos of the elite special forces (then known as RPKAD, now Kopassus), would attack across the land border of East Timor, then legally a Portuguese territory, in three places. The aim would be to roll back the pro-independence Fretilin forces that had predominated in the short civil war which had started two months earlier.

This would be the invasion the Indonesians had been planning for nearly a year, and on which Dan and Taylor had been given regular briefings by CSIS. The two Australians were told the exact details, down to the unmarked Portuguese-style uniforms the Indonesian troops would be wearing to maintain "deniability" and the fiction that only pro-Indonesian local irregulars were involved. A force of about 800 troops would concentrate on the area of Balibo, a village around an old Portuguese fort overlooking the north coast region near the border, and nearby Maliana, opening the approach to Dili along the road from the west.

The news was cabled by Ambassador Richard Woolcott to the Department of Foreign Affairs that same day, October 13. Canberra had almost three days to prepare for what would inevitably be a controversial development for Australian public opinion to accept. It had two clear days to make all efforts to get Australian citizens out of the path of danger.

Revelation that this prior knowledge existed is the first clear evidence that the Department of Foreign Affairs has been concealing vital information on the murder of five television journalists in Balibo from the Australian public, Parliament and the bereaved families for more than 23 years to protect its pro-Indonesian policies.

The irony is that this extended cover-up has actually served to poison Australia-Indonesia relations ever since.

On October 9, Gary Cunningham, Greg Shackleton and Tony Stewart, a news team from Channel 7 in Melbourne, had flown from Darwin into Dili, the capital of East Timor. They were followed the next day by Brian Peters and Malcolm Rennie from Channel 9 in Sydney. Both teams headed out of Dili towards the border. In the pre-dawn hours of October 16, the five reached Balibo – in the direct line of attack.

The Department of Foreign Affairs in Canberra knew, at least by early on October 17, there were Australian journalists "outside Dili". It did not specifically know that these five were in Balibo, but might have expected the border was of most news interest.

Nothing was attempted to warn them directly. Nor was the Australian Embassy in Jakarta asked to intercede with its high-level Indonesian contacts to seek an urgent modification of the attack plans. Nothing was said or done to protect the five journalists.

It was, as one senior Australian official at the time admits, one of the most serious breakdowns ever to have taken place in our diplomacy. "The whole operation collapsed from there," the official says. "Everything collapsed, because of the failure to pick up that one bit of information."

It was a failure that nearly everyone involved has tried to paper over ever since. The connection that resulted in the Australian Embassy's astonishingly detailed advance reportage of Indonesian moves in Timor was built up in the latter months of 1974.

Australian Prime Minister Gough Whitlam's talks with President Soeharto in central Java in September 1974 concerned the future of East Timor, in question after the military coup in Lisbon five months earlier. Soeharto left thinking tepo seliru (in Javanese, mutual understanding) had been reached that the territory's integration with Indonesia was Canberra's preferred option. Subject to formal adherence to self-determination and avoidance of force, Australian policies would support that outcome.

As he had earlier with Irian Jaya, Soeharto gave the running of Indonesia's Timor campaign to long-time adviser Lieutenant General Ali Murtopo, deputy chief of the intelligence agency BAKIN, on the political side and to armed forces intelligence chief Major General Benny Murdani on the military side.

The elements of this now well-known campaign included promotion of local supporters, intelligence surveys, propaganda through a radio station in Indonesian Timor and newspapers run by Murtopo's people, exploratory contacts with key Portuguese and Timorese figures – and preparations for military intervention. The CSIS was the focus of diplomatic contacts, with its information supplied by both Murtopo and Murdani.

Within the Australian Embassy, Dan and Taylor took up the contacts with CSIS, which increased to daily contact once conflict broke out in August 1975. Ambassador Woolcott usually held the top-level meetings, including those with Murdani.

As 1975 wore on, the closeness of the contacts and the astonishing insights they brought caused unease in some quarters of Canberra. Whitlam saw foreign policy as his forte and had made unassertive West Australian Senator Don Willesee his Foreign Minister. Alan Renouf, who as Ambassador in Paris had set up Whitlam's ground-breaking trip to China, was made Secretary of the Department of Foreign Affairs, but soon lost Whitlam's trust.

Both found themselves out of the loop on Indonesia and Timor. Whitlam made Timor policy unilaterally, without reference to Cabinet or its subcommittee on foreign affairs and defence, as far as Renouf is aware.

Willesee was angered by Woolcott's practice of lobbying policies and perceptions directly to editors and other media figures. "I complained bitterly to Renouf, who rebuked Woolcott for doing that," says Geoff Briot, then Willesee's press secretary.

As well as Whitlam's office, the inner loop on Timor included Woolcott and his senior embassy staff, the department's first assistant secretary in charge of the South-East Asia division, Graham Feakes, and several other department officers such as Geoffrey Forrester, the Javanese and Indonesian linguist who acted as Whitlam's translator in his Soeharto meetings.

A file in the department's records shows that Willesee expressed deep concern about the nature of the information being obtained from Tjan at the CSIS and from Murdani. With so much detail, apparently accurate, Australia could be compromised, Willesee pointed out. Australia would be seen as a party to an invasion of Timor.

"CSIS was seen as an easy window into Indonesian thinking," Briot confirms. "What worried me was it was too easy. They were only telling us what they wanted us to know."

In a written reply to his minister, Feakes argued that the department was conscious of dangers, but that it was a two-way street. The contacts gave the embassy the opportunity to put views by Australia urging the non-use of force and self-determination, to influence Indonesian policy-makers, and to learn what the Indonesians were planning. It was often the main source of information.

When the attack came against Balibo on October 16, Australia's Defence Signals Directorate was monitoring the radio messages by the units involved. At some stage that day, DSD picked up a message from the officer running the attack that the bodies of white men had been found in Balibo and that all traces of them had been obliterated.

It was not until well into the following working day, Friday October 17, that a report from the Defence Department on the intercept was circulated around a tight circle in Canberra indoctrinated into the world of signals intelligence. It was also seen by the most senior officials in the Jakarta embassy.

But as late as that morning, the October 13 warning from the Jakarta embassy and the awareness of Australian TV crews operating in Timor was not setting off any urgent alarms. On October 17, department secretary Renouf sent a submission to Foreign Minister Willesee: the department was aware of Australian journalists in Dili "and some outside Dili", as well as several Australian aid workers in Timor. The department would "have to consider next week whether evacuation plans need to be implemented". By that stage, the five journalists had been dead for 24 hours.

The reaction in the embassy was one of dismay, followed by a quick move to pass blame. The diplomatic cables exchanged between Jakarta and Canberra were widely copied and passed around the department, and one official recalls them in some detail. He remembers Jakarta saying in a cable to Canberra on October 18: "The news of the death of the five Australian journalists in Balibo came as a great shock to us all in the mission. But it has to be said that they took their lives in their own hands in exposing themselves in the frontline of the attack. It was foolhardy and unnecessary, and the blame must rest with them and their employers...

"The mission has reported over a long period the plans of Indonesia to take East Timor by force, and more recently, the specific plans of the invasion. It must be assumed that Australian nationals were warned by Canberra presumably of the dangers of travel in Timor at this time, and in particular that special briefings were given to the management of the media organisations."

Renouf exploded when he received this. "I wish to know by return telegram who in the mission sent this telegram," he messaged back.

Woolcott replied: "You know that we have all been under considerable pressure over an extensive period of time. I had to leave for a tour of the Java provinces the day after news of the tragic deaths of the journalists. In my absence Malcolm Dan sent the telegram that was intended only to make the point that reports on the impending Indonesian attack on Timor had been known well before the invasion and that the journalists would clearly have been forewarned of the dangers."

Renouf was already taking steps to check on that aspect, ordering the assistant secretary in charge of the department's executive branch (a secretariat to the department head), Geoffrey Miller, to see what warnings had been given.

In a submission to the Foreign Minister on October 20, Miller reported that the Department of Civil Aviation issued an instruction that all personnel boarding civilian flights from Darwin to Timor had to be warned of the dangers. The Channel 7 crew had left Darwin on October 9, the Channel 9 team the next day. The pilots of both flights had received the official warning. But the three who left on October 9 did not receive the warning. The flight on October 10 had left without proper authorisation, and so the two journalists did not receive the warning, though the pilot had when he received some earlier papers on the flight.

For two or three days, Willesee agonised over the knowledge that the journalists were in all probability dead, while family and colleagues of the five pressed inquiries.

"After all, most of his own kids had gone into journalism," says former press secretary Briot, now a senior official in the NSW Ombudsman's Office. "He really was aggrieved by the fact that we knew through the DSD intercepts that they had been killed."

Defence officials wanted to keep their intelligence methods secret. "At that time he [Willesee] was under intense pressure from the department not to reveal that knowledge," says another Willesee staffer, then diplomat, Alan Oxley. "The argument at that time was that if he did, he would have revealed the existence of intelligence gathering. Since then that convention has gone by the board. About three years later Malcolm Fraser broke that convention and we've been much less coy about it."

Despite the worries, the television station managements were discreetly notified over the weekend, and they in turn notified some family members. By the Monday, October 20, reports indicating the deaths also came out in the Jakarta press, passed on by the Australian Embassy, which allowed Canberra to cite an open source for what it already knew.

The department never conducted a further review of its failure to put all its information together and attempt to save the lives of two Australian citizens and three others resident in Australia working for Australian companies.

Woolcott later alluded to the key October 13 cable (published in the Herald of October 14, 1995) saying it included specific advice to warn any Australians in the Balibo area to get out. Yet Gough Whitlam, in his recent book Abiding Interests, states: "I am advised that I should not yet reveal why we did not know of the incursion across the border to Balibo and why we were able immediately afterwards to learn that five men had been killed."

Could they have been saved? Many people in the department point to the difficult logistics of getting an authoritative message out to the journalists in time. "Communications were poor," says Briot. "Just how you would have warned, or have got a warning to them is difficult to say." As to the lack of formal warning, Oxley asks: "Even if they had got one, would it have stopped them?"

But it is conceivable that direct and early advice to the Channel 7 and 9 managements might have resulted in telexes to Dili ordering the teams back, telexes which might have been carried to them in time. Such orders even could have been conveyed through Radio Australia, which all reporters covering Timor would have monitored.

But the other recourse would have been to tell the embassy in Jakarta, which would have resulted in an immediate approach by Woolcott to General Murdani, or even to Soeharto himself, to seek urgent steps to protect the five from the Indonesian attack.

From then on, the department's efforts to investigate the Balibo deaths were limited by two concerns: on one side to hide the signals intelligence from the Indonesians, on the other to hide the secret information given by the CSIS from the Australian public.

Political turmoil in Canberra, resulting from the dismissal of the Whitlam Government on November 11, 1975, freed the department of close political supervision for a significant period.

For the first attempt to find out more details in Timor itself, the Jakarta embassy sent third secretary Richard Johnson, to Kupang, in Indonesian Timor. Though a fluent Indonesian speaker, he was stonewalled and returned to Jakarta with little more than the standard line. Neither did a more senior officer, political first secretary Peter Rodgers, who joined him in Kupang, bring any better results.

The crowning example of selective disclosure of this period was the embassy's mission into East Timor from April 28 to May 10 in 1976, which visited Balibo twice. Including Johnson and the embassy's consular section head, David Rutter, it was led by Allan Taylor, the political counsellor who had been one of the two embassy officers given the advance briefing by the CSIS on the Balibo attack.

By that stage the embassy was also aware, through intelligence contacts in Jakarta itself, of the identity of the Indonesian officers involved in the Balibo attack, including that of the officer on the ground, a young RPKAD captain named Mohommed Yunus Yosfiah, who ironically became Indonesia's Information Minister earlier this year.

Not a word of this knowledge was contained in the Taylor mission's report, which was advised to Parliament on June 2, 1976, by the new Foreign Minister Andrew Peacock, and placed in the Parliamentary Library soon after.

The reason for this appears to be the mission's terms of reference contained in the preamble that "the team based its findings on the information obtained during its two visits to Balibo" meaning it would be an extremely limited exercise barring a major mistake by the Indonesians orchestrating the witnesses seen by the Australians.

Unsurprisingly, Peacock was only able to tell Parliament: "I regret that it is still not possible to come to firm and final conclusions as to the circumstances and manner of the deaths of the newsmen."

But perhaps the shabbiest exercise of all in the department's handling of the Balibo case was the funeral for the five journalists. On November 12, 1975, the head of the intelligence agency BAKIN, Lieutenant General Yoga Sugama, called in Ambassador Woolcott and handed over four shoe boxes of charred bone fragments said to be the remains of the journalists, along with some camera equipment, notes and personal effects, including Greg Shackleton's diary.

In a submission late in November to Peacock, the department's South-East Asia division head Feakes warned of public support for the bones to be returned to Australia for laboratory testing. (The Jakarta embassy's doctor, Henry Will, had been able to state only that the remains "appeared to be human".)

The department "must not favour this" because it could lead to an "anti-Indonesian campaign" and cause "public outrage", Feakes urged.

Feakes attached a draft of a letter for Peacock to send to the next of kin, advising them of the deaths and extending the Government's condolences. One government official at the time recalls Peacock returning the draft, demanding it be rewritten in less bureaucratic language and with more compassion.

The letters arrived around the time the remains were buried in a Jakarta cemetery, in a single coffin, in a ceremony attended by several embassy staff and their spouses and resident journalists.

Some of the relatives had agreed with the Jakarta burial anyway. "When I heard the bodies had been burned I said I didn't want the remains back to Australia, and made that known to Foreign Affairs," says Gary Cunningham's father, Jim. "It would have been too harrowing."

But others say they were hustled and misled into agreeing. Greg Shackleton's widow, Shirley, remembers the call from the department: "I was told "If you want to bring the remains back it would cost a lot and you would have to pay'. I didn't think they were the remains anyway, and said do what you like."

Brian Peters' sister, Maureen Tolfree, had been in Australia making inquiries and was on her way back to Britain just before the funeral, on a flight that stopped in Jakarta. She made an impromptu decision to get off in Jakarta to see if she could collect Peters's remains to take back for burial. On arrival, she was taken to a small office at the airport and kept until a British Embassy officer arrived. He advised her to get back on the plane, which she did.

A former fiancee of one of the journalists, who has asked not to be named, remembers as "quite cold" the person who rang from the department. "Foreign Affairs conned each family by saying the other families did not want the remains brought back," she says. Adding to her anger, she learned only in 1994 that two letters written by her fiance were in the effects handed back by the Indonesians.

The Government's handling of the Sherman Inquiry in 1995-96 has disappointed many interested parties. Notably, that although Sherman said his work was "preliminary" Foreign Minister Alexander Downer soon after ruled out the value of any further inquiry outside Indonesia.

Some Timorese exiles, too, had questioned the selection of Tom Sherman, a career government lawyer previously chairman of the National Crime Authority, arguing that while there was no question of his integrity, it allowed a perception that the inquiry was not completely independent of the Government.

Sherman interpreted the charter given by Evans – to inquire into "the circumstances surrounding the deaths" – in a narrow sense, excluding all questions about the department's conduct.

Most puzzling of all, he made no attempt to contact any of the department's officers working on Timor in 1974-76. Former ambassador Woolcott made a written submission to the inquiry, on his own initiative. Some of the other officers are now dead (Graham Feakes and Michael Curtin) but the others – including Allan Taylor (now heading the Australian Secret Intelligence Service), Malcolm Dan and Alan Renouf – are still around.

The department denies any instruction to clam up. "Mr Sherman was free to contact any individual, either in Australia or abroad, whom he considered to have relevant information on the deaths of the journalists," it said in a written reply signed by Colin Heseltine, the Maritime South-East Asia Branch head.

"The Department of Foreign Affairs and Trade did not, at any stage, instruct any individual not to come forward to Mr Sherman. Nor did the department attempt to exert any influence over individuals with whom Mr Sherman initiated contact in order to obtain evidence."

Nor does Sherman appear to have tapped the voluminous cable traffic between the Jakarta embassy and Canberra around the time of the Balibo attack, though the department says he had "unfettered access" to all its records and files, and routinely requested information from them.

Sherman does say that shortly before closing his inquiry, he asked for and was given access to intelligence material relating to Balibo, which he perused in one day and reported not to have any material "of sufficient evidentiary value" to warrant inclusion in the report, or cast doubt on any of its conclusions.

According to senior sources close to the department, this focus entirely misses the point. Canberra's best knowledge about Indonesia's Timor campaign in 1975 was not a result of intelligence work, but the tainted fruit of the connection built up by the Jakarta embassy with the Indonesians running that campaign. It was inside knowledge.

The sources also warn that it cannot be assumed that all will be revealed in seven years, when the files are opened under the 30- year rule. For one thing, the department has already withheld hundreds of pages of documents relating to Indonesia from records up to 1967 already opened.

But the sources also claim that the Timor records, occupying some nine metres of shelf space, are not secure and are open to interference. Documents are being lost, and it is not unknown for fabricated notes and advice to be slipped into files to enhance reputations.

In particular, the department's copy of the record of Whitlam's meeting with Soeharto in September 1974 is said to be missing from the DFAT archives.

The department's Heseltine says he "was not aware of the precise location of the document to which you refer. There are a number of historical documents that are currently being examined by this department with a view to their archiving and long-term storage. The document to which you referred may well be amongst them."

The department was also unable to locate immediately the October 13 cable from Woolcott warning of danger on the Timor border, though it had found a reference to this cable in a subsequent cable.

Bruce Haigh, who ran the department's Indonesia desk in 1984- 86, recalls that two files of key documents on Timor and Indonesia, including photographs of Balibo, were withheld from the DFAT registry and kept in a safe in his section.

"I said I didn't want to hang onto it any more, that no files should be kept like that," said Haigh, who is now a farmer. "It was illegal, I thought, to hold files like that. They should be in the registry, or they shouldn't exist at all, one of the two. They probably don't exist any more."