The appointment coincides with the retirement of former chairman John Morschel from the ANZ board.
Mr Morschel said it was pleasing to have someone of Mr Gonski's calibre succeed him.
"[Mr Gonski] knows ANZ well having previously served as a director from 2002 to 2007 and we have been working closely since February to ensure a smooth transition," he said.
“It has been an honour to have served as an ANZ director since 2004 and to have been chairman of ANZ over the past four years," said Mr Morschel.
“It is very pleasing to see the success of the super regional strategy, which has positioned ANZ strongly for the future," he added.
Mr Gonski said it was pleasing to have the opportunity to rejoin the board and succeed Mr Morschel.
"I am looking forward to working with my fellow directors, and with Mike Smith and his management team to continue building a super regional bank," said Mr Gonski.
