None of the four who were arrested had been on a national security watch list, Bush said, adding they were not known to Australian authorities either.
Morrison said Australians were shocked, appalled and outraged by the attack.
Three others remain in custody.
The foreign ministry says there are 330 Indonesian citizens in Christchurch, 130 of them students.
He added that there is no assumption the attack was contained to Christchurch. My condolences to the families of the victims.
The Bangladesh cricket team was in the vicinity of the shooting but all members were safe, a team coach told media.
Meanwhile, New Zealand police said they were hunting for "an active shooter" in the centre of Christchurch city. Hospital staff are expecting about 40 to 50 injured people, and private hospital Southern Cross has also been cleared to treat the injured.
"Authorities said they had four people in custody - three men and one woman - but later clarified that only three were believed to have been involved in the violence", The Washington Post reported.
New Zealand Police Commissioner Mike Bush said anyone thinking of going to a mosque should stay put and close their doors.
She said there police have got no reason to suspect there might be others out there, but they have not ruled it out. Police said the second mosque attacked was in the suburb of Linwood, but gave no details.
Christchurch, a relatively small city in the south of New Zealand, hit global headlines in 2011 when it was struck by a deadly natural disaster.
Prime Minister Jacinda Ardern condemned what she called "an extraordinary and unprecedented act of violence".
"Many of those who would have been affected by this shooting may be migrants to New Zealand", Ardern said. "He's an evil, hateful person who shouldn't be given a platform", she said.
Just before the alleged gunman opened fire, he urged viewers to subscribe to the popular YouTube channel PewDiePie, which itself has been criticized for posting offensive footage in the past.
A gunman, who identified himself as Brenton Tarrant from Grafton, Australia, shot at around 100 defenceless worshippers attending Friday prayers at the Al Noor mosque, with many jumping out of windows to escape, reports the NZ Herald. He also claimed that he saw the gunman flee before emergency services arrived.