The army intervened on Wednesday, days after Mr Mugabe sacked his deputy, signalling that he favoured his wife Grace as his successor.
Mr Mugabe, who is confined to his house, was to meet army chiefs on Sunday, state TV said. The 93-year-old has led Zimbabwe for 37 years.
The military says it will advise the public on the outcome of talks "as soon as possible".
Saturday's rally had the support of the army and members of the governing Zanu-PF party.
Veterans of Zimbabwe's war for independence - who until last year were loyal to the president, the best-known among them - are also saying Mr Mugabe should quit.
Outside State House, the official residence, some people staged a sit-down protest in front of a line of troops, and opposition leader Morgan Tsvangirai addressed the crowd, to cheers.
The BBC's Andrew Harding in Harare says this is a watershed moment and there can be no return to power for Mr Mugabe.
Our correspondent says the situation may appear to be getting out of Zanu-PF's control and there could be a broad push to introduce a transitional government that includes the opposition.