Zimbabwe’s only previous success against the world’s top-ranked 50-over side came in their very first meeting in Nottingham in 1983 and since then have endured 27 defeats and one abandoned match.
After Australia elected to bat, the tourists were restricted to 209 for nine in their 50 overs, a target Zimbabwe chased down with two overs to spare.
Chigumbura came to the wicket with his side floundering on 106 for five at the halfway point of their innings, and along with number nine Prosper Utseya (30 not out), added 55 in an unbeaten eighth wicket partnership.
The Zimbabwe skipper was unbeaten on 52 from 68 balls and provided the anchor for the win.
Utseya scored the winning runs by belting a six off Mitchell Starc that prompted a field invasion by jubilant fans who had never seen a day like it at the Harare Sports Club.
“I had to be there at the end and the most important thing was to stay calm under pressure,” Chigumbura said in a pitchside interview.
“The guys have worked really hard for this and the pleasing thing was that everybody contributed today.”
Adding to Australia’s misery, skipper Michael Clarke had to battle through the pain barrier to score an unbeaten 68 after he suffered a recurrence of his hamstring injury.
Clarke, returning to the side after missing Australia’s first two games of the series, retired hurt in the 43rd over as he battled with his hamstring, but returned for the final two balls of the innings to ensure the tourists were not bowled out.
Wicketkeeper Brad Haddin scored 49 to help the visitors pass 200 as he put on 50 with Clarke before the latter retired hurt, and 51 with number 10 Ben Cutting (26).
The home spinners managed to both contain and pick up wickets, with Sean Williams recording excellent figures of two for 21 in his 10 overs, as Australia stuttered to their lowest ever score against Zimbabwe of 209 for nine in 50 overs.
Utseya finished with two for 45 in his 10 overs.
Zimbabwe took a patient approach to their reply but looked in trouble as they slipped to 106 for five before Chigumbura grafted to get his side home.
Nathan Lyon used the conditions well to record figures of four for 44 in 10 overs, with all of the regular bowlers economical in restricting the home batsman.
The fifth match of the series will be on Wednesday when Australia face South Africa at the same venue.
South Africa lead the table with nine points, with Australia second on five and Zimbabwe third on four points.