The Butte Fire has burned 71,523 acres in San Andreas and is 35 percent contained, as of Monday night, according to Cal Fire officials.
The flames have destroyed 81 homes and 51 buildings since Saturday. At least 6,400 structures are threatened, officials said.
Cal Fire officials said several evacuation orders have been lifted in Amador County, but they are asking residents to be cautious of their surrounding environment.
"It's expanding like a balloon,'' said state fire spokeswoman Nancy Longmore. "It's moving very fast. There's many homes threatened.''
On Sunday, PG&E crews in Sutter Creek worked to restore electricity back to their customers.
PG&E officials said in a statement that approximately 6,400 customers did not have electricity Saturday morning. However, many residents in the Pine Grove area received power from temporary generators, officials said.
On Friday, the entire town of San Andreas was ordered to evacuate as the wildfire jumped from a few hundred acres Thursday to 50,000. Officials quickly lifted mandatory evacuations after 5:20 p.m.
More than 2,500 firefighters remain on the front lines. The wildfire started on Wednesday and the cause is under investigation.
The Associated Press contributed to this story.