The famed civil rights photographer captured some of the most pivotal moments of the movement, from the proud rows of "I am a man" marchers, to the drama of the Emmett Till trial, to the surreal moments after the assassination of his friend the Rev. Martin Luther King Jr.
So when it was discovered this week that Withers, who died in 2007, had done double duty as an FBI informant, divulging the most intimate and potentially damaging secrets of his confidants to a government agency then dedicated to discrediting the movement, the shock was palpable.
For some, so too was the hurt.
"If these allegations are true, I am shocked and extremely disappointed," Dorothy Gilliam, a journalist who worked with Withers while covering the 1957 integration of Little Rock, Ark.'s Central High School, wrote in a statement. "I never had any reason to suspect that he was doing this when we worked together."
A two-year investigation of the photographer by The Commercial Appeal in Memphis, Tenn., shows that Withers was on the FBI's payroll for at least two years as confidential informant "ME 338-R." Documents obtained by the paper show that Withers, who had access to scores of civil rights leaders, fed the FBI information about everything from the movement's strategy, to details about King's funeral, to tips that helped break up the Invaders, a militant black group in Memphis.
Harold Middlebrook isn't sure why Withers did what he did. But Middlebrook, who was one of King's top aides and was in the Memphis hotel room when the civil rights leader was slain, says the revelations about his longtime friend are sad.
"In my mind, I guess he did what he felt he had to do. We did what we felt we had to do," Middlebrook told AOL News today in a phone interview. "It is tragic that we were not on the same path and that he permitted himself to be used by others."
Withers' family appears to have been blindsided. "We, as a family -- none of us have ever heard anything like that," Withers' daughter, Rosalind Withers, told WMCTV 5 in Memphis on Monday. "It's very sad that someone would have the audacity to print something of that magnitude without him here to defend himself."
But others say this is a story about treason. "It is an amazing betrayal," Athan Theoharis, a historian from Marquette University, told The New York Times. "It really speaks to the degree that the FBI was able to engage individuals within the civil rights movement. This man was so well trusted."
In fact, Withers was far from the only black civil rights figure to be recruited by the FBI, or by other state organizations working against rights for black Americans. The agencies found it very difficult to infiltrate the movement with white informants, so they often used black tipsters instead.
One of the most high-profile of those informants was Percy Greene, founder and editor of the Jackson Advocate, a black newspaper in Mississippi. In 1958, the Mississippi Sovereignty Commission, a state-run group that fought integration, asked Greene to do research on Medgar Evers, a civil rights leader who was slain outside his home in 1963.
Withers can no longer explain his actions, so his photographs will have to speak for themselves. And his old friend Middlebrook says the iconic images that made Withers famous still matter, regardless of whatever his friend may have been doing on the side.
"His photographs still opened a lot of eyes," Middlebrook said. "I really thought then, and still believe now, that Ernest had a serious commitment to the movement. He had seen the death of Emmett Till. He had been an African-American in the South."