Instead, they argue the judge's decade-long relationship with another man poses a potential conflict because they might want to get hitched themselves.
They pointed out that while courts have not yet had to wrestle with sexual orientation as grounds for judicial recusal, judges typically have rejected efforts to remove jurists based on personal characteristics such as race, gender, religion or even the contents of their investment portfolios.
"I don't think this judge had any more duty to disclose his sexual orientation than a Christian or Jewish or Muslim judge has a duty to discuss their religion or a heterosexual judge has his duty to discuss their sexual orientation," retired Illinois state Judge Raymond McKoski said.
At the center of the dispute is Chief U.S. District Judge Vaughn Walker, who issued the ruling last August declaring Proposition 8 to be an unconstitutional violation of gay Californians' civil rights.
"We are not suggesting that a gay or lesbian judge could not sit on this case," attorneys for the backers of Proposition 8 wrote in their motion filed Monday to overturn the landmark ruling. "Simply stated, under governing California law, Chief Judge Walker currently cannot marry his partner, but his decision in this case ... would give him a right to do so."
They claim Walker should have disclosed the relationship while presiding over the case and said if he had any interest in marrying his partner.
DePaul University College of Law professor Jeffrey Shaman, co-author of a widely used textbook on judicial conduct, said the fact that Walker was rumored to be gay from the moment he randomly drew the Proposition 8 case "somewhat undercuts the argument that he should have disclosed he was in a long-term relationship."
Lawyers for backers of the ban seem to be grasping at straws in making their argument against the now-retired Walker, Shaman said.
"But it's their prerogative to do this as lawyers," Shaman said. "It might indicate they are worried about the judge's opinion, which was such a strong opinion, and they are trying to make an end run around it."
The Gay and Lesbian Victory Fund, a political action committee and recruitment organization for gay politicians, said there are now 102 openly gay, lesbian, bisexual and transgender judges in the U.S.
Only one, U.S. District Judge Deborah Batts in New York, serves at the federal level, although President Barack Obama has nominated two gay men for federal judgeships but they have not been confirmed.
Rumors that Walker was gay and had a long-term partner who accompanied him to social functions circulated during the 13-day trial that preceded his decision and after he handed it down. The judge declined to comment at the time.
Members of the Proposition 8 team openly complained about Walker's handling of the case and accused him of favoring the same-sex couples who had sued in his court for the right to marry. But they refrained from raising the specter of the judge's sexual orientation, saying media reports and gossip were an unsound basis for legal strategy.
"The bottom line is this case, from our perspective, is and always will be about the law and not about the judge who decides it," Jim Campbell, a lawyer with the Christian legal defense group Alliance Defense Fund, told The Associated Press in August.
That might have remained their position if Walker, who retired in late February after two decades on the federal bench, had not decided to end the speculation himself.
Earlier this month, Walker had a farewell meeting with a select group of courthouse reporters. When the topic came up, Walker said he never thought about recusing himself because he was gay and noted that no one had asked him to, according to the San Francisco Chronicle, which had a reporter at the gathering.
The judge also revealed that he'd been in a relationship with a man he identified only as a physician for a decade.
"If you thought a judge's sexuality, ethnicity, national origin (or) gender would prevent the judge from handling a case, that's a very slippery slope," Walker said. "I don't think it's relevant."
The lawyers who filed the motion to wipe out the judge's ruling declined to elaborate outside their written arguments about why they concluded that Walker's comment about his partner caused them to change course.
Retired California state Judge Jeffrey Rothman said bias claims have arisen in the past surrounding judges with strong religious views. But he noted that the bar for disqualification is purposefully set high. Lawyers representing a clinic that performed abortions, for example, would not be able to challenge a devoutly Catholic judge, he said.
"They would get absolutely nowhere with such a challenge unless that judge had gone out and made statements or speeches saying he believed that Roe v. Wade ought to be overturned if that case ever came before them," Rothman said. "The question is, can the beliefs be set aside and the judge decide the case on its merits and be fair."