Li Fuyan, 30, had been suffering from severe headaches, bad breath and breathing difficulties but never knew the cause of his discomfort, said the senior official at the Yuxi City People's Hospital in Yunnan Province.
Surgeons worked cautiously to remove the badly-corroded blade without shattering it, He said. The hospital's website also reported the successful surgery.
The case, which one of the doctors described as a "miracle," has been widely covered by the Chinese media and discussed on the Internet.
"We checked his mouth, but no wound or scar has been found. It is very strange as to how the blade got into his head," Xu Wen, deputy director of the hospital's stomatology department, told state broadcaster CCTV.
CCTV showed footage of the rusted knife and interviewed Li, who said: "As time passed, I used injections to kill the pain in my head and ears. It has been four years already."
Dr. Eugene Flamm, chairman of neurosurgery at New York's Montefiore Medical Center, said X-ray images of the man's head posted on the hospital's website show the knife sitting behind the man's throat, having missed the carotid artery and other key structures.
"There are planes and spaces between important organs. That's how one does surgery - you dissect in those planes, move the trachea one way, the esophagus the other," he said.
"Maybe out of sheer luck this knife passed through" one such area, Flamm said, adding that he was still surprised at the time the blade supposedly spent in the man's body.
AP Medical Writer Lauran Neergaard in Washington, D.C., and researcher Yu Bing in Beijing contributed to this report.