The Taliban claimed responsibility for the attack in the Afghan capital, and said it was meant as a message to the U.S. defense chief.
Pentagon spokesman George Little said the secretary was in a briefing at a U.S.-led military coalition facility in another part of the city when the explosion occurred. He said the briefing continued without interruption.
Afghan Defense Ministry spokesman Gen. Mohammad Zahir Azimi said the bomber on a bicycle struck just before 9 a.m. local time about 30 meters (yards) from the main gate of the ministry.
A man at the scene, Abdul Ghafoor, said the blast rocked the entire area.
"I saw death bodies and wounded victims lying everywhere," Ghafoor told AP Television News. "Then random shooting started and we escaped from the area."
The ministry said at least nine civilians were killed and others were wounded.
Reporters traveling with Hagel were in a briefing when they heard the explosion. They were moved to a lower floor of the same building as U.S. facilities in downtown Kabul were locked down as a security precaution.
Associated Press writer Lolita C. Baldor in Kabul contributed to this report.