After an uncertain Friday morning, BART trains began running again about 7:15 a.m. after an infrastructure upgrade to a network server halted service just after midnight.
At the peak of the morning commute -- about 8:30 a.m., service had been restored nearly 100 percent, said BART spokesman Jim Allison, with all but three trains running.
BART spokeswoman Alicia Trost said Thursday morning workers installed the system upgrade and 12 hours later experienced computer problems with the computer system in the Operations Control Center which controls train service.
"Eventually, the central computer went off line, which caused the last trains of the night to be delayed upward of two hours due to the need for train operators and maintenance crews to manually move and lock switches," Trost said. "BART worked through the night to identify the root of the computer problem and tracked it to the upgrade made the day before. The server was returned to the original configuration and the problem was fixed.
"BART engineers are investigating the technical nuances that caused the problems and will make any needed fixes to prevent the problem from reoccurring," she said. "At no time was train safety compromised. We apologize for the inconvenience to our late-night riders and for the late opening this morning."
Commuters arriving at BART stations early Friday morning were greeted with the frustrating news that trains would not be running until further notice due to a technical problem.
At one point, BART officials were advising commuters to arrange alternate transportation. By the time service resumed, however, both AC Transit and the San Francisco Bay Ferry Service had not yet added additional boat runs or buses. San Francisco International Airport had started a bus bridge from the Millbrae Caltrain station to the airport to help stranded commuters, a spokeswoman said.
Allison said the major delays began at 12:10 a.m. when trains were out doing their last runs. He described the problem as BART computer systems in central control not communicating properly with the track switches.
"Train operators and track crews are instead manually cranking switches into position to route the trains properly," Allison said. "Workers had to go out and switch the tracks manually, so it took some riders on some of the last trains about 90 minutes to get to their destination."
When those trains got to their destinations, he said, service ended.
Computer technicians spent the early morning hours working to restore normal operations, Allison said.
The message greeting commuters over loudspeakers at BART stations was: "Please use alternate means of transportation. We apologize for the delay."
Chris Plurad, 26, of Oakland rode his bike to the 19th Street BART station, hoping to get 6 a.m. train to Dublin, where he works at Specialty's Cafe and Bakery.
"I was expecting to catch my train," Plurad said. "I can't believe this ... dude, there is so much money invested in this. Look at these people, they can't get to work. I mean, god, it's Friday."
Holding a yellow hard hat in his hand, James Edgar, 42, descended into the same Oakland station about 6:30 a.m. to catch his daily train to Pleasanton for work.
"This is bad news. I've got to call my boss immediately," the IT consultant said as he scurried back up the station stairs above ground.
Check back for updates.