Bulger's lawyers want U.S. District Judge Richard Stearns to recuse himself because he was a federal prosecutor when Bulger claims he was given immunity for crimes he committed while leader of the notorious Winter Hill Gang and an FBI informant on the Mafia.
Stearns says there's no need for him to step down.
The 83-year-old Bulger claims former federal prosecutor Jeremiah O'Sullivan gave him immunity. O'Sullivan denied that before Congress in 2002.
The 1st U.S. Circuit Court of Appeals will hear arguments Jan. 8.
Bulger fled Boston in 1994 and remained a fugitive until last year, when he was captured in Santa Monica, Calif., with his girlfriend. He denies participating in 19 murders.