Whitworth and two Oakland linemen face significant fines for their ejection during the fourth quarter of the Cincinnati Bengals' 34-10 victory over the Raiders on Sunday.
"They were probably looking for a fight because they weren't doing much on the field," Whitworth said.
The Raiders (3-8) were frustrated after an inadvertent wiped out a fumble by Mohamed Sanu deep in Bengals territory with Cincinnati leading 27-10 midway through the fourth quarter. Tyvon Branch picked up the loose ball ran into the end zone, thinking he'd scored a touchdown.
The officials didn't make a call initially. After a long huddle, they announced that the play had been blown dead because of an inadvertent whistle, making it Cincinnati's ball.
"Unfortunately, referees make mistakes too," Raiders quarterback Carson Palmer said. "We make them as players, coaches make them, referees make them, too. It's unfortunate that happened to us there. Obviously that would have really changed the game. But that's part of football."
On the next play, right tackle Andre Smith was flagged for a false start. The play continued after the whistle, and defensive end Lamarr Houston tackled quarterback Andy Dalton.
Whitworth saw the tackle and ran over to Houston, their facemasks slamming together. Several other players then joined in the scrum.
"I was just standing there and got hit, and then everything took place," Dalton said.
Whitworth felt he had to defend his quarterback.
"When the whistle is blown four times and the guy does something like that to Andy, it's my job to step in there and I had to," Whitworth said.
The fight ended with Houston and defensive tackle Tommy Kelly on top of Whitworth on the ground. Whitworth and Houston were ejected for the fight, and Kelly for coming off the bench to join in.
"It gets a little rough out there sometimes," Houston said.
Bengals coach Marvin Lewis defended Whitworth for going after Houston.
"I think they—in my opinion—took a little bit of a cheap shot at the quarterback and Andrew came to his rescue, plain and simple," Lewis said.
Raiders coach Dennis Allen liked the way his team showed spunk after falling behind 24-0 in the first half. He didn't approve of the scrum.
"I think it's OK to play with anger, but we've got to play with poise and composure," Allen said. "We lost our poise a little bit."