"There's no room for Donald Sterling in the NBA — there is no room for him," the Heat forward said Saturday night before a playoff game against the Charlotte Bobcats.
The league said Saturday it was investigating the recording posted on TMZ's website, calling the comments "disturbing and offensive."
James said NBA Commissioner Adam Silver should take swift action if the comments are found to be made by Sterling.
"Obviously, if the reports are true it's unacceptable in our league," James said. "It doesn't matter, white, black or Hispanic — all across the races it's unacceptable. As the commissioner of our league they have to make a stand. They have to be very aggressive with it. I don't know what it will be, but we can't have that in our league.
"We're the model citizen of all sports around the world because we are the most recognizable figures. ... It's very disrespectful and it's appalling."
One of James' best friends, Chris Paul, plays for the Clippers.
James said he hasn't spoken with Paul, but "I can only imagine what is going through his head."
"I have kind of wavered back and forth if I would actually sit out," James said. "If our owner would come out and say the things he said I would have to sit down with my teammates and talk to my family because at the end of the day, our family and teammates are way more important than that,. Basketball is huge and the playoffs have been unbelievable so far and I hate that something has to come out."
In the recording, the man on the recorder questions his girlfriend's association with minorities.
James said he's not angry about the comments, just disappointed.
"I can only imagine if a player came out and said something of that stature what would happen to us as players," James said. "I believe in Adam (Silver) and I believe in the NBA and they have to do something very fast before this really gets out of hand. But like I said there is no room for Donald Sterling in our league."