I listened to Gumbel's comments and he never accused Stern of being a racist. He accused him of treating NBA players as if they were his slaves. That would include all of the players, white, black, US, and foreign.
Slavery in and of itself has nothing to do with skin color. It has to do with how someone is treated and in the context of being "owned" and that was the view that Gumbel was trying to convey.
Stern is on a power trip and thinks the players are his property and that he can take away their rights whenever he wants to. In that context, Gumbel's commentary is correct.
I don't recall any plantation owners having white slaves.
I highly, highly doubt that was the message being conveyed. The NBA is predominantly black, and seems to me the message is/was that Stern is trying to exert his authority over black people.
I think his exact words were chosen very carefully, and he should have chosen better. He could have said "he's treating them like slaves" and it could be defended that your point is what he was saying.
Choosing to use "plantation owner" is deliberately conjuring an image of civil-war era South with a white owner of black slaves.
All jmho, of course.
Edit: All that said, the whole thing is ridiculous. Yeah, sign me up to be a slave, if that's what being a slave is.