It's my understanding that Ferry ranted for 5 minutes about how great of a basketball player and person Deng was prior to making this statement. Removing that from the audio and from the letter that he released is, by definition, taking the statement out of context.
In my opinion, the type of job has no bearing on what comments should or shouldn't be made. Ferry was ABSOLUTELY wrong in making those comments, and would have been wrong making those comments even if he was associated with NASCAR rather than the NBA, and in either case, punishment and sensitivity training should be required.
If he wanted to reference those comments from the scouting report, he should have heavily paraphrased, abstracting all racial context from the report. Listening to the audio, it sounds like he had not pre-read the report, and so was not prepared to redact the inappropriate statements.
I agree with the meaning of your last paragraph whole-heartedly. If Ferry is unable to put a quality product on the floor because his reputation is tarnished, then we terminate for a lack of job performance, not because of the comments themselves. It is entirely too early to make that assessment right now, which is why I feel Ferry should be retained.