I'm aware of it, as a term best avoided, though whether it would cause offence would depend on context.
Obviously from 'black as the ace of...' But of course it's considered a virtue, especially in the north of England,
to call a spade a spade (and not a bloody shovel).
Growing up in the South in the 60s, I occasionally heard the term "spade" used for black people, mostly by folks from my grandparents' generation. I think it was already falling out of favor then. The Urban Dictionary says it was most common in the years after the Civil War. I think the last time I heard it used was by a black comedian.