I think it goes along the lines that watermelon was considered to be a really low grade food and so was fed to slaves. Leading to the "idea" that black people liked watermelon which is why it still has unfortunate implications today.

I am Australian though so I might have the wrong idea. ^^o