Watching the episode, he was at least willing to note that the kitchen looked clean and well-organized and the cake tasted good (regardless of who made it). So I could believe if he had come into a situation where the food was actually good but the couple had no clue how to run a restaurant**, he would say that... probably with swearing and yelling.

But, yes, the basic show premise relies on there being something nightmarish about the kitchen. If it had been a case of 'haters'*, there probably wouldn't be the incentive to do the show.

* And I'm having trouble thinking of a situation short of Different People (gay folks, immigrants, racial minorities, etc.) move to Very Culturally-Uniform Town/Suburb and the locals are being unpleasant to them in hopes they'll move elsewhere, where this would actually be the reason a restaurant was failing.

** As opposed to the situation where the couple had no clue how to run a restaurant, the food wasn't all that good (and the good stuff was prepared elsewhere), and the couple seem to have missed Customer Service 101 and Employee Relations 100.