Worst of all: Explaining everything
I have had a lot of clients and experiences in my 23 years as a personal trainer. The pay depends on the clients and what you offer, if you are qualified and experienced it can be good. The hours depend on how much you need to work to get a good pay.
What doesn't depend on any of this is having a client that has no idea of what you are talking about. This can be quite difficult to imagine but its really exasperating, because you cannot leave a customer if he pays and you have no way to get rid of him. In addition, every single day you have to explain your exercises to your client, which takes extra time and decreases the effectivity of your work because your customer spends more time listening than exercising. All in all, you must find some way to politely cut your relation with the customers you "dislike".