They've been ostracised by neighbours, frozen out of local social circles and accused of threatening other people's marriages - simply because of how they earn a living.