What difficulties might there be in increasing diversity in the workplace?

What difficulties might there be in increasing diversity in the workplace?
There are difficulties in increasing diversity in many ways. For example with small populated companies it is very hard to get a representation of diversity in it especially if it is situated in a not so diverse place. Another way that it may be difficult to increase diversity in the workplace is in financial implications. It may be that an ethnic minority do not go for the job due to the money they would earn not being enough to feed their family. This tends to be a problem with Asian families who are normally said to have the biggest families.