Departure of foreign workers risks adding another drag to an economy in which per capita income has barely grown in a decade.