Laborers are the unskilled labor or "grunts" of Farthest Frontier. They are responsible for chopping wood and collecting stone. Any villager who is not assigned a Profession is by default a Laborer. The game will automatically draw from this pool of laborers and assign them as a Builder when needed.
It is recommended to have Laborers available so you can harvest raw resources. If the number of Laborers ever becomes orange, and not white, that means the game thinks your Laborer pool is too low in comparison to your total population.