[sgen] Don't use workers during finishing pause if we're not parallel