Reducing idle time on job threads

[josemendez]

I was wondering if this code might lead to a situation where jobs are not scheduled when they could be?

Code:

public void ScheduleBatchedJobsIfNeeded()
{
    if (scheduledJobCounter++ > 16)
    {
        scheduledJobCounter = 0;
        JobHandle.ScheduleBatchedJobs();
    }
}

Scheduling jobs has an associated cost, and worker threads can't start working on a job that's not been scheduled. For this reason, scheduling jobs immediately after creating them is not a good idea because this will happen:

Code:

Main thread schedules job (workers wait)--->worker threads process the job---->Main thread schedules another job (workers wait)-----> worker threads process the job.....

The workers finish their job quickly and they have to wait for the main thread to schedule more stuff, wasting time they could spend working. This is specially true when you have many small jobs (one job per batch in Obi's case). Pushing jobs only when we have a significant amount of work to do (16 jobs in this case) results in this:

Code:

Main thread schedules 16 jobs(workers wait)--->Main thread schedules another 16 jobs(workers still working on the previous 16)--->Main thread schedules another 16 jobs(workers still working on the previous 16).... etc

This way both the main thread and the workers are always busy, and you avoid starving the thread pool.

In your profiler pic, you can see a big JobHandle.Complete() in the main thread that takes 13 ms. That's the main thread waiting for the workers to finish processing all the jobs that have been scheduled. This is a very good thing because it means that the workers do not have to wait for the main thread, but the other way around: all workers are busy, working on whatever job dependency chain they need to get done.