2

As part of the Artisan Hub processing (global directory of Certified Salesforce Professionals) we need to run a daily process which assigns everyone a rank based on their total Artisan Hub points.

This has been working fine but the volume of Contacts (coming up to 120,000) started causing the batch apex code to hit governor limits about a week ago and the code now needs to be re-factored.

I'm wondering if anyone has a good strategy for processing this number of contact records in the right order?

Currently the code uses scheduled batch apex with a "next rank" value as a stateful class attribute. The main query used by the batch class sorted all the contacts into order.

What I've found is that whilst it appears watching the Apex Jobs page that the system is processing one batch at a time, they do not appear to be getting processed in the order imposed by the ORDER BY clause.

If batch Apex won't honor the sort order set in the batch query does anyone have a suggestion about how to batch process 120,000 contacts? Apart from exporting via the bulk API and updating outside of the Salesforce Platform!

Thanks

Richard

Richard Clarke
  • 105
  • 1
  • 6
  • 2
    Within each batch, the records are processed sequentially. One option to consider is implementing a "chained job" that allows the jobs to be run one after another by starting the second job as soon as the first job has finished (and so on). Set the batch size of the job to 1. If the single batch in the job can handle 200 records a time, 600 jobs will be executed sequentially. – Chun Dec 04 '20 at 06:23
  • 1
    Unfortunately (and I didn't realize this until researching just now), according to the docs, "Batches of records tend to execute in the order in which they’re received from the start method. However, the order in which batches of records execute depends on various factors. The order of execution isn’t guaranteed." – Brent Bowers Dec 04 '20 at 06:24
  • It might be possible to do something along these lines though to have one batch apex class initiate another batch run in the final() method passing enough parameters to force the query to select only those contacts not already processed. It's crazy to hack the batch apex system to work like this in a non-batch mechanism – Richard Clarke Dec 07 '20 at 05:17

1 Answers1

1

One option to consider is implementing a "Chained Job" that allows the jobs to be run one after another by starting the second job as soon as the first job has finished (and so on). The batch size should be set to 1 as within each batch the records are processed sequentially.

The Salesforce off-the-shelf solution for this "Chained Job" mechanism is Queueable Apex.

You can chain one job to another job by starting a second job from a running job.

Because no limit is enforced on the depth of chained jobs, you can chain one job to another.

Based on this strategy, the query needs to be tweaked to only select the next bunch of records that have not been processed. An SObject field can be added to maintain such a state so that the same query in the next job will exclude these processed ones and only pick up the next bunch. e.g.

public class ArtisanHubJob implements Queueable {
    public void execute(QueueableContext context) {
        // Set a limit; further tweak it if it is not selective
        Contact[] contacts = [select Id, Name from Contact where ArtisanHubProcessed__c = false order by CreatedDate limit 200];
        if (contacts.size() == 0) return;

    // ...

    // Update the flag so that the next query will move onto the next bunch
    for (Contact c : contacts) {
        c.ArtisanHubProcessed__c = true;
    }
    update contacts;

    // Chain this job to next job by submitting the next job
    System.enqueueJob(new ArtisanHubJob());
}



}

Suggest checking if the Queueable/Batchable penalties are still in place.

Chun
  • 378
  • 1
  • 8