Real-World Scenario — Scheduler vs. Job vs. Workflow: Picking the Right Background Tool

Part of the Real-World AEM Problems series. Background reading: Sling Jobs vs Schedulers in AEM — Choosing the Right Background Processing Model covers the failure-semantics concepts behind what's discussed here. Also see Real-World Scenario — When a Workflow Is Actually the Right Tool for the human-in-the-loop side of this same decision.

Fifth stop in the building-blocks pass: schedulers and jobs. This one's less about "should I build this at all" and more about "which of three real options do I reach for" — the Scheduler, the Jobs API, or the workflow engine from the previous post. All three can technically run background work; the right choice depends entirely on what happens if it runs twice or gets interrupted halfway through.

Where It's Actually Used

A recurring cron-style sweep with no per-item state to protect. A nightly report generation task, a periodic cache-warming sweep, a scheduled health check — these are genuinely fine as a Sling Scheduler component, because if the sweep somehow ran twice (or was skipped once), the consequence is negligible: the report generates again, the cache warms again, nothing is lost or duplicated in a way that matters.

Per-item background work that needs durable asynchronous processing. Exporting a completed order, sending a confirmation email, or processing an uploaded file may need retry and recovery. Sling Jobs provide at-least-once processing, so the business operation must be safe to repeat; use idempotency or deduplication when duplicate side effects are unacceptable.

A high-volume queue that needs backpressure, not a fire-and-forget scheduler. A bulk CSV import creating thousands of individual product updates needs the Jobs API's queue semantics (concurrency control, retry, and cancellation for permanently invalid work) — trying to do this as one giant scheduled sweep means a single failure partway through leaves you with no clear record of which items succeeded.

Real Problems You'll Hit

A scheduled task running twice in a cluster because RUN_ON=SINGLE was left off. This is the single most common real bug in this space — a scheduler component that works perfectly in single-node local development runs on every node once deployed to a clustered author or publish tier, and nobody notices until duplicate emails or duplicate exports start showing up. It's an easy one-line fix once found, but it's rarely found until the duplication is already visible to the business.

A permanently-failing job retrying forever and quietly consuming queue capacity. A job executor that returns failed() (meaning "retry me") for an error that's actually permanent — a malformed record that will never succeed no matter how many times it's retried — keeps consuming a retry slot indefinitely instead of correctly returning cancelled(). The visible symptom is usually "the job queue seems backed up" long before anyone realizes it's one bad record retrying in a loop, not genuine load.

A sweep interval too aggressive for the work it triggers. A five-minute scheduled sweep querying for unprocessed records works fine until volume increases enough that a single sweep's query-and-process cycle takes longer than five minutes — at which point overlapping sweeps compound the "ran twice" problem on top of everything else. This tends to get discovered during exactly the traffic spike (a sale, a campaign launch) where it's most costly to be debugging.

When You Actually Need It vs When It's Overkill

Use the Scheduler when: the task is a cron-style recurring sweep and duplicate or occasionally-skipped execution genuinely doesn't matter — and always set RUN_ON=SINGLE explicitly rather than assuming.

Use the Jobs API when: you're processing discrete items that need durable asynchronous execution or retry after transient failures. Design side effects to tolerate reprocessing because job processing is at least once, not exactly once.

Use a workflow instead of either when: a human actually needs to see or act on the item's state — reach back to the previous post in this series for that distinction. Skip all three and just do it inline when: the work is genuinely fast and synchronous-safe (a few milliseconds) — wrapping trivial, fast work in a job queue adds latency and operational surface area for a problem (blocking the request) that doesn't actually exist yet.

A genuinely common overkill pattern: building a full Jobs API pipeline for something that's actually just a five-minute cron sweep with no per-item state to protect — the added retry/distribution machinery is solving a duplication problem that was never actually a risk for that particular task.

Concrete Example

The order-export fix from the referenced theory chapter — moving from a scheduler-based sweep (which double-exported orders during a traffic spike) to the Jobs API:

@Component(service = JobConsumer.class,
           property = JobConsumer.PROPERTY_TOPICS + "=com/example/jobs/order-export")
public class OrderExportJobConsumer implements JobConsumer {

    private static final Logger LOG = LoggerFactory.getLogger(OrderExportJobExecutor.class);

    @Reference
    private FulfillmentClient fulfillmentClient;

    @Override
    public JobResult process(Job job) {
        String orderId = (String) job.getProperty("orderId");

        try {
            fulfillmentClient.submitOrder(orderId);
            return JobResult.OK;
        } catch (FulfillmentTimeoutException e) {
            // transient — retry
            LOG.warn("Fulfillment API timed out for order {}, will retry", orderId, e);
            return JobResult.FAILED;
        } catch (InvalidOrderException e) {
            // permanent — don't retry, surface for manual handling
            LOG.error("Order {} is invalid, will not retry: {}", orderId, e.getMessage());
            return JobResult.CANCEL;
        }
    }
}

@Component(service = EventHandler.class,
           property = "event.topics=com/example/events/order-completed")
public class OrderCompletedEventListener implements EventHandler {

    @Reference
    private JobManager jobManager;

    @Override
    public void handleEvent(Event event) {
        String orderId = (String) event.getProperty("orderId");

        Map<String, Object> jobProperties = new HashMap<>();
        jobProperties.put("orderId", orderId);

        // One job per order; processing can be retried, so the operation must be idempotent.
        jobManager.addJob("com/example/jobs/order-export", jobProperties);
    }
}

Notice the explicit JobResult.OK / FAILED / CANCEL distinction — transient failures can be retried, while permanently invalid work should be cancelled rather than retried forever.

Summary

• Scheduler is right for cron-style sweeps where duplicate or occasionally-skipped runs genuinely don't matter — always set RUN_ON=SINGLE explicitly, never assume single-node
• Jobs API is right for durable per-item work that needs retries; because processing is at least once, use idempotency or deduplication for external side effects
• Distinguish transient failures (failed(), retry) from permanent ones (cancelled(), stop) in every job executor — this single distinction prevents the most common "queue looks backed up" false alarm
• A workflow is the right choice only when a human needs visible state — otherwise it's unnecessary overhead compared to a job
• Match the sweep interval to how long the work actually takes as volume grows — a fixed interval that was fine at low volume can start overlapping itself under real load

What's Next

Next in this pass: Sling Models — where they're genuinely the right abstraction for pulling content into a component versus when raw resource/valuemap access is simpler, and the real injection and adaptation gotchas that show up once a project has dozens of them.