Sling Jobs vs Schedulers in AEM — Choosing the Right Background Processing Model

A requirement such as this sounds simple:

Run a process every night and update product data.

The first implementation idea is often a scheduler.

There is a time involved, so schedule some Java code with a cron expression and execute the work when the trigger fires.

That solves only one part of the problem.

What happens if the AEM instance is replaced while the work is running?

What happens when the application runs on multiple instances?

What happens if the operation fails after processing half of the data?

Does the work need to be retried?

Can the same work safely run twice?

Those questions decide whether the requirement is really about time, reliable background work, or both.

That distinction becomes especially important in AEM as a Cloud Service, where application instances are replaceable and background code cannot assume that the same JVM will remain alive for the entire operation.

Two Different Responsibilities

A scheduler answers a timing question:

When should something be triggered?

A Sling Job answers a work-processing question:

How should asynchronous work be queued and processed?

Those are different responsibilities.

A scheduler can trigger code at 2:00 AM.

That does not automatically make the work persistent, retryable, or resilient to instance replacement.

A Sling Job can represent a unit of asynchronous work that needs reliable processing.

That does not mean every job has to originate from a cron schedule.

A job may be created because:

• A servlet accepted a request.
• A workflow reached a processing step.
• An event occurred.
• An integration received new data.
• A scheduled trigger determined that work was due.

Before choosing an API, I first separate the trigger from the work.

Why a Plain Scheduler Becomes Risky

Apache Sling Commons Scheduler can schedule a Runnable or job using a period or cron expression.

That is useful for timing.

But work started through the scheduler is not persisted as a durable Sling Job. Apache Sling's scheduler API documentation also notes that jobs started through that API are not persisted and are not restarted after a bundle restart.

That limitation matters much more in cloud environments.

Adobe's AEM as a Cloud Service development guidance says background tasks must assume that the instance running them can be brought down at any time. For work that needs reliable execution, Adobe recommends Sling Jobs rather than Sling Commons Scheduler.

So this design:

@Component(
        service = Runnable.class,
        property = {
            "scheduler.expression=0 0 2 * * ?"
        }
)
public class ProductSyncScheduler
        implements Runnable {

    @Override
    public void run() {
        productSyncService.syncAllProducts();
    }
}

may express the timing correctly.

But the entire product synchronization is now tied directly to that scheduled execution.

If the runtime disappears during the work, the scheduler itself does not give us a durable work item that can simply continue through Sling's job queue.

A Better Boundary: Trigger Work, Then Process Work

For work that must survive runtime interruptions, the scheduled part should not necessarily perform the entire business operation itself.

A cleaner boundary is:

1. Determine that work is due.
2. Create a durable work item.
3. Let the background processing mechanism execute it.

In Sling, that work item can be a Sling Job.

For example, the trigger can add a job:

Map<String, Object> properties =
        new HashMap<>();

properties.put(
        "syncType",
        "product"
);

jobManager.addJob(
        "myproject/product/sync",
        properties
);

A consumer processes that topic:

@Component(
        service = JobConsumer.class,
        property = {
            JobConsumer.PROPERTY_TOPICS
                    + "=myproject/product/sync"
        }
)
public class ProductSyncJobConsumer
        implements JobConsumer {

    @Reference
    private ProductSyncService
            productSyncService;

    @Override
    public JobResult process(
            Job job) {

        productSyncService.syncProducts();

        return JobResult.OK;
    }
}

The trigger and the processing responsibility are now separate.

The job represents work that needs to be handled.

The consumer owns execution of that work.

What Sling Jobs Give Us

Sling Jobs are built for asynchronous background processing.

Apache Sling's job handling is designed around persisted job information and queue-based processing. In AEM as a Cloud Service, Adobe specifically recommends Sling Jobs for background work that needs reliable execution because they provide an at-least-once execution model.

That changes how the implementation should be designed.

At-least-once does not mean:

This business operation will happen exactly once.

It means the system is designed so that an interrupted or failed job can be processed again.

The business operation therefore needs to tolerate retry.

At-Least-Once Changes the Business Logic

Suppose a job does this:

paymentService.chargeCustomer(
        customerId,
        amount
);

If the job completes the external charge but fails before the processing result is safely recorded, retrying the same job could charge the customer again.

The job framework cannot solve that business-level duplication by itself.

The operation needs an idempotency strategy.

For repository processing, that may mean recording progress or checking whether an item has already been processed.

For an external API, that may mean using an idempotency key supported by the external system.

For batch work, it may mean processing small resumable units rather than one enormous job.

The important rule is:

If a Sling Job can be retried, the business operation must be designed for retry.

Job Payloads Should Describe Work

A job payload should contain enough information for the consumer to identify the work.

For example:

Map<String, Object> properties =
        new HashMap<>();

properties.put(
        "productId",
        productId
);

jobManager.addJob(
        "myproject/product/index",
        properties
);

The consumer can then load the current product state and perform the operation.

I would avoid treating the job payload as a place to serialize a large application object graph.

A smaller payload keeps the job focused on work identity.

It also avoids coupling queued work too tightly to one in-memory representation of a Java object.

Job Topics Are Part of the Processing Contract

The topic:

myproject/product/index

is not just a random string.

It connects the producer and the consumer.

A useful topic describes the type of work being requested.

For example:

myproject/product/index
myproject/product/sync
myproject/asset/process

The topic should remain stable enough that producers and consumers can evolve without hidden string mismatches.

If the topic changes, both sides of the contract need to agree on the change.

A Consumer Should Delegate Business Logic

A JobConsumer should not become the entire implementation.

For example:

@Override
public JobResult process(
        Job job) {

    String productId =
            job.getProperty(
                    "productId",
                    String.class
            );

    productIndexService.index(
            productId
    );

    return JobResult.OK;
}

The consumer handles the job boundary.

ProductIndexService owns the actual indexing behavior.

That separation gives us a useful testing boundary and prevents queue-specific APIs from leaking through the business implementation.

It also allows the same service to be reused from another controlled entry point if the architecture later requires it.

Failure Is Part of the Job Contract

A background operation can fail because:

• An external API is temporarily unavailable.
• Repository access fails.
• Required content does not exist yet.
• A downstream service times out.
• The payload is invalid.
• The failure is permanent and retrying will not help.

Those cases should not all be treated the same way.

The consumer's result tells Sling whether processing succeeded or how failure should be handled.

The business implementation should distinguish temporary failures from invalid work where possible instead of blindly retrying every exception forever.

Queue configuration also matters because retry behavior and processing characteristics belong to the job-processing design, not just the Java consumer.

Long-Running Work Should Be Resumable

Adobe's Cloud Service guidance is explicit that background tasks must assume an instance can disappear.

That makes one huge job risky.

Suppose a nightly synchronization processes 100,000 products in one job.

If the job is interrupted after product 92,000, restarting everything from product 1 is wasteful and may repeat side effects.

A stronger design is to make progress recoverable.

Depending on the requirement, that can mean:

• Breaking work into smaller jobs.
• Recording a checkpoint.
• Processing pages/batches.
• Making each item independently idempotent.
• Reconstructing pending work from durable state.

The exact mechanism depends on the business operation.

The architectural requirement is that interruption should not force the system to restart an enormous operation from zero.

Where Schedulers Still Fit

This does not mean the concept of scheduling disappears.

Some requirements genuinely start with time:

• Check whether an integration needs synchronization.
• Trigger a cleanup window.
• Start a daily reconciliation.
• Periodically inspect whether work has become stale.

The mistake is assuming that the scheduled trigger must also perform all of the work.

For AEM as a Cloud Service, Adobe recommends the Sling Jobs scheduler when scheduled work needs reliable execution. Adobe's development guidance specifically warns against using Sling Commons Scheduler when execution must be guaranteed.

That gives us an important distinction:

Sling Commons Scheduler is a timing mechanism.

Persisted Sling Jobs are the stronger model when the work must survive runtime changes and support retry.

For cloud design, reliability requirements should drive the choice rather than familiarity with a cron annotation.

Scheduler Concurrency Is a Separate Problem

Even outside persistence, scheduled execution raises another question:

How many instances are allowed to run this operation?

In a clustered environment, code cannot assume there is only one JVM.

A scheduled task that writes the same repository data from several instances can create duplicate processing or write conflicts.

Sling Commons Scheduler exposes controls for concurrent execution and instance selection, but those controls do not turn a non-persisted scheduled task into durable background work.

Concurrency and durability are separate concerns.

A task can be configured not to overlap and still be lost when the runtime that owns the schedule disappears.

Cluster Behavior Must Be Designed Explicitly

Sling Jobs also run in a clustered environment.

Adobe documents that jobs are distributed across instances by default in AEM as a Cloud Service.

If a particular job must be processed only once across the Author service, queue design matters.

For example, Adobe documents an ordered Sling job queue pattern for processing on the leader Author instance.

The important point is not to copy one queue configuration into every use case.

It is to ask:

• Can several jobs run in parallel?
• Must work stay ordered?
• Can two jobs modify the same content?
• Does this work need single-instance processing?
• What happens during retry?
• Is the operation safe if another instance takes over?

Those are queue-design questions.

They should be decided from the workload, not after production starts showing duplicate processing.

Scheduler vs Sling Job — The Decision I Use

When I review a background-processing requirement, I separate four questions.

Is the requirement triggered by time?

If no, a scheduler probably does not belong in the design.

The work may be created directly as a Sling Job from the event or request that requires it.

Does the work need reliable execution?

If yes, do not rely on an in-memory/non-persisted scheduled callback as the work itself.

Use a durable background-processing model such as Sling Jobs.

Can the work be retried?

With Sling Jobs, assume that retry/re-execution is possible and make the operation idempotent or resumable.

Can several instances process related work safely?

If not, queue topology, ordering, parallelism, and workload partitioning need deliberate design.

These questions usually make the API choice much clearer than starting with:

Should I use a scheduler or a job?

Creating Jobs With JobManager

The earlier example used:

jobManager.addJob(
        "myproject/product/index",
        properties
);

That is enough for a straightforward job.

Sling also provides the JobBuilder API:

Job job = jobManager
        .createJob(
                "myproject/product/index"
        )
        .properties(properties)
        .add();

The important part is the final add() call.

Building the job definition does not queue the work until the job is actually added.

Whichever API the project uses, job creation should be treated as a boundary with its own failure handling. If no job is created, the caller should not behave as if asynchronous processing has already been accepted.

Keep the Producer Small

A producer should normally decide what work is required and enqueue enough information for the consumer to identify it.

For example:

public boolean queueProductIndex(
        String productId) {

    Map<String, Object> properties =
            new HashMap<>();

    properties.put(
            "productId",
            productId
    );

    Job job = jobManager.addJob(
            "myproject/product/index",
            properties
    );

    return job != null;
}

The producer does not perform indexing.

It does not open a repository session that the consumer is expected to reuse.

It does not pass an application service object through the payload.

It creates a durable description of work.

Validate the Job Again at the Consumer Boundary

A queued job may execute later than the code that created it.

Repository state may have changed in between.

The external system may have changed.

The content identified by the payload may no longer exist.

That means a consumer should not assume that because the producer validated something earlier, the same condition is still true when processing starts.

For example:

@Override
public JobResult process(
        Job job) {

    String productId =
            job.getProperty(
                    "productId",
                    String.class
            );

    if (productId == null
            || productId.trim().isEmpty()) {

        return JobResult.CANCEL;
    }

    return productIndexService.index(
            productId
    )
            ? JobResult.OK
            : JobResult.FAILED;
}

Here an invalid payload is treated differently from a processing failure that may be worth retrying.

That distinction becomes important once queue retries are configured.

OK, FAILED, and CANCEL Mean Different Things

A JobConsumer should translate the processing outcome deliberately.

return JobResult.OK;

means processing completed successfully.

return JobResult.FAILED;

means the work did not complete but may be rescheduled according to the queue's retry configuration.

return JobResult.CANCEL;

means the work failed and should not be rescheduled.

A temporary upstream outage may justify FAILED. A malformed payload that can never become valid may justify CANCEL.

Those semantics are important because the queue can only apply a useful retry policy when the consumer distinguishes retryable and permanent failures.

Exceptions Are Not a Retry Strategy

Expected temporary failures should not escape from process() as the application's normal retry mechanism.

Sling documents that an exception from a JobConsumer is treated like cancellation rather than a retryable FAILED result.

For example:

try {
    productSyncService.sync(productId);
    return JobResult.OK;

} catch (TemporaryIntegrationException e) {
    log.warn(
            "Temporary failure while syncing product {}",
            productId
    );
    return JobResult.FAILED;

} catch (InvalidProductException e) {
    log.error(
            "Product job cannot be processed: {}",
            productId
    );
    return JobResult.CANCEL;
}

Unexpected runtime exceptions should still be logged and investigated, but retryable business failures should be mapped intentionally.

Queue Configuration Controls Processing Behavior

A job topic is associated with a queue.

If no specific queue configuration matches the topic, Sling can use the default queue.

For production workloads, I prefer important job topics to have an intentional queue design rather than inheriting behavior accidentally.

Queue configuration can control properties such as:

• Queue name
• Queue type
• Matching topics
• Maximum parallel processing
• Retry count
• Retry delay
• Thread-pool behavior

The queue is where operational processing policy meets the job topic.

A Queue Configuration Example

For AEM as a Cloud Service, a custom queue configuration can be deployed from ui.config using the Sling job queue factory configuration.

For example:

ui.config/src/main/content/jcr_root/apps/myproject/osgiconfig/config.author/
org.apache.sling.event.jobs.QueueConfiguration~product-index.cfg.json

{
  "queue.name": "My Project - Product Index Queue",
  "queue.topics": [
    "myproject/product/index"
  ],
  "queue.type": "ORDERED",
  "queue.retries": 3,
  "queue.retrydelay": 30000,
  "queue.maxparallel": 1.0
}

The values must come from the workload. This example deliberately serializes the topic, retries failed processing up to the configured limit, and waits before retrying.

A queue that processes independent work may need an UNORDERED design and higher parallelism instead.

Queue Type Is a Business Decision

Sling supports different queue types.

The choice affects how jobs are processed.

An ordered queue is useful when processing order matters or when the workload should be serialized.

An unordered queue allows work to be processed without preserving strict ordering and can support parallel execution depending on configuration.

Sling also supports topic-oriented round-robin behavior.

I would not choose an ordered queue simply because it feels safer.

Serialization reduces concurrency.

If 10,000 independent jobs can safely run in parallel, forcing all of them through one ordered queue can turn reliability into a throughput problem.

The queue type should reflect dependencies between units of work.

Ordered Does Not Remove the Need for Idempotency

An ordered queue serializes work; it does not create exactly-once business semantics. A job can still be retried after partial processing.

Ordering solves concurrency and sequence. Idempotency makes repeated execution safe.

Retry Count and Retry Delay Need a Reason

A retry policy can help with temporary failures:

{
  "queue.retries": 3,
  "queue.retrydelay": 30000
}

But retry is additional load. If thousands of jobs fail because the same downstream API is unavailable, aggressive retries can amplify the outage.

Choose retry count and delay from the expected recovery time, operation cost, failure volume, and downstream capacity. Sling supports -1 for endless retries, but that should be used only when the failure model justifies it. Permanently invalid work should be cancelled or moved into an operational recovery path rather than retried indefinitely.

One Large Job vs Many Small Jobs

Suppose product synchronization contains 50,000 products.

One option is:

myproject/product/full-sync

with one job that processes all 50,000.

Another option is to use one coordination step that creates smaller work units:

myproject/product/sync-item

or batch-oriented work:

myproject/product/sync-batch

Smaller jobs can improve:

• Retry isolation
• Progress visibility
• Recovery after interruption
• Parallelism
• Failure diagnosis

But creating an extremely large number of tiny jobs also has overhead.

The right unit is large enough to be meaningful and small enough to recover.

That is a workload decision, not a framework rule.

Repository Writes Need a Fresh Execution Scope

A job consumer that writes repository content should obtain repository access for that execution.

For example:

@Override
public JobResult process(
        Job job) {

    String productId =
            job.getProperty(
                    "productId",
                    String.class
            );

    try (ResourceResolver resolver =
            resourceResolverFactory
                    .getServiceResourceResolver(
                            AUTH_INFO
                    )) {

        productRepositoryService
                .updateProduct(
                        resolver,
                        productId
                );

        resolver.commit();

        return JobResult.OK;

    } catch (LoginException
            | PersistenceException e) {

        log.error(
                "Unable to process product job {}",
                productId,
                e
        );

        return JobResult.FAILED;
    }
}

The resolver belongs to that job execution.

It should not be cached in the consumer and reused by later jobs or other threads.

The service-user and permission rules from Chapters 26 and 27 still apply here.

Background execution changes the caller.

It does not change repository security or resolver lifecycle.

Be Careful With Direct Writes on Publish

For AEM as a Cloud Service, content-creation or migration-style background work should not be designed around direct writes to arbitrary Publish pods.

Adobe's current guidance for concurrent scheduled processing recommends running repository-changing work on Author where appropriate and distributing content through the platform rather than trying to keep direct Publish writes synchronized.

This matters when a scheduler or job was originally designed on a single local instance.

A design that appears harmless locally can become inconsistent when several cloud instances are involved.

Scheduled Sling Jobs

There is an important middle ground between a plain Commons Scheduler callback and manually adding a job from another trigger.

Sling supports scheduled jobs through JobBuilder.ScheduleBuilder.

For example:

ScheduleBuilder schedule =
        jobManager
                .createJob(
                        "myproject/product/reconcile"
                )
                .schedule();

schedule.daily(
        2,
        0
);

schedule.add();

A scheduled Sling Job combines a time-based trigger with Sling job handling.

Apache Sling documents that scheduled jobs are persisted and, when their scheduled time arrives, are added as regular Sling Jobs through JobManager.

That is different from putting the entire operation inside a non-persisted Commons Scheduler Runnable.

Do Not Register the Same Schedule Repeatedly

A component activation method may execute more than once during the life of an application.

If activation blindly creates another periodic schedule every time, duplicate schedules can appear.

Before adding a periodic scheduled job, check whether the intended schedule already exists.

Conceptually:

Collection<ScheduledJobInfo> jobs =
        jobManager.getScheduledJobs(
                TOPIC,
                1,
                null
        );

if (jobs.isEmpty()) {

    jobManager
            .createJob(TOPIC)
            .schedule()
            .daily(2, 0)
            .add();
}

Periodic scheduled jobs also need lifecycle thinking.

If the application no longer requires the schedule, it should be unscheduled deliberately rather than assumed to disappear automatically.

Scheduled Does Not Mean Missed Runs Are Replayed Automatically

Persistence improves reliability, but scheduling still needs recovery semantics.

A topology change or pod replacement can happen around the scheduled execution window.

Current Adobe guidance notes that missed scheduled executions are not automatically replayed simply because the runtime later recovers.

For business-critical schedules, one useful design is to track the last successful run.

Then a startup or periodic check can determine whether the expected work is stale and enqueue a catch-up job.

For example, instead of trusting only:

Run at 2:00 AM.

the business rule can become:

Product reconciliation must have completed successfully within the expected daily window.

That is a much stronger operational contract.

Scheduled Time and Business Requirement Are Not Always the Same Thing

A cron expression describes a trigger time.

The business may actually care about freshness.

For example:

0 0 2 * * ?

means "trigger at 2:00 AM."

But the business requirement may be:

Product data must be reconciled once every day before business opens.

If 2:00 AM is missed because of runtime changes, blindly waiting until tomorrow violates the real requirement.

Designing from the business invariant makes catch-up behavior easier to reason about.

Running Work Once Across an Author Service

AEM as a Cloud Service runs Author as a cluster. Adobe documents an ORDERED Sling job queue pattern when a topic must be processed only once across the Author service: ordered jobs are handled by the leader for that queue's capable instances.

For example:

{
  "queue.name": "My Project - Reconciliation Queue",
  "queue.topics": [
    "myproject/product/reconcile"
  ],
  "queue.type": "ORDERED",
  "queue.retries": 1,
  "queue.maxparallel": 1.0
}

This is appropriate when the workload genuinely requires serialized leader processing. It should not become the default for independent work that can safely run in parallel.

Queue Configuration Does Not Fix Duplicate Job Creation

Even if a queue processes jobs one at a time, producers can still create duplicate logical work.

Suppose two triggers both enqueue:

productId = 12345

Those are two job records.

The queue may process them sequentially, but both can still execute.

If duplicate logical work matters, the application needs a deduplication or idempotency strategy.

Possible approaches depend on the use case:

• Check whether equivalent work is already pending.
• Use a durable business status.
• Use an idempotency key.
• Make repeated execution harmless.
• Collapse repeated events into a later reconciliation job.

Queue serialization is not business deduplication.

Production Troubleshooting

When background processing fails, I separate the queue symptom from the business failure.

Symptom What I check first

Job keeps retrying Is the failure transient, or is the payload/permission/configuration permanently wrong?

Queue keeps growing Did processing slow down, retry volume increase, or producers begin creating more work than consumers can handle?

Work runs more than expected Duplicate schedules, duplicate producers, job retry, queue topology, and business idempotency

Job appears stuck Ordered work ahead of it, retry delay, slow external call, oversized work unit, or missing dependency

Increasing queue.maxparallel is not a generic fix for a growing queue. It helps only when the workload is safe to parallelize. Repository conflicts or downstream rate limits can become worse with more concurrency.

A useful question for repeated failures is:

Can this attempt succeed later without changing the payload, permissions, configuration, or dependency state?

If not, more retries are unlikely to solve the problem.

Observability Should Use Business Identifiers

A log line such as:

Processing Sling job

is almost useless during an incident.

A better log connects framework work to business work:

log.info(
        "Starting product index job. jobId={}, productId={}",
        job.getId(),
        productId
);

On failure:

log.warn(
        "Product index job failed. jobId={}, productId={}, reason={}",
        job.getId(),
        productId,
        reason
);

Now an operator can correlate the queue entry with the product that was being processed.

Avoid logging secrets or unnecessarily large payloads.

Monitoring the Queue Matters

Asynchronous work has no user waiting on an HTTP response, so queue health needs its own visibility.

Useful signals include queued work, repeated retries, processing duration, oldest pending work, queue growth after deployment, and repeated failure for the same business identifier.

The exact monitoring integration is project-specific. The important part is that a failed background process should not remain invisible until somebody notices stale content.

A More Complete Product Synchronization Design

We can now revisit the nightly product synchronization.

The requirement is:

Keep AEM product data synchronized with the external source every day.

A stronger design is:

Scheduling boundary

A persisted Sling job schedule represents when reconciliation should be initiated.

Reconciliation job

The reconciliation job determines which product work is required.

It does not need to perform every product update in one giant transaction.

Work jobs

Product or batch jobs represent smaller recoverable units.

Consumer

The consumer validates the job payload and delegates to the business service.

Repository access

Each execution obtains its own service resolver with only the permissions required for that operation.

Retry

Transient integration failures return a retryable result according to queue policy.

Permanent validation failures are cancelled rather than retried indefinitely.

Idempotency

Processing the same product more than once does not create duplicate repository state.

Recovery

The application records enough durable progress to detect a missed or incomplete reconciliation.

That design is more involved than adding a cron expression.

It is also much closer to the actual reliability requirement.

Final Decision Guide

Use a scheduler only when the requirement is fundamentally a lightweight, best-effort time trigger and losing an execution is acceptable.

Use Sling Jobs when work must be persisted, processed asynchronously, retried after supported failures, or recovered after an instance disappears.

When reliable work also starts on a schedule, use Sling's scheduled-job model and still design the work for at-least-once execution.

After that choice, the real architecture work is in idempotency, work-unit size, retry semantics, queue concurrency, repository identity, recovery, and observability.

Summary

Schedulers and Sling Jobs solve different responsibilities: a scheduler answers when, while a Sling Job represents work that needs processing.

For AEM as a Cloud Service, Adobe recommends Sling Jobs for background work that needs reliable execution because instances can disappear and Sling Jobs provide at-least-once processing. Scheduled Sling Jobs combine a time-based requirement with the same durable job model.

That reliability still depends on application design. Retried work must be safe, large operations should be resumable, queue concurrency must match the workload, and repository access must be scoped to each execution.

A cron expression can start the conversation. It should not be the architecture.

What's Next

Chapter 30 — AEM Event Handling

The next chapter moves from time-driven and queued background processing to event-driven behavior: how AEM and Sling events are observed, where event handling fits, and how to keep event listeners from becoming uncontrolled background-processing engines.