How to add new schedule job dynamically with Spring

Question

I am writing a Spring Boot App

My requirements are - In the resources (src/main/resources) folder if I add new xml files... I should read those files and get some url and other specific settings from each of them. and for those urls I need to download data everyday... So a new scheduler job will start with url and some settings

The new jobs will run in different schedule time which will use cron expression present in the xml files. Also, files will be added dynamically at any point of time.

How to implement it?

Answer 1

If you want to dynamically schedule tasks you can do it without spring by using ExecutorService in particular ScheduledThreadPoolExecutor

Runnable task  = () -> doSomething();
ScheduledExecutorService executor = Executors.newScheduledThreadPool(Runtime.getRuntime().availableProcessors());
// Schedule a task that will be executed in 120 sec
executor.schedule(task, 120, TimeUnit.SECONDS);

// Schedule a task that will be first run in 120 sec and each 120sec
// If an exception occurs then it's task executions are canceled.
executor.scheduleAtFixedRate(task, 120, 120, TimeUnit.SECONDS);

// Schedule a task that will be first run in 120 sec and each 120sec after the last execution
// If an exception occurs then it's task executions are canceled.
executor.scheduleWithFixedDelay(task, 120, 120, TimeUnit.SECONDS);

With spring you can rely on the Task and Scheduling API

public class MyBean {

    private final TaskScheduler executor;

    @Autowired
    public MyBean(TaskScheduler taskExecutor) {
        this.executor = taskExecutor;
    }

    public void scheduling(final Runnable task) {
        // Schedule a task to run once at the given date (here in 1minute)
        executor.schedule(task, Date.from(LocalDateTime.now().plusMinutes(1)
            .atZone(ZoneId.systemDefault()).toInstant()));

        // Schedule a task that will run as soon as possible and every 1000ms
        executor.scheduleAtFixedRate(task, 1000);

        // Schedule a task that will first run at the given date and every 1000ms
        executor.scheduleAtFixedRate(task, Date.from(LocalDateTime.now().plusMinutes(1)
            .atZone(ZoneId.systemDefault()).toInstant()), 1000);

        // Schedule a task that will run as soon as possible and every 1000ms after the previous completion
        executor.scheduleWithFixedDelay(task, 1000);

        // Schedule a task that will run as soon as possible and every 1000ms after the previous completion
        executor.scheduleWithFixedDelay(task, Date.from(LocalDateTime.now().plusMinutes(1)
            .atZone(ZoneId.systemDefault()).toInstant()), 1000);

        // Schedule a task with the given cron expression
        executor.schedule(task, new CronTrigger("*/5 * * * * MON-FRI"));
    }
}

And you can provide your own trigger by implementing Trigger

Don't forget to enable the scheduling by usin @EnableScheduling on configuration class.

About listening to directory content you can use WatchService. Something like:

final Path myDir = Paths.get("my/directory/i/want/to/monitor");
final WatchService watchService = FileSystems.getDefault().newWatchService();
// listen to create event in the directory
myDir.register(watchService, StandardWatchEventKinds.ENTRY_CREATE);
// Infinite loop don't forget to run this in a Thread
for(;;) {
   final WatchKey key = watchService.take();
   for (WatchEvent<?> event : key.pollEvents()) {
       WatchEvent<Path> watchEvent = (WatchEvent<Path>) event;
       Path newFilePath = myDir.resolve(watchEvent.context());
       //do something with the newFilePath
    }
    // To keep receiving event
    key.reset();
}

Take a look at this article: Watching a Directory for Changes for more details.

Answer 2

Try this library with external dynamic parameters configuration, real time monitoring:

https://github.com/tyrion9/mtask

Config params in mtasks.yml

-   code: complex
    scheduled:
        period: 1000
    name: Autowired Param MTask
    className: sample.sample2.ComplexMTask
    params:
        name: HoaiPN
    autoStart: true

Dynamic params configurations on the fly:

curl -X GET http://localhost:8080/api

curl -X POST http://localhost:8080/api/helloworld/stop

curl -X POST http://localhost:8080/api/helloworld/start

Answer 3

You can do that over a spring annotation:

@Scheduled(fixedRate = 360000)
public void parseXmlFile() {
    // logic for parsing the XML file.
}

Please note that the method must be void. Furthermore, in your main class, you must enable scheduling:

@SpringBootApplication
@EnableScheduling
public class Application {

    public static void main(String[] args) throws Exception {
        SpringApplication.run(Application.class);
    }
}

Please see the full reference here: https://spring.io/guides/gs/scheduling-tasks/