How to scale the Quartz scheduler?

java scalability scheduling quartz-scheduler scaling
David Rabinowitz picture David Rabinowitz · May 31, 2009 · Viewed 7.7k times · Source

I plan to use the Quartz scheduler as I read many good opinions about it.

My problem is as follows: I will have thousands of triggers living in the system at any given time. Most of the triggers will fire just one event and die. In addition, it is very likely I will have to cancel many jobs after their allocation (based on new input).

Can Quartz scale to this? Which JobStore is recommended? I planned to use the JDBC one over mysql.

Added Information: My jobs will either send an e-mail or post data via HTTP post to other services (over WAN).

Answer

markusk picture markusk · May 31, 2009

There are some tips and recommendations in the FAQ:

How many jobs is Quartz capable of running?

An excerpt from the answer:

So let me just say, there are installments of Quartz out there that are managing hundreds-of-thousands of Jobs and Triggers, and that are, at any given moment in time executing several dozens of jobs – without even utilizing Quartz’s load-balancing capabilities.

There is also some discussion in the Quartz Users forum, with experience reports from developers who have used Quartz under similar conditions.

  • Scalability & Performance - Jim Nichols is running a couple of thousand jobs clustered, and volunteers more information by e-mail
  • Quartz performance - Stey Baptiste is running 2 million jobs a day, 10 jobs a second

Update 2015-01-12: Unfortunately, the links in the list above broke when the forum was moved, and I'm unable to find new working links. I'll leave the links anyway, in case they help digging out the info from archive.org or elsewhere. The official Quartz forum is now quartz-scheduler on Google Groups.

Related questions

open source alternatives to oracle coherence?

Are there any open source alternatives to oracle coherence? (btw, how much does coherence cost anyways?)

Read More
Why does this Java code not utilize all CPU cores?

The attached simple Java code should load all available cpu core when starting it with the right parameters. So for instance, you start it with java VMTest 8 int 0 and it will start 8 threads that do nothing else than looping and …

Read More
Java + NodeJS communicating via socket: Bad Idea?

I like some features of NodeJS, particularly JQuerification, websocket compatibility via socket.io, view and css engines that I cannot use with JSP (and of course, asynchronous calls). At least, as far as I know. So I plan on creating …

Read More