Spring集成Quartz分布式定时任务框架

Quartz集群及理论介绍,内存释放进程过多问题

Posted by zhida on May 22, 2017

Quartz相对于其他方案解决了什么问题

  • 无法做灾难恢复
  • 集群环境的定时任务单一执行
  • 缺少对任务的管理

理论原理

Quartz集群中的每个节点是一个独立的Quartz应用,它又管理着其他的节点。该集群需要分别对每个节点分别启动或停止,不像应用服务器的集群,独立的Quartz节点并不与另一个节点或是管理节点通信。Quartz应用是通过数据库表来感知到另一应用。只有使用持久的JobStore才能完成Quqrtz集群。

Quartz集群数据库表

Quartz的集群部署方案在架构上是分布式的,没有负责集中管理的节点,而是利用数据库锁的方式来实现集群环境下进行并发控制。BTW,分布式部署时需要保证各个节点的系统时间一致

Quartz线程模型

在Quartz中有两类线程:Scheduler调度线程和任务执行线程。任务执行线程:Quartz不会在主线程(QuartzSchedulerThread)中处理用户的Job。Quartz把线程管理的职责委托给ThreadPool,一般的设置使用SimpleThreadPool。SimpleThreadPool创建了一定数量的WorkerThread实例来使得Job能够在线程中进行处理。WorkerThread是定义在SimpleThreadPool类中的内部类,它实质上就是一个线程。例如,CRM中配置如下:

集成步骤

导入sql文件

/*
 Navicat Premium Data Transfer

 Source Server         : docker-本地
 Source Server Type    : MySQL
 Source Server Version : 50635
 Source Host           : 127.0.0.1
 Source Database       : yea

 Target Server Type    : MySQL
 Target Server Version : 50635
 File Encoding         : utf-8

 Date: 08/11/2017 14:35:15 PM
*/

SET FOREIGN_KEY_CHECKS = 0;

-- ----------------------------
--  Table structure for `DATABASECHANGELOG`
-- ----------------------------
DROP TABLE IF EXISTS `DATABASECHANGELOG`;
CREATE TABLE `DATABASECHANGELOG` (
  `ID` varchar(255) COLLATE utf8_bin NOT NULL,
  `AUTHOR` varchar(255) COLLATE utf8_bin NOT NULL,
  `FILENAME` varchar(255) COLLATE utf8_bin NOT NULL,
  `DATEEXECUTED` datetime NOT NULL,
  `ORDEREXECUTED` int(11) NOT NULL,
  `EXECTYPE` varchar(10) COLLATE utf8_bin NOT NULL,
  `MD5SUM` varchar(35) COLLATE utf8_bin DEFAULT NULL,
  `DESCRIPTION` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `COMMENTS` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `TAG` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  `LIQUIBASE` varchar(20) COLLATE utf8_bin DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `DATABASECHANGELOGLOCK`
-- ----------------------------
DROP TABLE IF EXISTS `DATABASECHANGELOGLOCK`;
CREATE TABLE `DATABASECHANGELOGLOCK` (
  `ID` int(11) NOT NULL,
  `LOCKED` bit(1) NOT NULL,
  `LOCKGRANTED` datetime DEFAULT NULL,
  `LOCKEDBY` varchar(255) COLLATE utf8_bin DEFAULT NULL,
  PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_BLOB_TRIGGERS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_BLOB_TRIGGERS`;
CREATE TABLE `QRTZ_BLOB_TRIGGERS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `TRIGGER_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `BLOB_DATA` varchar(250) COLLATE utf8_bin DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_CALENDARS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_CALENDARS`;
CREATE TABLE `QRTZ_CALENDARS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `CALENDAR_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `CALENDAR` varchar(200) COLLATE utf8_bin DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_CRON_TRIGGERS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_CRON_TRIGGERS`;
CREATE TABLE `QRTZ_CRON_TRIGGERS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `TRIGGER_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `CRON_EXPRESSION` varchar(120) COLLATE utf8_bin NOT NULL,
  `TIME_ZONE_ID` varchar(80) COLLATE utf8_bin DEFAULT NULL,
  PRIMARY KEY (`SCHED_NAME`,`TRIGGER_NAME`,`TRIGGER_GROUP`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_FIRED_TRIGGERS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_FIRED_TRIGGERS`;
CREATE TABLE `QRTZ_FIRED_TRIGGERS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `ENTRY_ID` varchar(95) COLLATE utf8_bin NOT NULL,
  `TRIGGER_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `INSTANCE_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `FIRED_TIME` bigint(20) NOT NULL,
  `SCHED_TIME` bigint(20) NOT NULL,
  `PRIORITY` int(11) NOT NULL,
  `STATE` varchar(16) COLLATE utf8_bin NOT NULL,
  `JOB_NAME` varchar(200) COLLATE utf8_bin DEFAULT NULL,
  `JOB_GROUP` varchar(200) COLLATE utf8_bin DEFAULT NULL,
  `IS_NONCONCURRENT` tinyint(1) DEFAULT NULL,
  `REQUESTS_RECOVERY` tinyint(1) DEFAULT NULL,
  PRIMARY KEY (`SCHED_NAME`,`ENTRY_ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_JOB_DETAILS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_JOB_DETAILS`;
CREATE TABLE `QRTZ_JOB_DETAILS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `JOB_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `JOB_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `DESCRIPTION` varchar(250) COLLATE utf8_bin DEFAULT NULL,
  `JOB_CLASS_NAME` varchar(250) COLLATE utf8_bin NOT NULL,
  `IS_DURABLE` tinyint(1) NOT NULL,
  `IS_NONCONCURRENT` tinyint(1) NOT NULL,
  `IS_UPDATE_DATA` tinyint(1) NOT NULL,
  `REQUESTS_RECOVERY` tinyint(1) NOT NULL,
  `JOB_DATA` varchar(250) COLLATE utf8_bin DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_LOCKS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_LOCKS`;
CREATE TABLE `QRTZ_LOCKS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `LOCK_NAME` varchar(40) COLLATE utf8_bin NOT NULL,
  PRIMARY KEY (`SCHED_NAME`,`LOCK_NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_PAUSED_TRIGGER_GRPS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_PAUSED_TRIGGER_GRPS`;
CREATE TABLE `QRTZ_PAUSED_TRIGGER_GRPS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  PRIMARY KEY (`SCHED_NAME`,`TRIGGER_GROUP`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_SCHEDULER_STATE`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_SCHEDULER_STATE`;
CREATE TABLE `QRTZ_SCHEDULER_STATE` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `INSTANCE_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `LAST_CHECKIN_TIME` bigint(20) NOT NULL,
  `CHECKIN_INTERVAL` bigint(20) NOT NULL,
  PRIMARY KEY (`SCHED_NAME`,`INSTANCE_NAME`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_SIMPLE_TRIGGERS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_SIMPLE_TRIGGERS`;
CREATE TABLE `QRTZ_SIMPLE_TRIGGERS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `TRIGGER_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `REPEAT_COUNT` bigint(20) NOT NULL,
  `REPEAT_INTERVAL` bigint(20) NOT NULL,
  `TIMES_TRIGGERED` bigint(20) NOT NULL,
  PRIMARY KEY (`SCHED_NAME`,`TRIGGER_NAME`,`TRIGGER_GROUP`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_SIMPROP_TRIGGERS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_SIMPROP_TRIGGERS`;
CREATE TABLE `QRTZ_SIMPROP_TRIGGERS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `TRIGGER_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `STR_PROP_1` varchar(512) COLLATE utf8_bin DEFAULT NULL,
  `STR_PROP_2` varchar(512) COLLATE utf8_bin DEFAULT NULL,
  `STR_PROP_3` varchar(512) COLLATE utf8_bin DEFAULT NULL,
  `INT_PROP_1` int(11) DEFAULT NULL,
  `INT_PROP_2` int(11) DEFAULT NULL,
  `LONG_PROP_1` bigint(20) DEFAULT NULL,
  `LONG_PROP_2` bigint(20) DEFAULT NULL,
  `DEC_PROP_1` decimal(13,4) DEFAULT NULL,
  `DEC_PROP_2` decimal(13,4) DEFAULT NULL,
  `BOOL_PROP_1` tinyint(1) DEFAULT NULL,
  `BOOL_PROP_2` tinyint(1) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;

-- ----------------------------
--  Table structure for `QRTZ_TRIGGERS`
-- ----------------------------
DROP TABLE IF EXISTS `QRTZ_TRIGGERS`;
CREATE TABLE `QRTZ_TRIGGERS` (
  `SCHED_NAME` varchar(120) COLLATE utf8_bin NOT NULL,
  `TRIGGER_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `TRIGGER_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `JOB_NAME` varchar(200) COLLATE utf8_bin NOT NULL,
  `JOB_GROUP` varchar(200) COLLATE utf8_bin NOT NULL,
  `DESCRIPTION` varchar(250) COLLATE utf8_bin DEFAULT NULL,
  `NEXT_FIRE_TIME` bigint(20) DEFAULT NULL,
  `PREV_FIRE_TIME` bigint(20) DEFAULT NULL,
  `PRIORITY` int(11) DEFAULT NULL,
  `TRIGGER_STATE` varchar(16) COLLATE utf8_bin NOT NULL,
  `TRIGGER_TYPE` varchar(8) COLLATE utf8_bin NOT NULL,
  `START_TIME` bigint(20) NOT NULL,
  `END_TIME` bigint(20) DEFAULT NULL,
  `CALENDAR_NAME` varchar(200) COLLATE utf8_bin DEFAULT NULL,
  `MISFIRE_INSTR` smallint(6) DEFAULT NULL,
  `JOB_DATA` varchar(250) COLLATE utf8_bin DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin;


-- ----------------------------
-- 表概述
-- QRTZ_BLOB_TRIGGERS
-- QRTZ_CALENDARS
-- QRTZ_CRON_TRIGGERS:  记录定时类型的触发器,
-- QRTZ_FIRED_TRIGGERS:  手动类型的触发器
-- QRTZ_JOB_DETAILS: 任务的详情
-- QRTZ_LOCKS: 锁的状态 
-- QRTZ_PAUSED_TRIGGER_GRPS
-- QRTZ_SCHEDULER_STATE: scheduler状态,主机信息
-- QRTZ_SIMPLE_TRIGGERS
-- QRTZ_SIMPROP_TRIGGERS
-- QRTZ_TRIGGERS:触发器实时状态信息

-- ----------------------------

pom.xml 依赖包


<dependency>
	<groupId>com.google.code.gson</groupId>
	<artifactId>gson</artifactId>
</dependency>
<dependency>
	<groupId>org.quartz-scheduler</groupId>
	<artifactId>quartz</artifactId>
	<version>2.2.1</version>
</dependency>
<dependency>
	<groupId>mysql</groupId>
	<artifactId>mysql-connector-java</artifactId>
</dependency>
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-jdbc</artifactId>
  </dependency>

配置application.properties

quartz.enabled = true
spring.datasource.autoCommit=true
spring.datasource.defaultAutoCommit=true
spring.datasource.driver-class-name=com.mysql.jdbc.Driver
spring.datasource.password=mysqlpwd
spring.datasource.username=root
spring.datasource.url=jdbc\:mysql\://127.0.0.1\:3306/yea?characterEncoding\=UTF-8

配置数据库 以及 任务信息 crond.properties

spring.datasource.username=root
cron.frequency.SampleJob=0/1 * * * * ?
cron.frequency.jobwithcrontrigger=0/1 * * * * ?

配置 quartz.properties 信息

org.quartz.scheduler.instanceName=spring-boot-quartz-demo
org.quartz.scheduler.instanceId=AUTO
org.quartz.threadPool.threadCount=5
org.quartz.jobStore.class=org.quartz.impl.jdbcjobstore.JobStoreTX
org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.StdJDBCDelegate
org.quartz.jobStore.useProperties=true
org.quartz.jobStore.misfireThreshold=60000			//how late the trigger should be to be considered misfired
org.quartz.jobStore.tablePrefix=QRTZ_
org.quartz.jobStore.isClustered=true
org.quartz.jobStore.clusterCheckinInterval=20000

引入AutowiringSpringBeanJobFactory.java , 自动配置quartz

public final class AutowiringSpringBeanJobFactory extends SpringBeanJobFactory implements
        ApplicationContextAware {

    private transient AutowireCapableBeanFactory beanFactory;

    @Override
    public void setApplicationContext(final ApplicationContext context) {
        beanFactory = context.getAutowireCapableBeanFactory();
    }

    @Override
    protected Object createJobInstance(final TriggerFiredBundle bundle) throws Exception {
        final Object job = super.createJobInstance(bundle);
        beanFactory.autowireBean(job);
        return job;
    }
}

SchedulerConfig.java 能够生成触发器

注意 factory.setSchedulerName(“schedulerName-项目名称”);

@Configuration
@ConditionalOnProperty(name = "quartz.enabled")
public class SchedulerConfig {

	@Autowired
	List<Trigger> listOfTrigger;

	@Bean
	public JobFactory jobFactory(ApplicationContext applicationContext ) {
		AutowiringSpringBeanJobFactory jobFactory = new AutowiringSpringBeanJobFactory();
		jobFactory.setApplicationContext(applicationContext);
		return jobFactory;
	}

	@Bean
	public SchedulerFactoryBean schedulerFactoryBean(DataSource dataSource, JobFactory jobFactory ) throws IOException, SchedulerException {
		
		SchedulerFactoryBean factory = new SchedulerFactoryBean();
		factory.setOverwriteExistingJobs(true);
		factory.setDataSource(dataSource);
		factory.setJobFactory(jobFactory);
		factory.setAutoStartup(true);
		factory.setQuartzProperties(quartzProperties());

		if (!AppUtil.isObjectEmpty(listOfTrigger)) {
			factory.setTriggers(listOfTrigger.toArray(new Trigger[listOfTrigger.size()]));
		}

		return factory;
	}

	@Bean
	public Properties quartzProperties() throws IOException {
		PropertiesFactoryBean propertiesFactoryBean = new PropertiesFactoryBean();
		propertiesFactoryBean.setLocation(new ClassPathResource("/quartz.properties"));
		propertiesFactoryBean.afterPropertiesSet();
		return propertiesFactoryBean.getObject();
	}


	public static JobDetailFactoryBean createJobDetail(Class jobClass,String description) {

		JobDetailFactoryBean factoryBean = new JobDetailFactoryBean();
		factoryBean.setJobClass(jobClass);
		factoryBean.setDurability(true);
		factoryBean.setDescription(description);
		
		return factoryBean;
	}

	public static CronTriggerFactoryBean createCronTrigger(JobDetail jobDetail, String frequency,String description) {
		CronTriggerFactoryBean factoryBean = new CronTriggerFactoryBean();
		factoryBean.setJobDetail(jobDetail);
		factoryBean.setCronExpression(frequency);
		factoryBean.setDescription(description);
		factoryBean.setMisfireInstruction(SimpleTrigger.MISFIRE_INSTRUCTION_FIRE_NOW);
		return factoryBean;
	}

}

定义触发器

@Component
@DisallowConcurrentExecution
public class SampleTrigger implements Job {

	@Value("${cron.frequency.SampleJob}")
	private String frequency;

	@Bean(name = "SampleJob")
	public JobDetailFactoryBean sampleJob() {
		return SchedulerConfig.createJobDetail(this.getClass(), "");
	}

	@Bean(name = "SampleTrigger")
	public CronTriggerFactoryBean sampleJobTrigger(@Qualifier("SampleJob") JobDetail jobDetail) {
		return SchedulerConfig.createCronTrigger(jobDetail, frequency, null);
	}

	@Override
	public void execute(JobExecutionContext context) throws JobExecutionException {
		System.out.println("Running SampleTrigger | frequency {}" + frequency);
	}
}

将触发器归类在一起 JobCollections.java

为了解决 @Qualifier(“VipUpdateJob”) required a bean of type XXX that could not be found

@Service
@DisallowConcurrentExecution
public class JobCollections {

  @Bean(name = "PlatformProfitTotalJob")
  public JobDetailFactoryBean sampleJob() {
    return SchedulerConfig.createJobDetail(PlatformProfitTotal.class, "");
  }

  @Bean(name = "VipUpdateJob")
  public JobDetailFactoryBean VipUpdateJob() {
    return SchedulerConfig.createJobDetail(VipUpdate.class, "");
  }

}

详细代码查看Demo:spring-boot-quartz-custom 自己重构的Demo

问题

jobDetail is null

专门定义一个类放 Bean

@Bean(name = "TicketAbortJob")
public JobDetailFactoryBean TicketAbortJob() {
  return SchedulerConfig.createJobDetail(TicketAbort.class, "");
}

Error creating bean with name ‘liquibase’ defined in class path resource … /config/DatabaseConfiguration.class

删除 liquibase 相关的jar包

quartz Driver:com.mysql.jdbc.Driver@e34ffb2 returned null for URL:jdbc:h2

删除以下依赖

<dependency>
	<groupId>com.h2database</groupId>
	<artifactId>h2</artifactId>
	<version>1.4.185</version>
</dependency>

无法连接数据源

DataSource dataSource = DataSourceBuilder.create().driverClassName("com.mysql.jdbc.Driver")
			.url("jdbc:mysql://localhost:3306/yea").username("root").password("mysqlpwd")
			.build();

shadowsock 代理 导致连接出错

If you want an embedded database please put a supported one on the classpath. If you have database settings to be loaded from a particular profile you may need to active it (no profiles are currently

active).

jar包:mysql-connector-java 再添加一次

Failure obtaining db row lock quartz failure obtaining db row lock

多个项目用一张表导致的 要么改表名 要么 schedulerName 修改

To use the DataSourceBuilder you need to have commons-dbcp, or tomcat-jdbc or hikaricp on your classpath else it won’t work. I you don’t have one of those you will get the message as you get.

private static final String[] DATA_SOURCE_TYPE_NAMES = { “org.apache.tomcat.jdbc.pool.DataSource”, “com.zaxxer.hikari.HikariDataSource”, “org.apache.commons.dbcp.BasicDataSource”, “org.apache.commons.dbcp2.BasicDataSource” };

<!-- quartz -->
<dependency>
  <groupId>org.apache.commons</groupId>
  <artifactId>commons-dbcp2</artifactId>
  <version>2.1.1</version>
</dependency>
<dependency>
  <groupId>commons-pool</groupId>
  <artifactId>commons-pool</artifactId>
  <version>20030825.183949</version>
</dependency>
<dependency>
  <groupId>org.quartz-scheduler</groupId>
  <artifactId>quartz</artifactId>
  <version>2.2.1</version>
</dependency>
<dependency>
  <groupId>commons-dbcp</groupId>
  <artifactId>commons-dbcp</artifactId>
  <version>20030825.184428</version>
</dependency>
<dependency>
  <groupId>org.apache.tomcat</groupId>
  <artifactId>tomcat-jdbc</artifactId>
  <version>8.5.9</version>
</dependency>


spring version : 4.2.5.RELEASE

datasource 无法找到

<dependency>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>

性能不好的机器很容易出现程序挂死的问题

修改设置
  • 通过VisualVm 监控发现,可用堆空间急速升到2G,原来是1G, 正使用的堆空间急速降低

  • 通过lsof -i:port , 发现正常情况原来只有一个进程的Java程序,会多出来很多其他进程

参考答案:http://www.techpaste.com/2016/03/quartz-scheduler-shutdown/

Issues faced due to improper implementation of Quartz Scheduler Shutdown is servers will not go down in graceful or sometime force shutdown’s also and will casue servers to go to incompatible state and Application process will get hung till we manually kill the process from back end and restart again.

To over come issues like this Quartz have provided a quartz scheduler shutdown hook plugin which catches the event of the JVM terminating, and calls shutdown on the scheduler.

org.quartz.threadPool.makeThreadsDaemons=true
org.quartz.scheduler.makeSchedulerThreadDaemon=true
org.quartz.scheduler.interruptJobsOnShutdown=true
org.quartz.plugin.shutdownhook.class = org.quartz.plugins.management.ShutdownHookPlugin
org.quartz.plugin.shutdownhook.cleanShutdown = true

We need to mark the scheduler threads daemon as a daemon thread is a thread that does not prevent the JVM from exiting when the program finishes but the thread is still running which in turn does not interfere with the JVM while shutting down. An example for a daemon thread is the garbage collection.

org.quartz.threadPool.makeThreadsDaemons

Can be set to “true” to have the threads in the pool created as daemon threads. Default is “false”.

org.quartz.scheduler.makeSchedulerThreadDaemon

A boolean value (‘true’ or ‘false’) that specifies whether the main thread of the scheduler should be a daemon thread or not. Logs and Outputs:

When you have successfully implemented Quartz shutdown hook plugin you shall see below initialization logs while starting the JVM:

应用程序关闭时自动释放

引用关于IntrospectorCleanupListener一段解释: 在服务器运行过程中,Spring不停的运行的计划任务和OpenSessionInViewFilter,使得Tomcat反复加载对象而产生框架并用时可能产生的内存泄漏,则使用IntrospectorCleanupListener作为相应的解决办法。 Spring中的提供了一个名为org.springframework.web.util.IntrospectorCleanupListener的监听器。它主要负责处理由 JavaBeans Introspector的使用而引起的缓冲泄露。 Spring中对它的描述如下: 它是一个在web应用关闭的时候,清除JavaBeans Introspector的监听器。web.xml中注册这个listener,可以保证在web应用关闭的时候释放与掉这个web应用相关的class loader 和由它管理的类。如果你使用了JavaBeans Introspector来分析应用中的类,Introspector 缓冲中会保留这些类的引用.结果在你的应用关闭的时候,这些类以及web 应用相关的class loader没有被垃圾回收.不幸的是,清除Introspector的唯一方式是刷新整个缓冲。这是因为我们没法判断哪些是属于你的应用的引用.所以删除被缓冲的Introspection会导致把这台电脑上的所有应用的Introspection都删掉。 需要注意的是,Spring 托管的bean不需要使用这个监听器.因为Spring它自己的Introspection所使用的缓冲在分析完一个类之后会被马上从JavaBeans Introspector缓冲中清除掉。应用程序中的类从来不直接使用JavaBeans Introspector。所以他们一般不会导致内部查看资源泄露。但是一些类库和框架往往会产生这个问题。 例如:Struts 和Quartz。单个的内部查看泄漏会导致整个的web应用的类加载器不能进行垃圾回收。在web应用关闭之后,你会看到此应用的所有静态类资源(例如单例)。这个错误当然不是由这个类自身引起的。 用法很简单,就是在web.xml中加入:

<listener>  
    <listener-class>org.springframework.web.util.IntrospectorCleanupListener</listener-class>  
</listener> 

只知道servlet标准不允许在web容器内自行做线程管理,quartz的问题确实存在。 对于Web容器来说,最忌讳应用程序私自启动线程,自行进行线程调度,像Quartz这种在web容器内部默认就自己启动了10线程进行异步job调度的框架本身就是很危险的事情,很容易造成servlet线程资源回收不掉,所以我一向排斥使用quartz。 quartz还有一个问题就是不支持cluster。导致使用quartz的应用都没有办法做群集

spring 监听器 IntrospectorCleanupListener简介

转载请注明出处 来源:paraller’s blog

参考网站

springBoot-quartz-demo

pavansolapure/opencodez-samples

spring-boot-quartz-custom 自己重构的Demo

springboot-quartz-mongodb

分布式定时任务(一)

Scheduling in Spring with Quartz

xml的格式:Spring 4 + Quartz Scheduler Integration Example

Quartz应用与集群原理分析