Shutting down ExecutorService 'applicationTaskExecutor'

ajoy sinha picture ajoy sinha · May 2, 2019 · Viewed 10.9k times · Source

I have a Spring-boot application deployed in docker container within in AWS ECS cluster.

My application stack is => Spring Boot -- JPA -- MySQL RDS

Initially application is deployed and accessible through EC2 public IP.

But after few minutes only application is Shutting down ExecutorService 'applicationTaskExecutor' and restart container again. it is happening constantly in every 3/4 mins.

I am not getting this kind of error in local deployment connecting to same RDS.

Here is my application.properties

server.port=8192

spring.datasource.url=jdbc:mysql://multichannelappdatabase.cvjsdfsdfsdfsfs.us-east-1.rds.amazonaws.com:3306/multichannelappdb
spring.datasource.username=xxxxx
spring.datasource.password=xxxxx
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver


spring.jackson.serialization.FAIL_ON_EMPTY_BEANS=false

spring.jpa.database-platform=org.hibernate.dialect.MySQLDialect
spring.jpa.show-sql=true
#spring.jpa.hibernate.ddl-auto=create
spring.datasource.testWhileIdle = true
spring.datasource.timeBetweenEvictionRunsMillis = 10000
spring.datasource.tomcat.testOnBorrow=true 
spring.datasource.tomcat.validationQuery=SELECT 1

and here is my cloudwatch log


16:20:50
2019-05-02 16:20:50.514 INFO 1 --- [ main] o.s.b.w.embedded.tomcat.TomcatWebServer : Tomcat started on port(s): 8192 (http) with context path ''

16:20:50
2019-05-02 16:20:50.516 INFO 1 --- [ main] c.api.app.runner.UserManagerApplication : Started UserManagerApplication in 9.338 seconds (JVM running for 10.291)

16:20:57
2019-05-02 16:20:57.117 INFO 1 --- [nio-8192-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'

16:20:57
2019-05-02 16:20:57.117 INFO 1 --- [nio-8192-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'

16:20:57
2019-05-02 16:20:57.131 INFO 1 --- [nio-8192-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 14 ms

16:23:19
2019-05-02 16:23:19.253 INFO 1 --- [ Thread-4] o.s.s.concurrent.ThreadPoolTaskExecutor : Shutting down ExecutorService 'applicationTaskExecutor'

16:23:19
2019-05-02 16:23:19.254 INFO 1 --- [ Thread-4] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'

16:23:19
2019-05-02 16:23:19.257 INFO 1 --- [ Thread-4] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...

16:23:19
2019-05-02 16:23:19.274 INFO 1 --- [ Thread-4] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.

Need suggestion to resolve this issue.

Still getting the same issue .. any solution will be helpful

Answer

groovedigga picture groovedigga · Sep 17, 2019

Probably your health-checks are not setup correctly. AWS is regularly trying to reach your application on a endpoint certain . If this port/path is not reachable or replies with an error, AWS will restart the container.

Find your Load-Balancer (even if you use ECS it's on the EC2 Dashboard).

  • Under "Instances" you will find instances of your container marked as unhealthy.
  • Under "Health check" you can edit the health check to talk to the correct Port/Url. Make sure it is reachable on your container (and in case of http(s) it should answer with 200 on plain requests).