Scheduled jobs stop executing
I setup special project for server clean up and scheduled it for each day to run. It was working for some time and stopped. I see project in scheduler calendar, but I don't see it in executed project list. And server looks like it is not maintained as planned. :)
Will You need any additional info? I won't change anything in existing configuration, so if You need I could show You everything.
Will be waiting for You reply.
Have a nice day!
|rapiddeploy-web-app.zip (821 KB) rapiddeploy-web-app.zip||Vadim, 10/27/2020 03:28 PM|
|rapiddeploy-web-app.zip (822 KB) rapiddeploy-web-app.zip||Vadim, 10/28/2020 07:16 AM|
|firefox_KglQoLJCL4.png (24.3 KB) firefox_KglQoLJCL4.png||Vadim, 11/19/2020 03:39 PM|
|firefox_RYhKu3jq4C.png (94.2 KB) firefox_RYhKu3jq4C.png||Vadim, 11/19/2020 03:39 PM|
|firefox_Fv9PGICRVX.png (49.6 KB) firefox_Fv9PGICRVX.png||Vadim, 11/19/2020 03:39 PM|
|firefox_j7h8DtQLFn.png (25.6 KB) firefox_j7h8DtQLFn.png||Vadim, 11/19/2020 03:39 PM|
|firefox_TeYAXVxhxT.png (182 KB) firefox_TeYAXVxhxT.png||Vadim, 11/20/2020 01:35 PM|
|firefox_adrKiW8LZm.png (257 KB) firefox_adrKiW8LZm.png||Vadim, 11/23/2020 12:35 PM|
- Status changed from New to Feedback
- Assignee set to Vadim
Based on what you described, for some weird reason the scheduled job was removed from QUARTZ but it is still in future jobs.
So, the quick solution is rescheduling the job, please try:
1 - Go to Future Jobs
2 - Go to Pipeline Jobs or Project Jobs tab depending of job type.
3 - Find the job that you have this problem and press the button Reschedule in Actions column.
4 - Check if everything is correct and then press "Accept".
This would synchronize QUARTZ and the job to be executed.
Let me know if you have any issue or doubt.
Please close this ticket if this was solved.
Thank You for the reply. It works. But schedule stops working after some time. And I don't understand what it depends on. System works stably, does not restart, no any other tasks or plans are being scheduled. But after several executions job plan is not executed automatically anymore. Maybe there is any possibility to fix it permanently? :)
- Target version set to 5.0-FIX
Have you identified any reason for this desynchronisation like the number of executions of the scheduled job, a failure on a deployment...?
As we haven't really been able to reproduce this, it would be very helpful for us any information you can find out.
Something useful for us could be the server logs when a job was supposed to be executed but didn't run, along with some information about what was the job and when was it supposed to be executed.
As this is very particular and very random, I'm not really sure how can I help you better. :-S
And please let me know if we can help you in any other way.
No, there are no any problems in system when task should be executed. No any failures. Job just does not start.
Which logs do You need web-app?
Project is quite simple - it runs single command to prune Docker images. Single command, three targets. If I run it manually - no any error occurs. Command can fail only if Docker is not working properly. Task scheduler to run daily at 23:00.
I attach web-app log as see there more errors that I expected. :) Take a look please and let me know what's wrong.
- File firefox_KglQoLJCL4.png firefox_KglQoLJCL4.png added
- File firefox_RYhKu3jq4C.png firefox_RYhKu3jq4C.png added
- File firefox_Fv9PGICRVX.png firefox_Fv9PGICRVX.png added
- File firefox_j7h8DtQLFn.png firefox_j7h8DtQLFn.png added
Am not sure if I found a problem. But one error in all that setup I found. :) And that's really very fun thing. :) The scheduled job plan is not presented in a job plan list. I don't know how, but I remember that it disappeared after some update. I thought then that it was erased or something else hapend to it and created a new one. The difference is in id and one target. :) Schedule has link to an old job plan, but not a new one. Maybe that affects somehow the scheduler? I'll try to scheduler another one.
- Assignee changed from Mariano to Vadim
You are right, As I mentioned, for scheduled jobs, you have two main types, one is the schedule job that it meant to be executed once and the other type are the recurrent ones where same job is executed N times.
In the first case, when is time to execute the job and if something goes wrong you can see everything in the same job record (ID). But for recurrent jobs the history is completely different. When is time to execute a recurrent jobs, the scheduler clone the original scheduled job with same definition data and runs it as a new job record which have a different job id and the scheduled job. If any error happens on this new job, everything is logged into this new job record.
In other words, in your case, you have a recurrent scheduled job which runs a pipeline job. When is time to execute the job, the scheduler creates a new job based on the recurrent scheduled one which runs the pipeline persisted in the job. The new job will have their one log and record in RD. But the recurrent scheduled job will be in the same state as before.
Let me know if it is not clear enought or you have any doubt.
- Tracker changed from Support to Bug
- Status changed from In Progress to Resolved
- Assignee changed from Mariano to Vadim
The good news is that I found the bug, I could reproduce it. The prolem comes when you schedule or reschedule a job and the start date is before current datetime. This is a limitation of quartz framework. I amended our code to avoid this issue and this fix will be available on next release 5.0.27.
Meanwhile, to make your current cron scheduled jobs, you can reschedule them but set the start datetime in the future (one minute from the current datetime it perfect) and leave the same values to the rest of fields.
Let me know if it worked for you. If not, please assign this to me and add a note.
- Status changed from Resolved to Marked for Closure
Finally, version 5.0.27 of RapidDeploy is out with this fix included! :-)
Please let us know if you still see this issue with the upgraded version.
As this is a problem with scheduled jobs, it will take longer to check the fix, so I'll let more days to time this ticket out, but if we don't hear from you in a considerably longer period I'll happily close the ticket as resolved. :-)