Bug #3653
Scheduled jobs stop executing
0%
Description
Hello.
I setup special project for server clean up and scheduled it for each day to run. It was working for some time and stopped. I see project in scheduler calendar, but I don't see it in executed project list. And server looks like it is not maintained as planned. :)
Will You need any additional info? I won't change anything in existing configuration, so if You need I could show You everything.
Will be waiting for You reply.
Have a nice day!
Best regards,
Vadim.
Files
Updated by Mariano 6 months ago
- Status changed from New to Feedback
- Assignee set to Vadim
Hi Vadim,
Based on what you described, for some weird reason the scheduled job was removed from QUARTZ but it is still in future jobs.
So, the quick solution is rescheduling the job, please try:
1 - Go to Future Jobs
2 - Go to Pipeline Jobs or Project Jobs tab depending of job type.
3 - Find the job that you have this problem and press the button Reschedule in Actions column.
4 - Check if everything is correct and then press "Accept".
This would synchronize QUARTZ and the job to be executed.
Let me know if you have any issue or doubt.
Please close this ticket if this was solved.
Regards,
MP
Updated by Vadim 6 months ago
Hello Mariano.
Thank You for the reply. It works. But schedule stops working after some time. And I don't understand what it depends on. System works stably, does not restart, no any other tasks or plans are being scheduled. But after several executions job plan is not executed automatically anymore. Maybe there is any possibility to fix it permanently? :)
Best regards,
Vadim.
Updated by Rafael 6 months ago
- Target version set to 5.0-FIX
Hi Vadim,
Have you identified any reason for this desynchronisation like the number of executions of the scheduled job, a failure on a deployment...?
As we haven't really been able to reproduce this, it would be very helpful for us any information you can find out.
Something useful for us could be the server logs when a job was supposed to be executed but didn't run, along with some information about what was the job and when was it supposed to be executed.
As this is very particular and very random, I'm not really sure how can I help you better. :-S
And please let me know if we can help you in any other way.
Cheers!
Rafa
Updated by Vadim 6 months ago
- File rapiddeploy-web-app.zip rapiddeploy-web-app.zip added
Hi Rafael.
No, there are no any problems in system when task should be executed. No any failures. Job just does not start.
Which logs do You need web-app?
Project is quite simple - it runs single command to prune Docker images. Single command, three targets. If I run it manually - no any error occurs. Command can fail only if Docker is not working properly. Task scheduler to run daily at 23:00.
I attach web-app log as see there more errors that I expected. :) Take a look please and let me know what's wrong.
Best regards,
Vadim.
Updated by Vadim 6 months ago
- File rapiddeploy-web-app.zip rapiddeploy-web-app.zip added
Hi Rafael.
I rescheduled job plan yesterday, but it didn't run this night. Attached please find updated log.
Best regards,
Vadim.
Updated by Vadim 5 months ago
- File firefox_KglQoLJCL4.png firefox_KglQoLJCL4.png added
- File firefox_RYhKu3jq4C.png firefox_RYhKu3jq4C.png added
- File firefox_Fv9PGICRVX.png firefox_Fv9PGICRVX.png added
- File firefox_j7h8DtQLFn.png firefox_j7h8DtQLFn.png added
Hello.
Am not sure if I found a problem. But one error in all that setup I found. :) And that's really very fun thing. :) The scheduled job plan is not presented in a job plan list. I don't know how, but I remember that it disappeared after some update. I thought then that it was erased or something else hapend to it and created a new one. The difference is in id and one target. :) Schedule has link to an old job plan, but not a new one. Maybe that affects somehow the scheduler? I'll try to scheduler another one.
Best regards,
Vadim.
Updated by Mariano 5 months ago
- Assignee changed from Mariano to Vadim
Hi Vadim,
You are right, As I mentioned, for scheduled jobs, you have two main types, one is the schedule job that it meant to be executed once and the other type are the recurrent ones where same job is executed N times.
In the first case, when is time to execute the job and if something goes wrong you can see everything in the same job record (ID). But for recurrent jobs the history is completely different. When is time to execute a recurrent jobs, the scheduler clone the original scheduled job with same definition data and runs it as a new job record which have a different job id and the scheduled job. If any error happens on this new job, everything is logged into this new job record.
In other words, in your case, you have a recurrent scheduled job which runs a pipeline job. When is time to execute the job, the scheduler creates a new job based on the recurrent scheduled one which runs the pipeline persisted in the job. The new job will have their one log and record in RD. But the recurrent scheduled job will be in the same state as before.
Let me know if it is not clear enought or you have any doubt.
Cheers,
MP
Updated by Vadim 5 months ago
- File firefox_TeYAXVxhxT.png firefox_TeYAXVxhxT.png added
Hi Mariano.
Thank you. Everything is clear.
Both job plans (old "1" and new one "2") were deployed successfully on a scheduled time. Will check on Monday if they still run. :)
Best regards,
Vadim.
Updated by Vadim 5 months ago
- File firefox_adrKiW8LZm.png firefox_adrKiW8LZm.png added
Hi Mariano.
One of two scheduled jobs didn't run today. :( On a screenshot You can see that both were running over a weekend, but today one didn't start.
Best regards,
Vadim.
Updated by Mariano about 2 months ago
- Tracker changed from Support to Bug
- Status changed from In Progress to Resolved
- Assignee changed from Mariano to Vadim
Hi Vadim,
The good news is that I found the bug, I could reproduce it. The prolem comes when you schedule or reschedule a job and the start date is before current datetime. This is a limitation of quartz framework. I amended our code to avoid this issue and this fix will be available on next release 5.0.27.
Meanwhile, to make your current cron scheduled jobs, you can reschedule them but set the start datetime in the future (one minute from the current datetime it perfect) and leave the same values to the rest of fields.
Let me know if it worked for you. If not, please assign this to me and add a note.
Regards,
MP
Updated by Rafael about 1 month ago
- Status changed from Resolved to Marked for Closure
Hi Vadim,
Finally, version 5.0.27 of RapidDeploy is out with this fix included! :-)
Please let us know if you still see this issue with the upgraded version.
As this is a problem with scheduled jobs, it will take longer to check the fix, so I'll let more days to time this ticket out, but if we don't hear from you in a considerably longer period I'll happily close the ticket as resolved. :-)
Cheers!
Rafa