Skip to main content
OutSystems

OutSystems Platform Timers and Asynchronous Processes

A Timer is OutSystems Platform's tool that allows executing application logic periodically on a scheduled time. These are also known as batch jobs.

Different timers can be executed at the same time, but the same timer never has more than one execution at a time.

Here follows some common scenarios where Timers may be used:

Scenario Example
Scheduled Jobs

Execute the same job every day at the same time. For example, send every day at 4 a.m. e-mails to subscribers with digest news.

Create a timer to execute an Action that does the job of sending e-mails to subscribers every day at 4 a.m. 

Executing Long Running Actions

Execute application logic that usually takes long time to finish. For example, at 2 a.m. of the 1st day of every month, the system must archive a lot of database records. It takes about 2 hours.

Create a timer to execute an Action that archives records and set it to run at 2 a.m. of 1st day of every month and with the default timeout of 150 minutes (this value can be adjusted in Service Center).

Architecture

In the OutSystems Platform, the following elements are related with all that has to do with Timers:

Element Description
OutSystems Scheduler Service This is the service that has the responsibility of checking for Timers to be executed. It is a multi-threaded service that allows having different Timers executing at the same time.
Runtime Database

The Runtime Database contains all System Entities for managing Timers, such as:

  • The record of all existing timers;
  • The schedule for executing Timers;
  • Current execution of Timers: when started, their timeout, or next execution.
Log Database When a Timer is executed, an entry is created in the Log Database.
Platform Server Configuration Tool This is the tool that allows configuring the maximum number of timers that may execute at the same time in each Front-end Server node.
OutSystems Log Service  In Service Center, it is possible to have access the information about Timers. The Log Service has the responsibility of writing this data in the Log Database.
Deployed eSpace (Application) The deployed application contains the code for the Timer. i.e., it has the application logic designed in the Action executed by the Timer, and also some stub code needed to get the system database up-to-date about the state of the Timer.

Runtime and Log Databases

This section describes the database entities that support Timers and their functionality.

Entity Meta_Cyclic_Job

This entity contains the definitions of Timers created in eSpaces which are stored in records in the ossys_Meta_Cyclic_Job database table. These records are created and managed by the OutSystems Platform and the Effective_Timeout is the only attribute that can be changed by you.

Here is the complete description of the attributes: 

Attribute Description
Id The primary key.
Espace_Id The eSpace where the timer is defined.
Name The Timer name, as defined in the eSpace.
Default_Schedule

The default scheduled time for the Timer, as defined in the eSpace.

This is a string with the time and the frequency the Timer is executed. It has a special editor in Service Studio to set its value. To type it inline, refer to Service Studio’s online help.

This value is used to set the Schedule property in the Cyclic_Job_Shared entity.

Priority

The Timer priority, as defined in the eSpace. It is a value between 1 (Highest) and 4 (Lowest).

When the number of timers being executed at the same time is greater than the maximum allowed for the Front-end Server, timers with highest priority are firstly executed. 

Is_Active Indicates whether the Timer is active. 
SS_Key The Timer key in Service Studio.
Timeout

The Timer default timeout, as defined in the eSpace. The value is in minutes.

If the timer does not finish the job in this time limit, the timer execution is aborted by the system.

Effective_Timeout

If the value is different from zero to it overrides the above Timer default timeout.

Use it to adjust a new timeout, which is done in Service Center, in the TimerDetail page.

IsShared When True, it indicates that instances of the Timer are created in the Cyclic_Job_Shared entity, otherwise they are created in the Cyclic_Job entity.

Entity Cyclic_Job

This entity contains the information about all Timers to be executed by the Scheduler Service in multi-tenant eSpaces. You can browse this information in Service Center (see Monitoring the Execution of Timers).

As there may be multiple Front-end Server nodes executing Timers, each with its own time, the time reference for executing Timers is always the database clock.

Records of this entity are stored in the database in the ossys_Cyclic_Job_Shared table. They are created and managed by the OutSystems Platform.

The Schedule and Next_Run are the only attributes of this entity that can be changed by you. However, don’t set them when the Timer is being executed, which can be verified by checking the value in the Is_Running_Since attribute:

  • If set: the Timer is running;
  • If not set: the Timer is not running.

Here is the complete description of the attributes:

Attribute Description
Meta_Cyclic_Job_Id A reference to the the Meta_Cyclic_Job entity.
Tenant_Id The reference to the tenant where the Timer is running.
Schedule

After the timer has finished its execution, this attribute is used to calculate the time of the next execution.

For example: Schedule: "02:00 10:00 18:00" (run every day at 2 a.m., 10 a.m., and 6 p.m.). If a timer finished on the 10-23-2012 at 18:00:50 then the Next_Run is set to 10-24-2012 at 2:00:00.

This property is initially set by the value defined in the Default_Schedule property of the Meta_Cyclic_Job entity, but it can be changed in Service Center. 

Last_Run The date and time at which the Timer started its last execution (using the database clock).
Next_Run

The date and time of the next execution (using the database clock).

Scheduler Service executes timers which have Next_Run >= current date time.

Last_Duration The time in seconds the timer took to finish its last execution.
Is_Running_Since

The date and time when the Schedule Service started executing the timer (database clock time).

This attribute is also used to synchronize the several Front-end Server nodes because the Scheduler Service will not execute timers with this attribute set (unless in error recovering actions).

When the execution of the timer finishes, this attribute is cleaned. 

Is_Running_By This attribute has the name of the Front-end Server node that is executing the timer. The Is_Running_Since attribute is also set. 
Number_Of_Tries The number of consecutive times the timer was executed without success. 

Entity Cyclic_Job_Shared

This entity contains the information of Timers to be executed by the Scheduler Service in single-tenant eSpaces. You can browse this information in Service Center (see Monitoring the Execution of Timers).

As there may be multiple Front-end Server nodes executing Timers, each with its own time, the time reference for executing Timers is always the database clock.

Records of this entity are stored in the database in the ossys_Cyclic_Job_Shared table. They are created and managed by OutSystems Platform.

The Schedule and Next_Run are the only attributes of this entity that can be changed by you. However, don’t set them when the Timer is being executed, which can be verified by checking the value in the Is_Running_Since attribute:

  • If set: the Timer is running;
  • If not set: the Timer is not running.

Here is the complete description of the attributes:

Attribute Description
Meta_Cyclic_Job_Id A reference to the the Meta_Cyclic_Job entity.
Schedule

After the timer has finished its execution, this attribute is used to calculate the time of the next execution.

For example: Schedule: "02:00 10:00 18:00" (run every day at 2 a.m., 10 a.m., and 6 p.m.). If a timer finished on the 10-23-2012 at 18:00:50 then the Next_Run is set to 10-24-2012 at 2:00:00.

This property is initially set by the value defined in the Default_Schedule property of the Meta_Cyclic_Job entity, but it can be changed in Service Center. 

Last_Run The date and time at which the Timer started its last execution.
Next_Run

The date and time of the next execution.

Scheduler Service executes timers which have Next_Run >= current date time.

Last_Duration The time in seconds the timer took to finish its last execution.
Is_Running_Since

The date and time when the Schedule Service started executing the timer (database clock time).

This attribute is also used to synchronize the several Front-end Server nodes because the Scheduler Service will not execute timers with this attribute set (unless in error recovering actions).

When the execution of the timer finishes, this attribute is cleaned. 

Is_Running_By This attribute has the name of the Front-end Server node that is executing the timer. The Is_Running_Since attribute is also set. 
Number_Of_Tries The number of consecutive times the timer was executed without success. 

Tables oslog_Cyclic_Job_< N >

These tables store all logging information about executed Timers. You can browse it in Service Center (see Checking the Timers Log).

Here is the complete description of the attributes:

Attribute Description
Instant

The date and time when the Timer execution started ('Time of Log' in Service Center).

This is the IIS clock date and time.

Duration The time in seconds the timer took to finish the execution.
Cyclic_Job_Key The Timer key in Service Studio.
Espace_Id The reference to the eSpace where the timer was created.
Tenant_Id  The reference to the tenant where the timer was executed. 
Executed_By  The name of the Front-end Server node that executed the timer ('Server' in Service Center). 
Error_Id 

The reference to the [oslog_]Error_< N > entity.

If set it means there was an error executing the application logic of the timer and in Service Center Timers Log page displays a red link called Error in the row of the timer.

Should_Have_Run_At 

The date and time when this timer should have been executed.

Note that if a system is very loaded, this value can be quite different than the Instant attribute. This may happen because the same eSpace has many tenants and all have the same schedule, other timers are taking too much time to execute, etc.

This is the database clock date and time.

Next_Run  Is a date time representing when this timer should start (based on schedule, as explained above) again.
Cycle Log cycle number for internal use.

How Timers are Executed 

The Scheduler Service is responsible for executing all Timers. This service is multi-threaded, therefore, it allows executing different timers at the same time, but for simplicity we describe below the execution of a single timer.

The steps to execute Timers are as follows:

1. The Scheduler Service cyclically fetches the database for timers to be executed (Next_Run).

2. The Scheduler Service launches a thread to execute the timer that calls a Web Service in the eSpace where the timer is defined for executing it.

3. The Web Service checks first whether the timer is already executing in any Front-end Server node. If not, the Is_Runnining_Since and Is_Runnining_By attributes of the Cyclic_Job_Shared entity are updated. This will lock the timer from executed in any other Front-end Server node.

4. The eSpace executes the action associated with the timer.

5. After the action execution is finished, the Is_Runnining_Since and Is_Runnining_By attributes are cleaned, which frees the timer for a new execution. The Next_Run attribute is recalculated based on the Schedule attribute and the current date and time.

Regarding the Next_Run attribute, it is updated only if it did not change during the execution of the action. This can happen if, for example, the action updates itself that attribute.

6. A record with the information about the timer execution is sent to the Log Service that stores it in the Log Database.

7. The Web Service returns the control to the Scheduler Service, which now becomes ready to execute another timer (step 1.).

Why a Timeout in Timers?

As described before, the Scheduler Service executes a Timer by calling a Web Service built in the eSpace where the timer was created. This means that there is a Web Request involved in Timers execution and, like in any Web Request, there is a maximum time allowed for it to execute in the server. This is the reason why Timers have the 'Timeout in Minutes' property in Service Studio, which by default it is set to 20 minutes, but you may adjust it to each case. 

The Role of the Timeout in Unexpected Errors

If the execution of a Timer terminates due to an unexpected error, the Scheduler Service will recover the timer and synchronize the database only after the timeout time plus 20% has passed. Until then, the data about Timers in the database may be inconsistent.

It is also important that to implement the login in the Timer action so that you make sure that data is kept consistent in case of the Timer execution terminates unexpectedly.

Retrying after an Execution Error 

Whenever a timer has an execution error the OutSystems Platform may execute it again for a number of retries. You can set this number of retries in Service Center, in the Environment Configuration screen, under the Administration folder. By default it is set to 3 retries.

Changing the Schedule of a Timer Dynamically

Besides setting regular and fixed executions for your Timers, it is also possible to schedule the execution of your Timers dynamically, based on the logic you want. For that, simply edit the Timer action, calculate the new date and time of execution, and update it on the entity holding the Timer execution information.

To have access to all necessary entities to set dynamic executions, proceed as follows:

  1. In the eSpace with the Timer action, add a reference to the following System entities:
    • Meta_Cyclic_Job;
    • Cyclic_Job_Shared.
  2. Calculate the date and time of the next execution;
  3. Update the Next_Run attribute of Cyclic_Job_Shared with the value calculated in step 2. 

Example:

Consider the CleanUp timer, which periodically cleans up data from the database, and that is to be executed 2 hours after ending the last execution.

Open the action of the timer and at the end of the flow add the following logic:

  1. Create a Query to get the Cyclic_Job_Shared record of the timer. The Query has the eSpace identifier and the Timer name and input parameters:
  2. Set the query parameters with the following values:
    • eSpaceId: GetOwnerEspaceIdentifier()
    • TimerName: “CleanUp”
    •  
  3. Calculate the next execution date and time to 2 hours from now and set it to the Next_Run attribute of the Cyclic_Job_Shared record: AddHours(CurrDateTime(),2) 
  4. Update the Cyclic_Job_Shared record in the database;
  5. Commit the transaction to clear the lock over the updated record. 

The action flow should end like this:

Managing Timers in Service Center

Service Center provides a set of functionality that allows you to manage your timers, namely:

  • Monitor the execution of Timers;
  • Edit the Timer settings;
  • Force the execution of a Timer;
  • Deactivate/Activate a Timer;
  • Browse the logs of previous executions.

Monitoring the Execution of Timers

In Service Center, you may monitor the execution of your timers in the ‘Environment Health’ option, under Monitoring. The page has a Timers section with a list that shows the order by which Timers are executed.

The criteria to sort Timers in the list are based on the following:

  • The Timer priority;
  • The time a Timer takes to be executed, based on its previous duration. Timers with faster execution times are executed first;
  • The time a Timer is waiting to be executed. As a Timer waiting time grows it tends to ascend in the list.

The next picture shows an example of the Timers section in two moments with 10 minutes of difference. 

Taking the Send Notifications timer in the example, we can see that:

  • It was scheduled to be executed at 14:15:00.
  • It was in fact executed at 14:14:10.

The conclusion is that the timer was executed before what was expected. This may be due to either being explicitly awakened by the application or someone forced its execution in Service Center.

Checking the Timers Health 

In Service Center, you may check the Timer threads the Scheduler Service is executing, in the ‘Environment Health’ option, under Monitoring. 

In the Front-end Servers section click on the detail link in the Scheduler column to display the page with the details of the service. In our case we are only interested in Timer-related threads.

In example above, the Scheduler Service has the following threads related with Timers:

  • The Timers Fetcher is a thread that awakes from time to time to fetch Timer jobs to be executed and puts them in the queue to be picked. At the current time, it was sleeping.
  • There are three Time Processor threads that execute pick Timer jobs from the queue and execute them. The number of Timer Processor threads is configured in OutSystems Platform Server Configuration Tool.

We can also conclude that:

  • No Timer job was being executed as all Timer Processor threads were in ‘Idle’ status.
  • The Queue has no entries, which, once again, means that there are no Timer jobs to be executed at the time.

Editing a Timer

Edit Timers by clicking on the Timer name in one of following places in Service Center:

  • eSpace: edit the eSpace where the timer is defined, select the Timers tab, and a list of Timers is displayed;
  • Timers Log: go to the Timers log page where each line in the list has the Timer name;
  • Environment Health: go to the Environment Health page, check the Timers section where each line in the list has the Timer name.

In the example above we can see that:

  • The timer runs every 15 minutes;
  • The priority is set to normal;
  • The timeout is not set, so it is the default set in Configuration Tool (20 min);
  • The last execution took place on 08-01-2013 at 18:15:09;
  • The next execution is set to 08-01-2013 at 18:30:00.

Forcing the Execution of a Timer

To force a Timer to be executed without waiting for the scheduled time of the next run, simply press the ‘Run Now’ button at the bottom of the page.

Deactivating/Activating a Timer

To stop a timer from being executed, that is, fetched by the Scheduler Service and consume resources, simply press the ‘Deactivate’ button at the bottom of the page. The timer will no longer be fetched by the Scheduler Service.

If the timer is deactivated, the button shows ‘Activate’ and you should press it to get the timer back to work again.

Checking the Timers Log

In Service Center, you may look into how Timers execution has been going. For that, select the ‘Timers’ option, under Monitoring. In case of errors, the Error link is shown, and you can click on it to see the details.