An OutSystems Core Service around a business concept should always provide reusable OutSystems public elements that abstract the business concept. Take the example of a Customer Core Service (module Customer_CS).
Entities to allow consumers to retrieve business data through Aggregates and Queries. Always expose them as read-only, so that consumers are not allowed to use the basic update operations in an ad-hoc way.
Actions to perform business transactions and data updates, encapsulating all the complexity and extra requirements (auditing, system integration, etc.)
Example: Customer_Create, Customer_Approve, Customer_AddAddress
Reusable UI widgets
Example: Customer_Summary, Customer_Card
A recommended Service-Oriented Architecture will abstract a reusable Core Service, when the master of the data is an external system of records.
Abstracting Core Services according to integrations
The following sections present several patterns, according to different integration needs. The key point, is that no matter what kind of integration is required, consumers of Customer_CS always get the same type of abstraction, and become completely independent of any external system of records.
External Core Service
In this scenario, Customer data is coming from an ERP system. Customer_IS interfaces with the ERP, abstracting a normalized API, both for retrieving information and performing transactions.
This creates the flexibility to change the external systems without impacts on core services, as long as the Integration Service API is maintained.
Also note that this scenario does not include entities in Customer_CS to keep a local replica of data. Not keeping replicated data outside the ERP might be due to a business constraint, or because (almost) real-time demand is incompatible with having a delayed data replication (information changes too often).
However, this pattern has several limitations:
- Extra load on the ERP each time data is retrieved.
- More perceived latency caused by extra online communication with the ERP.
- Impossibility to maximize the power of OutSystems Aggregates and Advanced Queries to retrieve Customer data or to combine it with other information.
- Constant need to augment Customer_IS and a strong dependency on the ERP team to provide new APIs each time a different data retrieval is required (e.g. to filter some criteria or to include more or less detail). Typically, to avoid this dependency, developers tend to inappropriately reuse the available API, for instance, by using a method that returns 50 fields, when only 4 are required.
External Core Service with data replica
Adding local entities to Customer_CS will overcome the issues with the External Core Service pattern and actually create a full-blown Core Service.
The Integration Service becomes restrict and stable. Instead of providing a myriad of actions for different data retrieval needs, it only has to supply a method to fetch all customer relevant data, updated since the last sync.
Customer_CS has a timer to regularly synchronize information through the Integration Service. This synchronization should be unidirectional, to avoid complex merges of information - from the ERP (the master of data) to the Core Service. On the opposite direction, when an update is made in Costumer_CS, you must be careful to make sure that the update is successful and synchronously committed in the ERP first (a write-through policy).
You may add some variations to handle certain requirements:
- Summary information only
Replicate only summary data that it’s frequently listed, joined or searched. Full detail is fetched directly at the ERP.
Use this variant when the it is too costly to synchronize the entire database and detail is only required for single entries (not lists). While summary data must be present to search for any entry, synchronizing the entire entry is unnecessary when only 10% of them will be actually visited in detail.
- Read-through caching
Instead of synchronizing data upfront, try to fetch the data from the local cache entity. If it is not there, get the single entry from the external system and cache that record. Cached data can have an expiration date, to force fetching from the source again after a certain period and refresh it locally.
Use this variant when you only need access to single entries, i.e. there is no need to access the entire record list.
These variations can be mixed. Cache summary data regularly and read-through cache the details.
External Core Service with isolated synchronization
When the synchronization process is too complex and constantly tuned, it is recommended to extract it from the Core Service, further isolating Customer_CS from the external system.
Consumers of Customer_CS don’t need to be impacted by the synchronization code. Additionally, if in the future the ERP is deprecated and replaced by functionality built in OutSystems, stripping out the synchronization code has no impact.
In this example, Customer_Sync is the one regularly fetching updated information through the Integration Service, to sync into Customer_CS. Customer_CS still consumes Customer_IS to perform online transactions.
When the source of information comes from different record systems, usually with different formats and APIs, the synchronization process becomes more complex.
Each System of records is interfaced through a different Integration Service. These services may have different APIs to accommodate different protocols, but they must all return data normalized with the same type of structures.
Customer_Sync then orchestrates the synchronization with all Systems of records, updating a single Customer data replica in Customer_CS. This creates a transparency service.
In this example, Customer_CS is not able to update Customers. This pattern is the most common transparency service, where information comes from different sources but does not flow in the opposite way. Examples are electricity or toll readings coming from different sources/formats. They can be corrected or fixed locally, but never sent back to the source.
However, if sending changes back is a requirement, an extra module must be added. Customer_IS abstracts the update actions that will actually send the transaction to the correct system.
To learn more about how to design your application architecture check the Designing the architecture of your OutSystems applications guide.
You can also check additional patterns to get further recommendations for your OutSystems architecture design.