System API’s: An Enterprise Anti-Corruption Layer

Anti-Corruption Layers (ACLs) are an important Domain Driven Design pattern. ACLs translate data from foreign systems into the Ubiquitous Language (the practice of developing a common and rigorous vocabulary between the developers and users within a given business domain to minimize miscommunication, ambiguity, delays, and errors) of the Bounded Context.

ACL's serve a tangential purpose in the context of enterprise integration. In addition to data translation, ACL's also hide the complexity of integration with external systems. Combined with resiliency patterns like Circuit Breakers, the ACL can become an integration abstraction in its own right.

While generally implemented as a tactical pattern, the data translation and resiliency aspects of the ACL can be borrowed and leveraged in a cross-cutting manner. This is generally accomplished by implementing a version of the ACL and a standalone API that is consumed across many Bounded Contexts. Applications will still need to implement "local" ACL's but with reduced complexity limited to translation off the standalone API.

For the context of this blog post we'll consider these sorts of API's through the lens of API-Led Connectivy. With this approach API's are taxonimized as follows:

  • Experience API's are tuned for consumption by a particular engagement channel, like web or mobile.
  • Process API's contain business logic. These most closely map to a Bounded Content and Ubiquitous Language in Domain Driven Design.
  • System API's facade access to systems-of-record and are the focus of this blog post.

A variety of benefits are realized by surfacing System API's in this manner:

  • Integration Consolidation Integration with systems-of-record (SoR) is often an exercise of tribal knowledge abstraction. Mainframes, databases in different lines of business, disparate SaaS services, etc often have a small group of people with silo'd knowledge who bottleneck access. Implementing a System API can capture this knowledge in a single place that can be re-used across different Bounded Contexts as well as other API's. They can also reduce the burden on these teams to constantly field requests for access to their systems to unblock application delivery.

  • Streamlined Data Access While care must be taken to avoid pursuit of a Canonical Model, the System API can still introduce simplifications to a data model to make it easier to consume by downstream Bounded Contexts. This is particularly the case when facading complex data formats in a way that a generalist developer can easily consume. Most developers, for instance, don't have the specialized expertise (or desire) to consume formats like Copybook, IDoc, BAPI, HL7, ACORD, etc.

  • Centralized Resilency A System API centralizes integration resiliency across applications in a single location. This means developers can avoid re-inventing the wheel on each integration to the SoR. Perhaps more importantly it also provides a central point where the resiliency measures can by managed and monitored.

  • Centralized Governance Digital modernization can put new, or at least different, workloads on SoR's that previously served only a single engagement channel. Having a System API facade on a single SoR, or strapped across related SoR's, allows for concerns like rate limiting, throttling and security to be centralized. This has the additional benefit of reducing load on the Central IT team responsible for data governance.

System API Design Approaches

Successful System API's will be used by many downstream consumers. As such care must be taken in the design of the API's contract so its value as an abstraction layer is maximized.

Prefer Schema Evolution to Upfront Definition

It bears repeating but System API's should not be treated as an enterprise wide Canonical Model either explicitly or implicitly. The best way to avoid this is, assuming you have the ability to evolve the API over time, is to only expose fields out you know are necessary for downstream applications. As new fields are requested you can add these Just-in-Time (JIT). Provided the field additions are additive you can avoid having to roll a new major API version. The requests and responses for System API's should lean closer to a Data Access Object then a Business Entity.

You occasionally will have situations where incrementally modifying the API isn't feasible. This usually occurs in the vacuum of a mature C4E or when the upstream owners of the SoR don't have the bandwidth or are otherwise unable to "own" the System API. In these situations it may make sense to "throw up" the complete, underlying data schema directly in the API abstraction.

The risk here is direct coupling to the underlying SoR's schemas. This is dangerous in the same way that directly coupling a database schema to a front-end UI is: any non-additive changes to the database schema will have an impact on the user interface. A similar danger exists with System API's that take this approach. For uncooperative teams or schemas that rarely / never change it can, however, be a workable middleground.

Facade Access to Systems of Record with Duplicate Data

Enterprise data (users, products, customers, sales, etc...) is often shared across multiple systems and lines of business. System API's provide an opportunity to introduce API's that provide an abstraction across this integration complexity. While this composition can occur in the "Process API" layer there is value in hiding the complexity underneath a System API. This is particularly true when considering orchestration. Having cross system transactions abstracted away from downstream consumers makes integration easier, and following the other themes in this post, centralizing concerns like monitoring and governance.

This approach additionally allows you to hide orchestration complexity and pursue choreographed approaches "above" the System API's.

(In subsequent blog posts I'll cover how asynchronous messaging can be used to choreograph state between System API's and data coming from the Process and Experience layers.)

Transparently Leverage Messaging for Resiliency

Messaging brokers, when misused, can become a nightmare of point-to-point complexity. They can also introduce complexity for developers to interact with. This is often the case when using legacy messaging infrastructure that has limited support for modern development platforms (ss is is often the case when older JMS brokers are prevalent. )

System API's can provide an alternative to giving application developers unfettered access to messaging infrastructure. One approach is to immediately bridge to asynchronous messaging for certain kinds of requests.

For example, an "order" object sent via an HTTP PUT could immediately be pushed to a JMS queue in a transaction. Once the transaction completes a 201 response is returned with a link for the client to poll to check the status of the submission. This hides the complexity and explosion of queues while still introducing some level of guaranteed delivery.

The introduction of an unreliable transport, HTTP in this case, to the messaging interaction does pose an issue. If the transaction fails, and the API still has a connection open to the client, it can return a 5xx series error with instructions on how the client should retry the transaction. If the connections to the client is closed (ie, times out), then the client generally will try again. This won't be an issue is the transaction failed but will, in this scenario, create duplicate orders if the transaction was successful.

This can be mitigated by introducing idempotent reception to the API implementation or on the messaging broker to handle duplicate requests.

Data Validation and Error Handling

System API's provide an ideal location to do input validation prior to sending data off to backend systems. This stops needless traffic to the SoR's while allowing you to return intelligent error response codes to downstream clients.

You could, for instance, proactively return 4xx series error codes immediately for upfront validations that fail. Once requests are passed to the SoR you can subsequently make a decision to return a 5xx or 4xx series error based on its response.

System API's also provide an opportunity to streamline error messages. Rather then returning obtuse error messages (or worse, stacktraces) you can add helpful content to API responses that inform the user of what went wrong and what to do. In the case of validation errors, you can return a link to documentation, an enumeration of fields that caused the issue or an example payload. In the case of backend system errors, you might return backpressure data indicating when the client can try the request again or documentation on how to file a ticket to get more help.

In Conclusion

System API's borrow from the Anti-Corruption Layer Domain Driven Design pattern to provide streamlined, governed and secure access into Systems-of-Record. This makes previously hard to get at data readily available for consumption by downstream applications. System API’s also provide an opportunity to facade additional backend complexity, including messaging integration, data validation and error handling, from the same applications. This all serves to establish a clear and effective Anti-Corruption Layer for your SoR.