CO-LaN Management Board has reviewed the request made by the Methods & Tools Special Interest Group for initiating additional steps within Work Package 1 -WP-1) of COBIA Phase III development project. The decision to move forward on these steps has been taken.
The scope of WP-1 (Marshaler Work Package) is to develop marshalers and specify how the data used for method invocations are standardized for transport across process boundaries: “the Marshaler Work Package is the initial step in the Phase III development and consists of developing the marshaling functionality that generalizes interoperability between COBIA-based objects and allows them to interact indirectly through proxies. The product of the Marshaler Work Package is a prerequisite for the remaining work packages.”
Much of the work in WP-1 has been completed, as demonstrated at the CAPE-OPEN 2021 Annual Meeting within the presentation given on the status of COBIA Phase III.
Two additional steps have now been defined by Methods & Tools Special Interest Group and approved by CO-LaN Management Board.
On-the-fly generation of proxies for non-CAPE-OPEN interfaces
CAPE-OPEN is ensuring interoperability of different applications which may or may not execute in different processes. The scope of Work Package 1 (WP1) is to specify, in the COBIA framework, how the data used for method invocations between applications are standardized for transport across process boundaries (if needed) and to develop the marshalers to conduct this transport. To transport the data used for method invocations by a caller on a callee, the view of the caller to the callee needs to be generated and made available to the caller.
For CAPE-OPEN interfaces, this view, a proxy, is produced by a code generator, directly from the CAPE-OPEN interface definition language (IDL). The marshaler creates and exposes a proxy for an interface present in the CAPE-OPEN IDL. This proxy knows the methods available on the interface, the parameter data types, the returned value data types, and a mechanism to return the error code indicating whether the call succeeded, or nature of the call failure. Being based on the known CAPE-OPEN IDL, the entire code of a marshaler can be automatically generated, compiled and made available on a machine as part of the COBIA distribution.
Now COBIA-based applications may contain non CAPE-OPEN interfaces: obviously these non CAPE-OPEN interfaces are private to each application. On-the-fly generation of non-CAPE-OPEN proxy interfaces is therefore necessary. Prototyping of such an automatic generation was performed with success early within WP-1, using libffi. The libffi library provides a portable, high level programming interface to various calling conventions.
A full-scale implementation remains to be done.
Additional data transport mechanism
The generic marshaling functionality, using Transmission Control Protocol/Internet Protocol (TCP/IP) has been delivered and demonstrated. As a data transport protocol, TCP/IP provides a general solution that can be used in the widest range of applications. Due to the overhead inherent to the TCP/IP transport protocol and its multiple layers, it is not the most efficient means for marshaling on a single machine. On a single machine, performance can be improved using more direct interactions using a common shared memory accessible by both the callee and caller.
Memory-based transport eliminates the overhead required for copying data between caller and callee. The implemented mechanism memory-based transport on a single machine will be a circular buffer. A circular buffer is a type of first-in/first out (FIFO) buffer that reuses the memory allocated for data storage by circulating back to the start of the memory block when it reaches the end of the memory block. The circular buffer is made thread safe to prevent data races from occurring, which could compromise the transport of data, leading to unpredictable behavior, or deadlocks. This solution has been prototyped and demonstrated with success to Methods & Tools Special Interest Group by AmsterCHEM.
It will be implemented in COBIA.