The Ingescape network layer
The Ingescape network layer
Ingescape enables to create distributed systems based on sets of connected software agents using the Ingescape library. Such architectures can be visualized and edited using the Ingescape editor.
This article presents standard and advanced capabilities brought by Ingescape to support a wide variety of network patterns, familiar in the domain of distributed systems and software buses.
Before going further we advise that you read at least:
The communication between Ingescape agents relies on a software bus, which is hidden behind our model-driven paradigm using definitions and mappings, presented in the first Ingescape agent and in the editor tour.
Fully decentralized, asynchronous and scalable
Most software buses and network systems use central brokers to make applications connect to the others. Some other solutions use hardcoded adresses and ports for these connections. The Ingescape community had the strong objective to provide a fully decentralized solution, and thus avoid these two practices, or at least not make them mandatory.
That is why Ingescape environments are extremely flexible and insure that:
- The absence of an agent will not have another consequence than just making the agent’s services temporarily unavailable,
- Subsets of agent can be formed quickly and easily to enable software development, testing and even mobility,
- Agent duplicates (i.e. having the same definition and possibly the same name) won’t cause any trouble,
- Agents can come and go on the network without strong consequence on the other agents,
- Malfunctioning agents won’t cause other trouble than possibly increasing the network trafic (and absolutely avoid saturation in other agents).
Agents on the same environment only have to share a common network port in order to create their connections to the others. Any Ingescape environment is a fully connected network between all its agents. This will fit with most on site local network configurations. And if heavier network constraints shall apply, Ingescape also supports more controlled network environments, involving security and connection brokers.
Modern systems are reactive or interactive, and heavily rely on multithreading. This is consistent with having asynchronous communications. The Ingescape API is fully asynchronous as well: writing an output value will result in its publication to the inputs of the interested agents. However, the publication is transparent to the developer. And so is the information reception, which triggers the observe callbacks for the implied agent inputs when the received data is actually available.
Ingescape has been designed to support hundreds of agents possibly involving hundreds of thousands of IOPs. This scalability is at the core of the Ingescape network layer, together with the search for low latency, so that Ingescape environments can reach operational and industrial levels (without losing their simplicity).
Patterns based on the Ingescape paradigm and library
To organize a given environment, Ingescape proposes a simple paradigm based on data flows between inputs and outputs of agents. This paradigm was designed to be as simple and straightforward as possible. However, this simplicity still enables to cover most of the possibilities offered by software bus solutions, often without their usual pitfalls.
The most common pattern enabled by Ingescape is Publish/Subscribe, also known as PUB/SUB. This pattern enables one source to feed one or several destinations. The particularity is that the source has no actual knowledge of the destinations. This pattern is adapted to wide information distribution requirements.
This pattern is the preferred one because it shows both its relevance in many industrial situations, and its flexibility for modern information systems. Until now, the projects handled by the community have always been able to fit this pattern, even when involving third-party protocols and architectures.
Message-Oriented Middleware (MoM) are the most common family among the software buses. They enable to exchange text or binary messages between components of distributed systems.
Ingescape addresses this possibility in two different manners:
- The first one is simply by using mappings from one output to several inputs, so that the inputs are fed with text, binary of numeric data written on the other agent’s output. The mapping concept easily enables to create the links between all agents requiring to exchange messages.
- The second one is an actual MoM embedded in the Ingescape library. This MoM is used internally for metadata exchanges between agents. We made it available in the advanced Ingescape API so that developers can create topics (a.k.a. discussion channels) and also send private messages of any kind to any other agent on the network.
Data Distribution Service
Ingescape does not support the Data Distribution Service (DDS) as defined by the Object Management Group. However, it offers a simpler equivalent solution to the DDS DCPS layer and partially to the DLRL layer with the same spirit and objectives.
Based on the Ingescape paradigm and PUB/SUB pattern, it is very easy to conceive an agent (or more) serving as data storage. Such an agent offers as many sets of inputs and outputs as the variables it shall maintain. Once running, this agent becomes a shared persistent distributed memory space on the network, just as any DDS broker would do.
Agents interested in a variable’s value just have to connect one of their inputs to the corresponding output. Agents needing to modify a variable may map one of their outputs to the corresponding input on the data storage agent to modify it.
When an agent maps one of its inputs to one of the data storage outputs, this output is immediately and specifically sent to the relevant agent so that it is always updated with the latest value: later joiners or restarting agents do not miss the latest current value provided by the data storage.
Enterprise Service Buses
An Enterprise Service Bus (ESB) enables various types of software to offer and receive services over in network. The Ingescape paradigm is fully compliant with this approach. The definition models in Ingescape, that can be versioned if needed, are very similar to services descriptions. An agent can both provide and receive services to or from other agents, and redundancy is possible.
In addition, the advanced Ingescape API provides a services description mechanism, already used by the Ingescape editor and some existing agents, to adapt their behavior according to the presence or absence of a given service, attached or not to a specific agent. This description mechanism works in the same way as Zeroconf, enabling each agent to publish sets of keys/values describing its capabilities.
What about good old Client/Server ?
The Ingescape dataflow paradigm based on definitions and mapping does not cover the Request/Reply (a.k.a REQ/REP) pattern completely: with this dataflow paradigm only, it is difficult to have an agent handle queries and send its replies to specific agents which generated each query.
That is why Ingescape introduced the concept of calls. A call can be seen as a public service exposed by an agent to any other agent. The service is called from any agent by naming the target agent and the service name and by providing the expected complementary information as arguments. This is just like a function call in any programming language, except that the call is transmitted to an agent on the network. When a call on an agent is executed, the caller is known and is thus able to receive a private custom reply, using another predefined reply call (or any other relevant mean).
In addition, the MoM features proposed by Ingescape, especially when using private messages for replies, also covers REQ/REP. And finally, the ZeroMQ network layer used by Ingescape implements the REQ/REP pattern natively with special care of the ordering of requests and replies, ready to be used if required for performance or compatibility constraints.
The PUB/SUB and REQ/REP patterns may not be fully convenient to cover advanced task delegation or parallelization scenarios, for instance when a ventilator shall distribute tasks to an arbitrary and variable number of workers, which, in turn, send their results to a collector. Fortunately, the Ingescape network layer, combined with some expertise, can handle these situations.
For other patterns engaging reliability, redundancy, synchronicity, etc., ad-hoc solutions are always possible, often without breaking the simplicity of the Ingescape paradigm. The paradigm exposed by Ingescape relies on the ZeroMQ network layer that already covers many difficulties related to network communications and can be extended to face the most complex real-world challenges, at least as efficiently as all the other commercial and open-source software buses.
Other important and advanced network features
Ingescape aims at making distributed architectures as easy as possible. But it does not neglect or ignore common and advanced challenges faced by heterogeneous systems relying on network communications.
In addition to the advanced patterns described here above, the Ingescape network layer covers the following features:
- Data serialization, even for very large data streams,
- Low-level protocols other than TCP/IP and including UDP, Multicast, IPC, TIPC, shared memory, etc.
- Time management (latency, synchronicity, timeouts, etc.)
- Multithreading and threads synchronisation (with a sane state of mind, avoiding the multiplication of mutexes and other sources of deadlocks)
- Authentication and encryption