Real-Time Technologies RFI Response:
Real-Time CORBA in the DHDA Project
Point of Contact
The Real-time Object-Oriented Systems Group
Victor Fay Wolfe, PI (email@example.com)
Dept of Computer Science
258 Tyler Hall
The University of Rhode Island
Kingston, RI 02881 USA
The Distributed System Group
Russell Johnston, PI (firstname.lastname@example.org)
RDT&E DIV (NRaD)
53140 Systems Street
San Diego, CA 92152 USA
This RFI response comes from a group of researchers at the University of Rhode Island and the U.S Navy’s NRaD laboratories. For the past 3+ years we have been developing a project called the Distributed Hybrid Database Architecture, which is designed to allow,
among other things, real-time access to distributed real-time databases through a CORBA environment. Developing real-time extensions to CORBA, within the open systems philosophy of adhering to standards such a CORBA and POSIX, has been a major effort in this project.
This RFI summarizes our current work, most of which is published in the accompanying five supporting documents (we refer to these as "Supporting Doc 1, Supporting Doc 2, etc. in the rest of this paper).
1. Real-Time Method Invocations in Distributed Environments. A 1995 paper presented at the International High Performance Computing Conference which provides our original requirements for time-constrained method calls in CORBA.
2. Real-Time CORBA. A draft 1996 NRaD technical report with a section on real-time CORBA requirements which became the original base draft of the OMG RT PSIG whitepaper Sections 1,2, and 4 (edited by Victor Fay Wolfe in the summer of 1996). The report also contains a survey of commercial 1996 CORBA technology, an introduction of the use of CORBA in C3I systems, and a glossary.
3. The Design of An Open System with Distributed Real-Time Requirements. A 1996 paper presented at the IEEE Real-Time Applications Symposium. This paper describes using CORBA to "wrap" a legacy real-time database using the actual requirements of US Navy’s New Attack Submarine C3I system as an example application.
4. Real-Time COTS Message Communication For C3I Systems. A 1997 University of Rhode Island Master’s thesis and technical report that details the use of CORBA on an ATM network to support a real-time C3I application. The report includes performance measurements identifying sources of latency in a CORBA/ATM system and proposes a CORBA Latency Service to provide estimates and/or guarantees of message latencies.
5. Expressing and Enforcing Timing Constraints in a CORBA Environment. A 1997 University of Rhode Island technical report that has been submitted to the 1997 Real-Time Applications Symposium. This paper details real-time extensions to CORBA that have been implemented on the DHDA project including support for Timed Distributed Method Invocations, synchronized clocks, a Real-Time Event Service, a Global Priority Service for uniform real-time scheduling across the distributed system, and a Real-Time Concurrency Control service which handles priorities inversions.
All of these documents and other work on real-time object-oriented databases, real-time object modeling, and CORBA can be found at our Web site:
The remainder of this RFI response follows the outline of Section 4 of the RFI.
Our requirements are derived primarily from military C3I applications, with which we have had extensive experience. Members of our team have been developing military C3I applications at NRaD for many years. Other members of our team have worked closely with the Naval Undersea Warfare Center and Lockheed/Martin in the development of
the C3I system of the Navy’s New Attack Submarine. Many of the New Attack submarine C3I requirements are described in Supporting Document 3. We just list our RT CORBA requirements in this section of our RFI response; we specify more details and justifications in Section 3 of Supporting Document 1 and in pages 25-32 of Supporting Document 2. The requirements of a RT CORBA system are:
Our approach to providing real-time capabilities in military C3I systems is to use a commercial off-the-shelf (COTS) ORB and Services that operate on a COTS operating system that adheres to the POSIX 1003.c standard, all of which are connected by a COTS network, preferably ATM. With this as a baseline, we have extended the CORBA system, as closely as possible within the OMG CORBA architecture, to support real-time. Our extensions take the form of libraries and header files, IDL files, for inclusion with CORBA clients and servers, some modifications to the ORB which have been implemented on Orbix using "filters", and several new Object Services to support real-time
Our approach to designing the (distributed, real-time) C3I system of the US Navy’s New Attack Submarine is described in Supporting Document 3.
We suggest that real-time CORBA technologies be grouped into the following four categories:
Most of our work has been in Dynamic, Soft Real-time that allows aperiodic tasks and has only limited parts of the application requiring high-performance. Thus, we believe that the requirements that we list in this RFI should be grouped under this characterization of RT CORBA categories.
Our prototype architecture consists of two workstations running POSIX 1003.1c compliant operating systems, connected by (preferably) an ATM network, with all entities in the system participating in a COTS CORBA system. Our current prototype uses Solaris 2.5 for the operating systems (on Sun Sparcs) and Orbix 2.0.1 MT as the CORBA software. The CORBA software has been extended with RT CORBA features described throughout this RFI response.
The more involved architecture of the Navy’s New Attack Submarine is described in Supporting Document 3.
All of our extensions to CORBA and use of CORBA in C3I applications make the following assumptions:
Extensions/Modification to OMG Specifications
We have been very careful in our work to stay within the OMG specifications whenever possible. For instance, our work has not changed the IDL language at all. Our changes to existing ORBs are relatively minor and focus on transmittal of real-time information and on enforcement of priority queuing at all places in the ORB. We have prototyped all ORB modification in Orbix without access to source code, but with minor additions as Orbix "filters", which indicates that our ORB modifications are relatively minor. We have modified two existing services (Event, and Concurrency Control), but have preserved their original interface and made only small, well-justified, extensions to their interface. Our modifications to the service semantics, however, are more substantial, as warranted by the new semantics required by real-time. We also have developed several new services (Global Priority, Global Time, Latency) and have preliminary designs for two others (ORB Guarantee Service, Real-Time Bind). We summarize these extensions and modifications here:
Real-time manager. The RT manager is a C++ class with most of the capabilities for Timed Distributed Method Invocations. It is described on page 8 of Supporting Doc 5.
Real-time Environment. The RT environment is a structure that we have defined and then specified in OMG IDL. We envision that this would be a standard type definition in RT CORBA systems. This structure is described on page 9 of Supporting Doc 5.
Real-Time Exceptions. Real-time exceptions are generated within the CORBA exception mechanism for violations of real-time and quality-of-service constraints specified in the RT Environment. The implementation of RT Exceptions is described on page 8 of Supporting Document 5.
Global Time Service. The global time service requires synchronized clocks and allows clients to query for the current time and be assured that the time is common (to within a bounded skew) and understood by all objects in the CORBA environment. This service is described on page 9 of Supporting Document 5.
Global Priority Service. This service translates timing-constraints (e.g. deadlines), importance specifications, and quality-of-service specifications to a single priority number that is meaningful relative to all priorities in the CORBA system. We have further augmented this service to all priorities to "age", which means that priorities increase as time passes, to allow for implementing dynamic earliest-deadline-first scheduling. The Global Priority Service is described on page 10 of Supporting Doc 5.
Distributed Real-Time Scheduling. This ORB modification requires that the ORB and all operating systems, networks, and other resource managers enforce priority-based scheduling and queuing using the global priority assigned by the Global Priority Service. Note that with appropriate choices of Global Priority assignment, that rate-monotonic scheduling can be performed.
Real-Time Event Service. This service extends the CORBA 2.0 Event Service by allowing events to indicate the time at which they occurred (for use in establishing timing
constraints relative to the events), and by allowing for prioritized events based on the Global Priorities described above. The RT Event Service is described on page 12 of Supporting Doc 5.
Real-Time Concurrency Control Service. This service extends the CORBA 2.0 Concurrency Control Service by allowing for priority inheritance (based on Global Priority), It is described on page 14 of Supporting Document 5.
Network Latency Service. This service allows for high-confidence estimates of message latency in a CORBA system. Currently it has been implemented on an ATM network with Orbix. . The service is described in Section 4 of Supporting Doc 4.
Real-Time Bind. This service has not been fully designed nor implemented, but is current work. It will provide for an "intelligent" bind which finds the appropriate server to which to bind a client based on real-time and quality-of-service specifications, including performance polymorphism. Performance Polymorphism is described on page 30 of Supporting Doc 2. We expect this service to be developed by the end of 1998.
Relationships With ORB, Services, Common Facilities.
As stated in the last section, all of our work relates to the ORB and Services, sometimes as extensions, sometimes as modifications, some times as new entities (Services). The details of these relationships are covered in our response on the previous page.
IEEE POSIX. As described in the previous "Dependencies" section of this RFI response, we assume that all operating systems are compliant with IEEE POSIX 1003.1c (real-time and multi-threaded operating system standard). We need many of the capabilities specified in this standard.
ATM. Although not required, the network latency server work we did assumes a network compliant with the ATM standard.
IP/Multicast. Our RT Event Service assumes IP/multicast capabilities (and eventually a reliable IP/multicast protocol).
SQL. Although not directly part of our RT CORBA work, we have done substantial work on extending SQL for real-time (and are original members and whitepaper authors of the ANSI working group on RT SQL). Supporting Doc 3 describes the use of CORBA to "wrap" a real-time database with an SQL standard wrapper. ODMG compliance is also interesting since much of our work has been in real-time object-oriented databases, but we have not investigated it fully yet.
The major limitation of our RT CORBA technology is its relative immaturity. We have a prototype system, built on Orbix, for most of the extensions that we have described (all except the RT Bind). We are working on integrating it into real C3I systems like the US Navy’s Next Attack Submarine, and the US Joint Forces JFLEX planning tool, but these systems do not employ our RT CORBA software yet.
Another limitation is that the new technology is not necessarily fast. Whenever possible, we tried to optimize our software for speed, since speed is often necessary in a real-time system. However, our approach has been that a fast CORBA environment may be necessary is some circumstances, but that it is often not sufficient. Enforcement of timing constraints can be a sufficient achievement. Thus, our work places higher importance on the specification and enforcement of timing constraints, than on fast computing. Consequently, our technology may not be suitable for high-speed applications. Performance numbers for ATM messages are included in Supporting Doc 4, but performance numbers for the other services are being done now and are expected in two URI Master’s thesis reports by the end of May 1997.
The status of our project is:
Answers To Questions in Section 5 of RFI:
What are relevant RT features? This has been covered in the rest of this document.
What features support scalability to large numbers of objects? The fact that we allow a dynamic system in which objects can come and go, provides more flexibility for handling large numbers of objects than fixed, predetermined real-time systems. Also, all of our scheduling and queuing is dynamic and priority-based, so that it is not dependent on the number of clients or servers. In general, we believe our approach scales as well as CORBA itself. It may even scale better since it supports quality-of-service (described below) which allows for flexible, intelligent reduction of QoS when scaling causes excessive demand for resources.
What is our approach to modularity? Our main approach has been to adhere to open system philosophy: stay within standard interfaces where possible and assume underlying standard interfaces when possible. This supports modularity by facilitating plug-and-play of modules. We however, realize that we could go further in this area and have initiated discussions with Peter Krupp’s group at MITRE towards incorporating our work into their "frameworks" highly-modular CORBA.
What is our approach to QoS? We allow for specification of QoS parameters in the RT environment that gets passed with all requests. Currently, this specification is unused since we have not defined QoS parameters for servers yet. We also allow for QoS specifications on servers as described in Section 3.1 of Supporting Doc 1 and page 31 of Supporting Document 2. We further propose to support QoS through "performance polymorphism" where the ORB can select which method to invoke based on system state and QoS requirements. Performance polymorphism is described on page 30 of Supporting Doc 2. Our philosophy of QoS is described on page 22 od Supporting Doc 2.
We allow for QoS to be incorporated into our generation of a Global Priority, as described on pages 10-12 of Supporting Doc 5.
What is the relationship between the ORB and the OS and Communication System? The ORB must work with the OS and communication system to enforce global priority scheduling and queuing at all points. Our real-time extensions to the ORB assume real-time capabilities in the OS (all from POSIX 1003.1c) as described in several places in this response paper.
Is the technology compliant with CORBA 2.0? Yes, with extensions, as detailed throughout this response paper.
What is our approach to interoperability between RT and non-RT systems? We have had this listed as a requirement from the beginning, but are trying to understand and develop real-time first. Thus, we have no current concrete approach.
Is the technology integrated with existing commercials ORBs? Yes, with Orbix. We are giving our source code to Iona in April for them to consider for incorporation into their products.