DCOP: Desktop COmmunications Protocol Preston Brown October 14, 1999 Revised and extended by Matthias Ettrich Mar 29, 2000 Extended with DCOP Signals by Waldo Bastian Feb 19, 2001 Motivation and Background: -------------------------- The motivation behind building a protocol like DCOP is simple. For the past year, we have been attempting to enable interprocess communication between KDE applications. KDE already has an extremely simple IPC mechanism called KWMcom, which is (was!) used for communicating between the panel and the window manager for instance. It is about as simple as it gets, passing messages via X Atoms. For this reason it is limited in the size and complexity of the data that can be passed (X atoms must be small to remain efficient) and it also makes it so that X is required. CORBA was thought to be a more effective IPC/RPC solution. However, after a year of attempting to make heavy use of CORBA in KDE, we have realized that it is a bit slow and memory intensive for simple use. It also has no authentication available. What we really needed was an extremely simple protocol with basic authorization, along the lines of MIT-MAGIC-COOKIE, as used by X. It would not be able to do NEARLY what CORBA was able to do, but for the simple tasks required it would be sufficient. Some examples of such tasks might be an application sending a message to the panel saying, "I have started, stop displaying the 'application starting' wait state," or having a new application that starts query to see if any other applications of the same name are running. If they are, simply call a function on the remote application to create a new window, rather than starting a new process. Implementation: --------------- DCOP is a simple IPC/RPC mechanism built to operate over sockets. Either unix domain sockets or tcp/ip sockets are supported. DCOP is built on top of the Inter Client Exchange (ICE) protocol, which comes standard as a part of X11R6 and later. It also depends on Qt, but beyond that it does not require any other libraries. Because of this, it is extremely lightweight, enabling it to be linked into all KDE applications with low overhead. Model: ------ The model is simple. Each application using DCOP is a client. They communicate to each other through a DCOP server, which functions like a traffic director, dispatching messages/calls to the proper destinations. All clients are peers of each other. Two types of actions are possible with DCOP: "send and forget" messages, which do not block, and "calls," which block waiting for some data to be returned. Any data that will be sent is serialized (marshalled, for you CORBA types) using the built-in QDataStream operators available in all of the Qt classes. This is fast and easy. In fact it's so little work that you can easily write the marshalling code by hand. In addition, there's a simple IDL-like compiler available (dcopidl and dcopidl2cpp) that generates stubs and skeletons for you. Using the dcopidl compiler has the additional benefit of type safety. This HOWTO describes the manual method first and covers the dcopidl compiler later. Establishing the Connection: ---------------------------- TDEApplication has gained a method called "TDEApplication::dcopClient()" which returns a pointer to a DCOPClient instance. The first time this method is called, the client class will be created. DCOPClients have unique identifiers attached to them which are based on what TDEApplication::name() returns. In fact, if there is only a single instance of the program running, the appId will be equal to TDEApplication::name(). To actually enable DCOP communication to begin, you must use DCOPClient::attach(). This will attempt to attach to the DCOP server. If no server is found or there is any other type of error, attach() will return false. TDEApplication will catch a dcop signal and display an appropriate error message box in that case. After connecting with the server via DCOPClient::attach(), you need to register this appId with the server so it knows about you. Otherwise, you are communicating anonymously. Use the DCOPClient::registerAs(const QCString &name) to do so. In the simple case: /* * returns the appId that is actually registered, which _may_ be * different from what you passed */ appId = client->registerAs(kApp->name()); If you never retrieve the DCOPClient pointer from TDEApplication, the object will not be created and thus there will be no memory overhead. You may also detach from the server by calling DCOPClient::detach(). If you wish to attach again you will need to re-register as well. If you only wish to change the ID under which you are registered, simply call DCOPClient::registerAs() with the new name. TDEUniqueApplication automatically registers itself to DCOP. If you are using TDEUniqueApplication you should not attach or register yourself, this is already done. The appId is by definition equal to kapp->name(). You can retrieve the registered DCOP client by calling kapp->dcopClient(). Sending Data to a Remote Application: ------------------------------------- To actually communicate, you have one of two choices. You may either call the "send" or the "call" method. Both methods require three identification parameters: an application identifier, a remote object, a remote function. Sending is asynchronous (i.e. it returns immediately) and may or may not result in your own application being sent a message at some point in the future. Then "send" requires one and "call" requires two data parameters. The remote object must be specified as an object hierarchy. That is, if the toplevel object is called "fooObject" and has the child "barObject", you would reference this object as "fooObject/barObject". Functions must be described by a full function signature. If the remote function is called "doIt", and it takes an int, it would be described as "doIt(int)". Please note that the return type is not specified here, as it is not part of the function signature (or at least the C++ understanding of a function signature). You will get the return type of a function back as an extra parameter to DCOPClient::call(). See the section on call() for more details. In order to actually get the data to the remote client, it must be "serialized" via a QDataStream operating on a QByteArray. This is how the data parameter is "built". A few examples will make clear how this works. Say you want to call "doIt" as described above, and not block (or wait for a response). You will not receive the return value of the remotely called function, but you will not hang while the RPC is processed either. The return value of send() indicates whether DCOP communication succeeded or not. QByteArray data; QDataStream arg(data, IO_WriteOnly); arg << 5; if (!client->send("someAppId", "fooObject/barObject", "doIt(int)", data)) tqDebug("there was some error using DCOP."); OK, now let's say we wanted to get the data back from the remotely called function. You have to execute a call() instead of a send(). The returned value will then be available in the data parameter "reply". The actual return value of call() is still whether or not DCOP communication was successful. QByteArray data, replyData; QCString replyType; QDataStream arg(data, IO_WriteOnly); arg << 5; if (!client->call("someAppId", "fooObject/barObject", "doIt(int)", data, replyType, replyData)) tqDebug("there was some error using DCOP."); else { QDataStream reply(replyData, IO_ReadOnly); if (replyType == "TQString") { TQString result; reply >> result; print("the result is: %s",result.latin1()); } else tqDebug("doIt returned an unexpected type of reply!"); } N.B.: You cannot call() a method belonging to an application which has registered with an unique numeric id appended to its textual name (see dcopclient.h for more info). In this case, DCOP would not know which application it should connect with to call the method. This is not an issue with send(), as you can broadcast to all applications that have registered with appname- by using a wildcard (e.g. 'konsole-*'), which will send your signal to all applications called 'konsole'. Receiving Data via DCOP: ------------------------ Currently the only real way to receive data from DCOP is to multiply inherit from the normal class that you are inheriting (usually some sort of TQWidget subclass or TQObject) as well as the DCOPObject class. DCOPObject provides one very important method: DCOPObject::process(). This is a pure virtual method that you must implement in order to process DCOP messages that you receive. It takes a function signature, QByteArray of parameters, and a reference to a QByteArray for the reply data that you must fill in. Think of DCOPObject::process() as a sort of dispatch agent. In the future, there will probably be a precompiler for your sources to write this method for you. However, until that point you need to examine the incoming function signature and take action accordingly. Here is an example implementation. bool BarObject::process(const QCString &fun, const QByteArray &data, QCString &replyType, QByteArray &replyData) { if (fun == "doIt(int)") { QDataStream arg(data, IO_ReadOnly); int i; // parameter arg >> i; TQString result = self->doIt (i); QDataStream reply(replyData, IO_WriteOnly); reply << result; replyType = "TQString"; return true; } else { tqDebug("unknown function call to BarObject::process()"); return false; } } Receiving Calls and processing them: ------------------------------------ If your applications is able to process incoming function calls right away the above code is all you need. When your application needs to do more complex tasks you might want to do the processing out of 'process' function call and send the result back later when it becomes available. For this you can ask your DCOPClient for a transactionId. You can then return from the 'process' function and when the result is available finish the transaction. In the mean time your application can receive incoming DCOP function calls from other clients. Such code could like this: bool BarObject::process(const QCString &fun, const QByteArray &data, QCString &, QByteArray &) { if (fun == "doIt(int)") { QDataStream arg(data, IO_ReadOnly); int i; // parameter arg >> i; TQString result = self->doIt(i); DCOPClientTransaction *myTransaction; myTransaction = kapp->dcopClient()->beginTransaction(); // start processing... // Calls slotProcessingDone when finished. startProcessing( myTransaction, i); return true; } else { tqDebug("unknown function call to BarObject::process()"); return false; } } slotProcessingDone(DCOPClientTransaction *myTransaction, const TQString &result) { QCString replyType = "TQString"; QByteArray replyData; QDataStream reply(replyData, IO_WriteOnly); reply << result; kapp->dcopClient()->endTransaction( myTransaction, replyType, replyData ); } DCOP Signals ------------ Sometimes a component wants to send notifications via DCOP to other components but does not know which components will be interested in these notifications. One could use a broadcast in such a case but this is a very crude method. For a more sophisticated method DCOP signals have been invented. DCOP signals are very similair to Qt signals, there are some differences though. A DCOP signal can be connected to a DCOP function. Whenever the DCOP signal gets emitted, the DCOP functions to which the signal is connected are being called. DCOP signals are, just like Qt signals, one way. They do not provide a return value. A DCOP signal originates from a DCOP Object/DCOP Client combination (sender). It can be connected to a function of another DCOP Object/DCOP Client combination (receiver). There are two major differences between connections of Qt signals and connections of DCOP signals. In DCOP, unlike Qt, a signal connections can have an anonymous sender and, unlike Qt, a DCOP signal connection can be non-volatile. With DCOP one can connect a signal without specifying the sending DCOP Object or DCOP Client. In that case signals from any DCOP Object and/or DCOP Client will be delivered. This allows the specification of certain events without tying oneself to a certain object that implementes the events. Another DCOP feature are so called non-volatile connections. With Qt signal connections, the connection gets deleted when either sender or receiver of the signal gets deleted. A volatile DCOP signal connection will behave the same. However, a non-volatile DCOP signal connection will not get deleted when the sending object gets deleted. Once a new object gets created with the same name as the original sending object, the connection will be restored. There is no difference between the two when the receiving object gets deleted, in that case the signal connection will always be deleted. A receiver can create a non-volatile connection while the sender doesn't (yet) exist. An anonymous DCOP connection should always be non-volatile. The following example shows how TDELauncher emits a signal whenever it notices that an application that was started via TDELauncher terminates. QByteArray params; QDataStream stream(params, IO_WriteOnly); stream << pid; kapp->dcopClient()->emitDCOPSignal("clientDied(pid_t)", params); The task manager of the TDE panel connects to this signal. It uses an anonymous connection (it doesn't require that the signal is being emitted by TDELauncher) that is non-volatile: connectDCOPSignal(0, 0, "clientDied(pid_t)", "clientDied(pid_t)", false); It connects the clientDied(pid_t) signal to its own clientDied(pid_t) DCOP function. In this case the signal and the function to call have the same name. This isn't needed as long as the arguments of both signal and receiving function match. The receiving function may ignore one or more of the trailing arguments of the signal. E.g. it is allowed to connect the clientDied(pid_t) signal to a clientDied(void) DCOP function. Using the dcopidl compiler --------------------- dcopidl makes setting up a DCOP server easy. Instead of having to implement the process() method and unmarshalling (retrieving from QByteArray) parameters manually, you can let dcopidl create the necessary code on your behalf. This also allows you to describe the interface for your class in a single, separate header file. Writing an IDL file is very similar to writing a normal C++ header. An exception is the keyword 'ASYNC'. It indicates that a call to this function shall be processed asynchronously. For the C++ compiler, it expands to 'void'. Example: #ifndef MY_INTERFACE_H #define MY_INTERFACE_H #include class MyInterface : virtual public DCOPObject { K_DCOP k_dcop: virtual ASYNC myAsynchronousMethod(TQString someParameter) = 0; virtual QRect mySynchronousMethod() = 0; }; #endif As you can see, you're essentially declaring an abstract base class, which virtually inherits from DCOPObject. If you're using the standard KDE build scripts, then you can simply add this file (which you would call MyInterface.h) to your sources directory. Then you edit your Makefile.am, adding 'MyInterface.skel' to your SOURCES list and MyInterface.h to include_HEADERS. The build scripts will use dcopidl to parse MyInterface.h, converting it to an XML description in MyInterface.kidl. Next, a file called MyInterface_skel.cpp will automatically be created, compiled and linked with your binary. The next thing you have to do is to choose which of your classes will implement the interface described in MyInterface.h. Alter the inheritance of this class such that it virtually inherits from MyInterface. Then add declarations to your class interface similar to those on MyInterface.h, but virtual, not pure virtual. Example: class MyClass: public TQObject, virtual public MyInterface { TQ_OBJECT public: MyClass(); ~MyClass(); ASYNC myAsynchronousMethod(TQString someParameter); QRect mySynchronousMethod(); }; Note: (Qt issue) Remember that if you are inheriting from TQObject, you must place it first in the list of inherited classes. In the implementation of your class' ctor, you must explicitly initialize those classes from which you are inheriting from. This is, of course, good practice, but it is essential here as you need to tell DCOPObject the name of the interface which your are implementing. Example: MyClass::MyClass() : TQObject(), DCOPObject("MyInterface") { // whatever... } Now you can simply implement the methods you have declared in your interface, exactly the same as you would normally. Example: void MyClass::myAsynchronousMethod(TQString someParameter) { tqDebug("myAsyncMethod called with param `" + someParameter + "'"); } It is not necessary (though very clean) to define an interface as an abstract class of its own, like we did in the example above. We could just as well have defined a k_dcop section directly within MyClass: class MyClass: public TQObject, virtual public DCOPObject { TQ_OBJECT K_DCOP public: MyClass(); ~MyClass(); k_dcop: ASYNC myAsynchronousMethod(TQString someParameter); QRect mySynchronousMethod(); }; In addition to skeletons, dcopidl2cpp also generate stubs. Those make it easy to call a DCOP interface without doing the marshalling manually. To use a stub, add MyInterface.stub to the SOURCES list of your Makefile.am. The stub class will then be called MyInterface_stub. Conclusion: ----------- Hopefully this document will get you well on your way into the world of inter-process communication with KDE! Please direct all comments and/or suggestions to Preston Brown and Matthias Ettrich . Inter-user communication ------------------------ Sometimes it might be interesting to use DCOP between processes belonging to different users, e.g. a frontend process running with the user's id, and a backend process running as root. For this you can use tdesu with the --nonewdcop option. tdesu will then forward the address of the DCOP server as well as the authentication information to the new user. *WARNING*: This gives the user that you su to, full access to your session! If you su to root this will not be a problem, but it may be a problem if you su to another user. By default, KDE applications (e.g. the ones that run as root) that connect to the dcopserver of another user will not accept any incoming DCOP calls. You can override this with DCOPClient::setAcceptCalls() after you have carefully reviewed that your DCOPClient does not provide objects/functions that could be abused for privilege escalation. Example: tdesu --nonewdcop -u root -c kcmroot This will, after tdesu got the root password, execute kcmroot as root, talking to the user's dcop server. Performance Tests: ------------------ A few back-of-the-napkin tests folks: Code: #include int main(int argc, char **argv) { TDEApplication *app; app = new TDEApplication(argc, argv, "testit"); return app->exec(); } Compiled with: g++ -O2 -o testit testit.cpp -I$TQTDIR/include -L$TQTDIR/lib -ltdecore on Linux yields the following memory use statistics: VmSize: 8076 kB VmLck: 0 kB VmRSS: 4532 kB VmData: 208 kB VmStk: 20 kB VmExe: 4 kB VmLib: 6588 kB If I create the TDEApplication's DCOPClient, and call attach() and registerAs(), it changes to this: VmSize: 8080 kB VmLck: 0 kB VmRSS: 4624 kB VmData: 208 kB VmStk: 20 kB VmExe: 4 kB VmLib: 6588 kB Basically it appears that using DCOP causes 100k more memory to be resident, but no more data or stack. So this will be shared between all processes, right? 100k to enable DCOP in all apps doesn't seem bad at all. :) OK now for some timings. Just creating a TDEApplication and then exiting (i.e. removing the call to TDEApplication::exec) takes this much time: 0.28user 0.02system 0:00.32elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (1084major+62minor)pagefaults 0swaps I.e. about 1/3 of a second on my PII-233. Now, if we create our DCOP object and attach to the server, it takes this long: 0.27user 0.03system 0:00.34elapsed 87%CPU (0avgtext+0avgdata 0maxresident)k 0inputs+0outputs (1107major+65minor)pagefaults 0swaps I.e. about 1/3 of a second. Basically DCOPClient creation and attaching gets lost in the statistical variation ("noise"). I was getting times between .32 and .48 over several runs for both of the example programs, so obviously system load is more relevant than the extra two calls to DCOPClient::attach and DCOPClient::registerAs, as well as the actual DCOPClient constructor time.