MQTT powered event bus for Android applications

MQTT powered event bus for Android applications
december 18, 2013 Fredrik Sandell

This is the second blog post describing a recent private android project I’ve worked on together with Daniel Arenhage. In the first blog post I described how we used a subset of JPA to enable the use of common database entities shared between the Android client and the backend. (

Once we had implemented a common DAO layer for both the backend and the client we started wondering if it was possible to push more functionality down into a common artifact and thereby reduce the number of functionally similar components shared by the clients and the backend. The messages sent over the network connection were the first candidate since they are by necessity identical on backend and client. Other parts of the client-server interfaces were however harder to make generic since traditional client-server setups are inherently asymmetrical with only clients being able to initiate requests.

But one of the goals of the application was to achieve bi-directional message passing between the backend and clients to allow instant updates being pushed to client devices. Because this was a private project we opted to forego the normal bi-directional communication option available to Android developers ( and build the fun stuff ourselves.

When I think of message passing in Java over a network I think of JMS message brokers such as ActiveMQ. But we quickly realized that a standard JMS message broker approach might not be what we were looking for since most brokers assume a stable connection to endpoints and a relatively low, constant number of queues. After some investigation we turned to a network protocol called MQTT ( It is a topic centered broker protocol specifically designed for limited bandwidth network devices with shaky connections. The protocol incorporates features such as QoS on messages, last will and testament and support thousands of concurrent active topics. It seemed ideal for our use case. A MQTT network have a central broker which directs messages to specific topics, in a way similar to normal JMS brokers. The MQTT ability to push messages from the backend to the clients opened the possibility to make the network interface on the backend and clients close to symmetrical.

In our setup each client listens to a unique topic, our backend software is, just as any other connected MQTT client, also listening to a unique topic, albeit a topic with a special ID known to all clients at startup. The network interface logic was simply moved to the common artifact shared between backend and clients. Sending and receiving messages are therefore logically handled in the exact same way on both clients and backend. In the backend we must however have a more complex threaded message handler implementation to ensure that the backend does not block when processing a request or sending a response. In the Android client the same problem is handled with a background service sending and receiving messages. Apart for that difference, which is solved by a shared interface and specific message handler implementations on the two types of endpoints, the receiving and sending of messages can be handled in a uniform way across both client and backend.

MQTT broker

Quick note: Even though the MQTT protocol has very small message overhead the Android cloud messaging service( have the ability to aggregate messages from several applications, something that reduces the total message overhead in a way which will never be possible for a single app. This makes the Android cloud messaging service a better message push option if you’re not afraid of the Google empire or super fascinated with MQTT. It is also worth noting that the standard Java serialization mechanism that we use in our project is inefficient, both in terms of speed and size of the serialized objects. A switch to a more efficient serialized format and serialization method would probably save both bandwidth and execution cycles.

Project module structure

Since both the DAO layer, database entities, network messages and network communication is shared between both backend and clients very little of the frustrating code duplication remains. The topic architecture provided by MQTT also allows clients to address each other directly. Most of the apps communication are between users and it became simple to just send communications to other users clients directly, instead of going through the backend. The backend is however often appended as a recipient only to persist the relevant state change that follows from a message to enable recovery of a client if the client storage where to be deleted. The main role of the backend have therefore been demoted to a persistence storage and client authenticator. Most of the business logic is managed by the clients.

The actual message transmission interface was implemented with a visitor pattern on all message types. All messages inherit from:

public abstract class ApplicationMessage implements Serializable {
    private HashSet<Endpoint> destinations = new HashSet<Endpoint>();
    private Endpoint from;
    private String messageId = UUID.randomUUID().toString();
    private String responseToId = null;

    protected abstract void acceptThis(BaseMessageVisitor visitor);

    public void accept(BaseMessageVisitor v) {

All handlers using an visitor inheriting from:

public abstract class BaseMessageVisitor {

    protected BaseMessageVisitor nextVisitor;

    public BaseMessageVisitor setNextVisitor(
                     BaseMessageVisitor nextVisitor){
        this.nextVisitor = nextVisitor;
        return this;

    public BaseMessageVisitor getNextVisitor() {
        return nextVisitor;
    public void beforeAnyMessage(ApplicationMessage msg) {}

    public void visit(EchoMessage msg) {nonHandledMessage(msg);}
    public void visit(LoginMessage msg) {nonHandledMessage(msg);}


    //more typed visit methods
     * Override this method to get notifications about messages 
     * which your implementation is not capable of handling. 
     * E.g. for error handling.
     * @param msg
    protected void nonHandledMessage(ApplicationMessage msg){}

    public void afterAnyMessage(ApplicationMessage msg) {
        if(nextVisitor != null) {

Both of them defined in the common artifact. Implementing business logic in either client or backend simply became a matter of providing a implementation of the BaseMessageVisitor.

In the android client some convenience features were added to allow callbacks to be provided and some logic for transitioning between a background service and a GUI thread. In the end Activities using the interface would do something similar to this if they needed to update the GUI based on a pushed message

public class MainActivity extends Activity  {
    protected void onPause() {
        //disconnect from android service controlling MQTT client

    protected void onResume() {
        //connect to android service controlling MQTT client
    private MQTTServiceConnector mqttServiceConnector =
     new MQTTServiceConnector(this, new ActivityBaseMessageVisitor() {
       public void visit(LoginMessage arg0){
           //method run in GUI thread
           //update som GUI thing
       public void visit(ChatMessage arg0){
           //method run in GUI thread
           //update another GUI thingy

And sending a message with a callback from the android client:

   new LoginAttemptMessage(email, password), //message to send
   new CallbackVisitor() { //how to handle the response
        public void visit(LoginResponseMessage msg) {
         //Method run in GUI thread.
         //Do stuff depending on msg content.
        public void visit(TimeoutMessage msg) {
          //method run in GUI thread
          //will be triggered after 3000 ms if no response is received
        protected void nonHandledMessage(ApplicationMessage msg){
          //Do some error handling since we got unexpected response
          //(Each message have a "responseToId" field which enables
          //us to match incoming messages to correct CallbackVisitor.)
    3000); //timeout

The project is still a work in progress. One of the features high on the wish list is a asymmetrical key system which would allow clients to authenticate the sender of messages.

4 Kommentarer

  1. Dominik Obermaier 4 år sedan

    Great write-up, thanks for sharing! One suggestion, though: If you want a scalable and reliable backend for your system, I would recommend to write a plugin for your MQTT broker of choice for the backend persistence. This has several key benefits:

    * Your MQTT client lib isn’t the bottleneck of your system
    * No artifical overhead (which is small in case of MQTT, but it matters!)
    * You don’t have to care about duplicate message handling or QoS (what if QoS message are not upgraded, even for QoS2 backend subscribers!)

    You can read more about it here:

    Thanks again for sharing,

    • Författare 4 år sedan

      Thank you.
      Great suggestion! I had just come to accept the bottleneck between the backend and broker. Will look into it.

  2. I like the ” build the fun stuff ourselves” attitude, but does that mean you’ve tried to overcome the shortcomings with such a solution as well?

    Keeping the MQTT TCP connection alive in the service and indeed keeping the service alive itself? Maybe you don’t need the ability to receive messages when the app is not running though. Even if you managed that, or don’t need that, you are probably wasting a lot of battery. Compared to GCM which has a global unified/common channel with all optimizations and considerations (such as always available) for this. Just curious.

    • Författare 4 år sedan

      Hi Mattias,

      Fair points. There are gaps in what we have tested. A correctly configured MQTT broker should persist undelivered QoS2 messages until the client collects them. This would mean that the service in android is not required to run all the time, however for push to be useful, our android service is required to be running, as you state, at least with some resonable interval.

      I belive our ”solution” currently is simply to start the accompanying android service as sticky, which would cause the OS to try to keep it alive and restart it when necessary. And, as you state, this is likely to drain more battery that the GCM solution. The GCM solution is probably preferrable in most (all?) cases if you intend to minimize data sent and battery consumption. But alas, less fun.

      I’m not sure how heavy the strain is on battery and network is though, since MQTT is specifically designed for low message overhead and the keep alive ping to the server can be configured for very long intervals.

Lämna ett svar

E-postadressen publiceras inte. Obligatoriska fält är märkta *