Steep To Compile

Programming, tea, and music


There’s a lot of talk about messaging systems in modern programming. Messages are a robust way of providing callback functions for various user events, in a very reactive way. Conceptually, a good messaging system should work along the lines of “when this event occurs, this other event should occur.”

There’s a lot of different ways messaging can be implemented. For these next few posts, I’m going to be talking about how to implement a messaging system in C++ that can serve as the basis for communication between several layers, or even across runtime boundaries to other languages, like JavaScript or Python.

The system I will be describing will not work for messaging across the network. This is a messaging system for communicating either between processes on the same machine, or within the same process. It works for connecting an event, such as a button being clicked, to the action that button should perform.

I’ll be writing two posts about this. The first is a normal messaging system, which you may find in various applications in the wild. The second post will talk about how to extend this system into a much more robust one that is easier to use. I’m expecting readers to understand how to use things like std::function, std::bind, lambdas and variadic templates from C++11 before reading this. It should be noted that this system will only work on compilers that support C++11, such as Microsoft’s C++ Compiler, GCC, and Clang. Check your compiler’s documentation for C++11 support.

Firstly, this is the interface we would like to implement:

// Adds an observer to a message.
// msgname: The name of the message to respond to with
// the callback function
// callback: A function to call when the message is sent
// to this object
// context: The context (Probably an object, depending
// on application) that is subscribing to the message.
// This is used to remove this observer when it is
// destroyed.
void add_observer(string msgname,
                  std::function<void(void *)> callback,
                  context_type context);

// Sends a message
// msgname: The name of the message to send
// data: Any extra data sent with the message (Probably
// a pointer to your message class)
void dispatch(string msgname, void *data);

So, lets take the example of using an AppUI object that we want to have an Update function. We want this to be called in our application’s main loop, and as a response to the “update ui” message. The messaging portion of this would look something like this:

  // Do other setup...

  // We're going to subscribe to the "update ui" message
  // sent to some message-able object. This might be the
  // application, it might be a window object, whatever
  // you want to it be.
  // We want the Update function to be called, so we're
  // going to use std::bind to make it call the Update
  // function of this particular object.
  // Finally, the context of the message the "this"
  // object.
  // This is important, because we don't want to keep
  // trying  to call the update function of an object
  // that has been destroyed. Without this, we would end
  // up calling a method on an object that has already
  // been deleted, probably causing a segfault or
  // garbage output.
  <message-able object>.add_observer(
    "update ui",
    std::bind(&AppUI::Update, this, std::placeholders::_1),

// ...

void AppUI::Update(void *data)
  // Do whatever work as a response to the "update ui"
  // message

Here’s an example of what you might see in a game, the context in which I developed the messaging system we’re building towards. We want to the player flash red when they take damage, or flash green when they gain health. We’re going to do this with a component entity system. We will create a player logic component subscribing to a message being sent to its owner entity.

void player_logic_component::initialize()
  // When the "health changed" message is sent, we want
  // to call the health_changed method.
    "health changed",
    std::bind(&player_logic_component::health_changed, this, std::playerholders::_1),

void player_logic_component::health_changed(void *data)
  health_change_data *hc_data = reinterpret_cast<health_change_data *>(data);

  // If the delta of the health change is positive, the
  // player was healed. Flash green.
  if (hc_data->delta > 0)
    // Logic to flash the player green should go here
  // Otherwise, the delta was negative, and the player
  // was damaged. Flash red.
    // Logic to flash the player red should go here

For the most part, this works very well. There is one problem with the way this messaging system works, but that’s what I’ll talk about in my second post on the matter. So, how do we implement this?

First, we need a base “observer” and a “delegate” class. The observer is the class that can respond to messages, and the delegate is what you send the messages to. Keep in mind, you can have one class that can do both of these things, though I would advise against this to avoid multiple inheritance. If you really would like to do this, however, you could combine observer and delegate into a single (encapsulation breaking) class.

You can read up on the observer pattern on oodesign, or you can just download the source code I have created here.

You can see that the observer and delegate base classes are designed to work together to implement this messaging system. Observers can subscribe to messages being sent to delegates, and when observers are destroyed, they will automatically remove themselves from the subscription links to prevent dangling pointers.

The advantages of this system are:

  • You don’t have to declare that the message exists. You can just send a message with a new string name, and it just works.
  • You can send any data along with the message in the pointer you pass to it.
  • Subscribing to messages can be done with a global function, a member function via std::bind, or even a lambda function.
  • You can communicate between classes without them having any dependency on each other, and instead having a common dependency for the delegate they communicate through.
  • This is a fairly fast method of communication. By replacing std::string with your own string class that implements lazy copy, and using a hash table that is more efficient than std::unordered_map, I’ve been able to send thousands of messages within the space of 1/60th of a second.
  • By using a reflection system on top of it, you can even extend this to communicate between a scripting language and your supporting C++ code.

However, there are a couple drawbacks:

  • This would be a lot faster if it weren’t for std::string. Using an integer ID for message ID’s would be faster, but not as versatile. In the end, use whatever works for you.
  • You still have to create a message class if you want to send data along with the message. The pointer you pass in can point to any amount of data, but I found myself annoyed with making new message classes when using it. This can also create new dependencies in your project that you wouldn’t need if you could send arbitrary message arguments.

To solve the first issue, you can use integer ID’s for your messages. I found this to be a bit unwieldy, and it requires programmers to spend more time than they should creating new message types, to avoid collisions on ID’s.

The second issue will be addressed in my next post on the matter, where I will show you how to use variadic templates to make it so you can send any arbitrary number of arguments along with your messages! This eliminates the need for creating message classes, and makes it much easier to use the messaging system.

Properly Access the User’s Home Directory on Windows

I don’t have my home directory on my C: drive. This is apparently blasphemy in the eyes of most windows users, but I’m not insane. I like to use my home directory as, well, my home directory, and my C: drive is too small. I have a solid state drive, so it has enough room just for the OS and all my programs. I also don’t like a bunch of read/writes to appdata on an SSD, since that can damage it. So I’ve moved my home directory from C:\Users\Howard to D:\Users\Howard. Just one tiny character change! However, a lot of programmers didn’t think that “hey, what if a windows user ISN’T insane, and keeps their documents on a different drive than their OS?” Therefor, I have to keep my C:\Users\Howard folder around just for a few programs. I will be calling them out now:

  • Android Studio and Gradle
  • Microsoft Media Player (This was surprising to me)
  • Avast Antivirus

If you work for any of the companies making any of these projects, email someone on the team that is working on it, and get them to fix this. Now.

So how do you fix it? Like this:

Batch Scripts

Getting the user’s home directory from these is really easy, since it’s just a macro that expands to the proper directory:



Python makes it pretty easy to do, mimicking a Unix way of doing things. To do this from python:

from os.path import expanduser
home = expanduser("~")


With Java, just use the System.getProperty function. You don’t even have to import anything new.

String home = System.getProperty("user.home");


C can of course be a bit more tricky, but its still just one function call.

#include <Shlobj.h>

// ...

SHGetFolderPathW(NULL, CSIDL_PROFILE, NULL, 0, home);


And finally, C#:

string home = Environment.GetFolderPath(Environment.SpecialFolder.UserProfile);

I have tested all of these and they work, so you have no excuse not to do this properly now. I have done your job for you, now you owe me money. If your language is not listed here, I would much rather have an email in my inbox from you asking for help than bug in a program I use.


Thoughts on Swift

Swift seems interesting. I’d love to play around with a programming language where I can confuse other people with emoticons for variable names. Dogcow… Oh Apple, its so cute that you spent the time to actually make that a feature of the language.

My London Fog Recipe

A London Fog (Also known as an Earl Grey Latte) is essentially Earl Grey tea and steamed milk, together in frothy goodness.

Its a beverage I enjoy every morning, and have developed a perfect recipe for.

What you’ll need:

  • 12 Ounces of boiling water
  • 1 tablespoon Earl Grey tea
  • 1 teaspoon Paris blended tea
  • About 4 ounces of milk (I prefer whole milk for the creaminess, but whatever you’d want in a latte will work perfectly)
  • 4 tablespoons vanilla syrup
  • 1/4 teaspoon Vanilla Paste

What devices you will need:

  • Something to steam milk with
  • A basket infuser, or the filter out of your coffee maker
  • A cup you can put boiling water in (Not the mug you’ll serve the drink in!)
  1.  Put the tea in the bottom of your cup (Both kinds). Do not put it in any infuser – mix it directly with the water (This makes it stronger!) Pour the boiling water over it. Give that 5 minutes to steep.
  2. While that is steeping, steam your milk with the vanilla syrup mixed in.
  3. Pour the tea through the basket infuser to filter out the leaves.
  4. Stir the vanilla paste into your brewed tea, and then pour the steamed milk in.

It’s not exactly healthy or easy to make, but this is my special recipe for a London Fog. I’m lazy, so I have one of those automatic milk frothing machine and I just use that to make the milk. If you use one of these, set it 140 Fahrenheit for ideal results.  A few extra notes:

  • You can use lavender Earl Grey tea for even more flavour
  • If you want to make the vanilla syrup yourself, just make simple syrup (1:1 water-to-sugar ratio) with 1 tablespoon of vanilla extract for every half cup of water
  • The Paris tea isn’t something you need for the recipe, but I think it adds a nice flavour to the drink.

Bloom Filters and Message Apathy in Game Engines

This is something I’ve been working on in my most recent project, but didn’t make it in to the final production. I finally got it working yesterday, and I’m quite proud.

Messaging systems are important for a lot of game engines, and can be extremely useful for divorcing logic from one system from the logic of another. For the context of this post, I will assume you are using some kind of component based architecture in C++, though the concept should extend to any architectural model and language.

Bloom Filters

Bloom filters are not a widely known data structure, so let me give a bit of background on them. If you are already familiar with them, you can go ahead and skip ahead to the next part. Bloom filters were first thought up by Burton Howard Bloom in 1970 as a space efficient test to check if an element is a member of a data set. They can return a false positive (Saying data is in a set when it is not) but can never return a false negative (Saying data is not in the set when it is).  They are also incredibly fast at determining this information, much more so than a hash table or binary search tree.  So, what are they useful for? They can only tell if something is not in a data set, they can’t actually hold any data! The name is a giveaway: Filters. Any time you want to filter something, bloom filters are a great way to do it. For example, say you run a giant search engine and want to filter out malicious sites from coming up on your search results. However, you don’t want this to hinder the speed at which your search results will be returned to your users, since you would have to check every single result returned by the search against this filter. So, you need something fast. Google does exactly this. If the bloom filter says a site might be malicious, then they can go run it through a more complicated check to see if it actually is. Here is the Wikipedia article.
This video provides a pretty good explanation of how bloom filters work:

So what do we do with them?

Well, there are two ways to do messaging that I’ve seen used. The simple way, and the proper way. The simple way looks something like this:

virtual void Player::SendMessage(unsigned message_id, void *message_data)
  switch (message_id)
      unsigned key_id = *reinterpret_cast<unsigned *>(message_data);

    case MESSAGE_KEY_UP:
      unsigned key_id = *reinterpret_cast<unsigned *>(message_data);
    // Maybe more

This switch statement approach works pretty well, and this is how Windows (The OS) handles sending messages to its UI. I wouldn’t exactly knock it, but there are a few problems:

  • There’s a lot more logic there than there needs to be
  • The SendMessage function is always called, even if the message being sent isn’t cared about
  • This function can get pretty huge and annoying to deal with, sometimes even with multiple developers working in the same functions, which component based architecture strives to avoid.

So, this won’t work. This leads to the proper way to use messages, and it looks something like this:

virtual void Player::Initialize(/* Whatever parameters */)
  RegisterMessageProc("KeyDown", (MESSAGE_PROC) &Player::key_down_logic);
  RegisterMessageProc("KeyUp", (MESSAGE_PROC) &Player::key_up_logic);

  // initialize other stuff

Where your RegisterMessageProc looks like this:

typedef (Component::*MESSAGE_PROC)(void *data);

void Component::RegisterMessageProc(const char *name, MESSAGE_PROC proc)
  m_message_table.insert(name, proc);

To call a message procedure based on the name, you just look up that name in the table, and call the function for it.

This is much less verbose, and it forces all logic for an event to be placed into its own function. Because, trust me, if you don’t force people to use good coding standards, they will not.

You may also notice that now message ID’s are strings, instead of unsigned integers. This is very useful for not having to reserve an identifier and making sure that you can avoid strange post-merge bugs from two developers using the same name. Granted, two developers may use the same string, but this is much less likely than two of them using the same integer because they just chose one at random. So, you can take those strings and throw them into a hash table. Great. This works pretty well, but we can do even better.

By using the bloom filter, we can immediately determine that an object doesn’t care about a message. Thus the name, message apathy. We can then take advantage of one property of bloom filters in component based architectures: You can run a bitwise or operation on them to combine them. So, you do something like this in your game object composition:

void GameObject::Initialize()
  for (unsigned i = 0; i < m_component_count; ++i)
    // Assuming you have the += operator overloaded for your bloom filter
    m_apathy_filter += m_component[i]->GetApathyFilter();

  // initialize other stuff

Where your GetApathyFilter function looks something like:

BloomFilter<string> Component::GetApathyFilter()
  BloomFilter<string> apathy_filter;

  // Whatever iterator type is appropriate for your hash table
  for (iterator it = m_message_table.begin(); it != m_message_table.end(); ++it)

  return apathy_filter;

Now, in your game object, you have an apathy filter. The game object can immediately ignore the message if it is determined from a check in that filter that it is not a message the object will ever respond to.

Going further

Perfect hashing. Its a beautiful thing. There are tools to generate functions, like gperf, and they will give you some pretty awesome results. So, how do we use these? First off, because the message ID’s we will be using will be some data set that will never change for the life of our program, we can generate our perfect hash function from this list of id’s. For example, we could have the ID’s “KeyDown,” “KeyUp,” “LogicUpdate,” and “Collision.” We can use these to generate a perfect hash function that will give us one hash value for each key in the set. How does this affect the bloom filter? Well, it will prevent false positives from happening, ever. Our bloom filter will use only one hash function, so checking if something is in the filter becomes a very cheap operation.

Because of this, we can prevent our objects from ever distributing a message it doesn’t care about. We can take this a step higher – Lets say we post a message to the entire world. The Zero Engine that is used at DigiPen has a composition called Game, one called Space, and one called COG. Games have spaces for children, spaces have COGS for children, and COGS have other COGS for children. So, moving up this hierarchy, we can have each one keep a filter of what it cares about for its own components, as well as what its children care about. So lets say there is a space for game objects that don’t care about key input messages. Maybe it’s a space for the stars in the background of the screen, and they don’t respond to user input in the slightest. Using the approach of a perfect hash function, and combining all of the apathy filters for each of those objects into an apathy filter for the children, we can avoid ever sending the message for key input to any object in the entire space. This can cut down on a lot of completely unnecessary operations.

Of course, to generate this perfect hash function, you need a list of all of your message ID names. This list can be pretty difficult to actually create, depending on how everything works. Generating the perfect hash function may not be trivial, either. Some tools can take hours to generate one, depending on the data set you are using. To solve these issues, I recommend using a hash like “return 0” or maybe a regular hashing function like FNV while developing the game, and then generating the perfect hash function when you are ready to deploy. Keep in mind that even if you get a false positive, the game will still work. This is only an optimization. You could even use your editor to make a list of all the message ID’s, and then as part of your packaging process have a step that creates this hash function, then your engine can load it from a DLL your editor spits out.


If you have any questions about how to implement this, or any specifics on how I have done it, please feel free to ask me in the comments section below or by emailing me!

Hello world!

Welcome to my website! Here is where I will post all my passing fancies of technology and my career, and document most of my own independant work.

© 2017 Steep To Compile

Theme by Anders NorenUp ↑