Multi-agent system simulation: Quick Start with ZeroMQ [Python]

Created using Python 2.7 and ZMQ 4.2.1.

Recent work has brought me towards the direction of multi-agent A.I. systems. This was fairly challenging because I had no prior training in terms of multi-agent systems. In fact, whenever I had to code, I had always resigned to function-driven code; always shying away from object-oriented  environments.

Realizing that there was no escaping object-oriented programming this time, I quickly dove into it. Subway rides quickly became refresher sessions on classes and boot camps into multi-agent systems and communication.

The biggest question that I had was, how do multiple agents, whether hosted locally or in different systems, communicate with each other? Quick research brought me to four libraries. The codes are located in this GitHub repo.

1. Threading Module
2. SimPy (Discrete Event Simulation for Python)
3. PyRo (Python Remote Objects)
4. ZeroMQ (Distributed Messaging)

My method of going through with this project was to start writing code and just reassess the process when I start breaking stuff. Take note that I didn’t dive into all the capabilities of the first three libraries. It just so happened that ZeroMQ gave me what I wanted faster than the other three.

Threading Module

Python’s threading module is beautiful, especially when you get multiple threads working. Unfortunately, I ran into a wall when I needed to change the agent type. As you can see below, although the Basic agents changed into Better agents, the Better agent threads that were started did not run past the initial three processes. I also had the problem that this simulation might not show what I wanted: which was multiple independent agents communicating with each other. Since they were housed in threads of the same python instance, I don’t see how my goal could be realized.

SimPy (Discrete Event Simulation)

SimPy was pretty cool as well. I wasn’t able to dive really deep into it, but from what I initially saw in the tutorials, SimPy objects had to run in an environment. This is generally how A.I. agents run, which is, in an environment, but I had a problem with the fact that I need to run an environment for these agents to work with each other. I need the agents to run in multiple computers in the future. Also, I need a brokerless system where the agents communicate with each other without having a central “server” or environment to make them run. Here is the implementation of a code that came from the tutorial. The code introduces three cars and a refilling station with two slots. It’s basically a queueing system.

PyRo (Python Remote Objects)

This library was closer to what I needed than Threading and SimPy. It allowed me to run a server and three (3) agents. Truth be told, I was set in using this, but a few things still bothered me:

• Will I be able to convert remote objects into servers as well? As I said, my goal is a brokerless system, where agents communicate from each other. I tried this but to no avail, it did not work. All communication had to go through the server/remote server.
• Pyro4’s expose function, which exposes a remote object’s class’ variables or functions to the other instances, seems to not be working correctly. I might have been using it wrong. I doubt since some of the variables were accessible.
• Communication was extremely easy… to tangle up.

There are a few things that I liked though.

• Pyro has a nameserver, which allows agents to connect to a server or remote object using its name. This is extremely convenient, in my opinion, for situations where the agents have to dynamically connect to new servers or remote objects, since they can just call the name.
• Daemon loop capabilities
• Supports multiple python instances.

*GIF demonstration and code to follow

ZeroMQ

Now, this. Among the four libraries, ZeroMQ gave me what I really needed (Brokerless system); and it did so in a truly fashionable way. Communication among agents was easy because ZeroMQ’s communication style is in terms of patterns. The basic patterns that you could execute are the following:

• PUSH-PULL: one-way communication. Commands can be executed this way.
• REQ-REP: Request and reply communication. This allows agents to respond to another agent’s request. This basically starts a conversation between the agents.
• PUB-SUB: Publish-Subscription communication. This allows an agent to continuously broadcast a message. Agents who are subscribed to a certain message will react to or get woken up whenever it gets pinged. Update streams, heartbeats, and wake-up signals come into mind when this type of comms is mentioned.

There are more patterns that can be used that would allow sending and receiving multiple messages simultaneously without locking the system and network, but even with these three patterns, it’s pretty doable and easy to set up multiple connection types between an agent and parallel instances of itself.

Quick Tips:

• Messages can only be delivered in string or serialized format. Currently, I’ve only tried serializing a dictionary to json format (using simplejson). To my knowledge, pickles, c-pickles, and message packs are also allowed.
• Agents can act as PUSHERS, PULLERS, PUBLISHERS, SUBSCRIBERS, REQUESTERS, AND REPLIERS (and other roles not mentioned above), all at the same time. This allows agents to communicate with each other without having to rely on a server to get the message across. If you are designing an agent to be more than of these types at the same time, you will have to assign different ports to each type of socket.
• There are different types of connections. I’m currently using TCP port connections in my local machine. If I will have to deploy more agents, there has to be a way to assign, reassign, and kill ports dynamically.
• It allows connections between agents that have been coded using different languages.
• Running multiple processes of functions of a certain agent instance will not work, at least in my current implementation, due to overlapping port connections.
• The best thing is, in my case, I only need to create one PY file for a specific agent type. I could run this code in different python instances and have them interact dynamically with each other, since they can take up different roles, as mentioned above.  This is pretty handy because it helps with code organization. This also allows an easier management and deployment of the code in multiple systems.

Below is a simple implementation of three Basic agents trying to establish who is the best among themselves. The faster the timer runs out, the faster an agent can become a ‘Better’ agent. Since the Better agent is the first one to communicate and receive replies from the Basic agents, he is deemed the Best agent. Afterwards, he sends a heartbeat to the two other agents to prevent them from thinking that there is no Best agent alive and start their countdowns again.

This article will be update as I learn more about ZeroMQ. Check the section below for the most recent updates. Drop me a comment if you have any questions as well.

Credits:

1. ZeroMQ’s guide – Due to this library being updated over and over again, most of the code that I saw outside of the site and number 2 below were outdated. Understanding this guide and thinking of how people think and communicate helped me become accustomed to ZeroMQ’s patterns.
2. PyZMQ’s docs – Python bindings of ZMQ.