This is part of the Semicolon&Sons Code Diary - consisting of lessons learned on the job. You're in the architecture category.

Last Updated: 2024-11-23

Abstract description of pub/sub event systems

There are at least three components: - an event: this is a data container holding information related to the event. E.g. OrderShipper will contain the $order. - a listener: this performs actions necessary to respond to the event. - some some of message bus to connect the events to the listeners.

There is probably also a configuration file with a dictionary mapping event types to arrays of listeners.

How might this design affect coupling?

Coupling is direct knowledge one component has of another. Tight coupling is a pointer directly to a concrete class that provides the required behavior. Loose coupling is a pointer to an interface of some description.

Imagine this (in ruby, not PHP mind you)

class Post
  after_create :create_feed, :notify_followers

  def create_feed
    Feed.create!(self)
  end

  def notify_followers
    User::NotifyFollowers.call(self)
  end
end

class PostsController
  def create
    @post = Post.build(params)
    @post.save
  end
end

gets rewritten as:

class Post
end

class PostsController
  def create
    @post = Post.build(params)
    if @post.save
      publish(:post_create, @post)
    end
  end
end

class FeedListener
  def post_create(post)
    Feed.create!(post)
  end
end

class NotifyFollowersListener
  def post_create(user)
    NotifyFollowers.call(user)
    # You can add more actions here too, if you like.
  end
end

What are the consequences of this rewrite?

The Post model will be simpler to test, since Feed and NotifyFollowers need not be mocked.
Separation of concerns is improved in the Post model, as it can focus on being a repository for data.
In the rewritten version, the PostsController now has an extra responsibility and has become fatter: it must now publish the relevant events.
In the rewritten version, each of the two listeners is coupled to an event name post_create that is also in the controller. This does not seem terribly different from a method call, since we have to pass parameters to the event.
However, the fact that many listeners may subscribe to a single event encourages us to keep this event signature stable and simple. We program to an interface.
On the other hand, this actually could be negative: once we have an event in place, we lose flexibility about payload. (Why? For one, all the listener methods post_create now need the new param with modified payload. Also because we need to have access to this additional information in other places, at least if we want not nullable params). Therefore refactoring is more difficult.
A small concern, but the new design makes it easier to forget to do the important after_create etc. work with. What if a junior writes code and forgets to call the associated events?
There are many more files to deal with. It is tempting to reduce this by doning the actual work itself in this listener (instead of calling another service object), but this might be ill-advised when mocking is needed.

Classic use cases

1. Components in enterprise software that should not be changed

e.g. because they are considered working, or you don't want to incur a round of QA, or, the code truly belongs elsewhere, or their team are hostile to you going in and making changes.

Here, if their code simply publishes an event whenever relevant, you can hook your code into (or swap out, accordingly) without having to mess with their code.

2. You do not want to introduce dependencies into the code doing the calling

I guess the obvious case of dependencies to avoid are binaries that need to be installed on a system. But even within a single binary program, it could be some dependency that is difficult to test, or that is slow/memory heavy to initialize or pass around, or which breaks the layering.

3. When designing libraries that will be used by people without access to the library's source code (or with it, but no interest in modifying it)

Your documentation states that specific events are raised under specific circumstances. They can then, in turn, subscribe and respond to those events.

Example: Client-side JavaScript hooking into browser events triggered by the user.

const el = document.getElementById("close_popup")
el.addEventListener("click", (event) => hide(event.target))

The browser code might look like publish("click", thisElement)

4. Async performance smoothing

Allow you to throttle heavy load periods (e.g. if Obama tweets a lot of work needs to be done and it would overload the system by using all RAM then going to super slow virtual memory in dealing with this). Better to release in manageable batches with a queue worker.

Difference between Pub/Sub and the Observer pattern

Firstly, their coupling differs:

Observer - an object ("the subject") maintains a list of all the Observers to notify after a certain event occurs. On occurence, the subject model itself calls notify(payload) on every entry in the list.
Pub/sub - publishers (the equivalent of the subject above), do not know about the existence of the observers. i.e. they do not maintain a list of them. Instead a third component, a message broker/event bus, filters incoming messages and distributes them accordingly.

Therefore with pub-sub we can communicate messages between different system components without these components knowing anything about each other's identity.

Secondly, although not mandatory, observers are usually synchronous, pub/sub is usually async (message queue).

Thirdly, the observer pattern is implemented in a single application, whereas pub/sub may be cross-application.

Advantages of pub/sub event systems

Easy to turn/on off functionality by environment (e.g. by not responding to certain events)
(in network world) - a manifestation of the loose coupling is that a producer need not know the IP address of a consumer. Compare to a HTTP post request to a micro-service, where the IP address is a must-have. (However, if defence of the HTTP solution: you could simply have another service that is pinged and gives up-to-date IP addresses for each service.)

Issues

People say that it's easy for events to get lost as the load gets higher. It is recommended to avoid custom solutions (and also even redis), and instead use an RDBMS if you don't need extremely high speeds.
(when your system is aysnc but don't strictly have to be) - you introduce the complexity of concurrency to a synchronous process where this complexity would not otherwise exist. (This suggests to sometimes chose sync listeners and sometimes chose aysnc ones on a case-by-case basis.)
Someone looking at the publish code (e.g. in the PostsController above), has more difficulty seeing the full picture than someone looking at the after_create macro. (Hmm... how would I get up to date in that situation? I guess I would just grep for post_create - or perhaps have a tool for mapping these out into a text file/diagram.
If external systems are used, encryption of sensitive data that gets published may be an issue.
What happens if the publisher assumes a subscriber is listening, but it is not? E.g. factor automation might publish error reports when a problem occurs. But the subscriber that tells humans might have crashed, therefore they won't receive notice of the issue. (monitoring would solve this)
Your storage costs go up, especially in the situation where thousands of listeners are subscribed to a topic, but some are slow/down, therefore, you must retain the data for that message until the slowest one in the system has handled it.

Tips

Carry out slow tasks (like email or HTTP requests) within a background queue.
Within the listener, especially if queued, you probably need code for handling job failures.
Watch out for infinite event loops
Use postgres: From HackerNews "Actually, with 20+ years working in financial services, I’d say the transactional RDBMS table as queue pattern is by far the simplest, most reliable, and almost always the best choice queueing system for business applications. Extra reasons why: terrible failure modes with non mainstream dbs, poorer tooling, less developer knowledge (compared to SQL, which everyone knows)
- Divide events into subcategories - e.g. in JS there are events in these categories: focus, network availability, web sockets, forms, mouse, keyboard etc.

Semicolon & Sons