Events And Messages¶

At Majority we try to be event driven. See Design Principles.

We use RabbitMq as the message broker.

RabbitMQ¶

RabbitMQ Management¶

Links to Management UI:
- https://stage-rabbitmq.minority.com/
- https://prod-rabbitmq.minority.com/

Connections & Channels¶

Connections: Each instance/pod that connects to Rabbit should have 1 connection
Channels: One Channel per queue per instance/pod that the service subscribes to. One additional Channel per instance/pod that is used to publish events.

Existing Topology (as of 2026-07-02)¶

Exchanges¶

Two Exchanges per Event, one main, and then one for dead letters.
The exchange name is the eventName given in the MessageAttribute: [Message("ignored in Rabbit", "Ignored in Rabbit", "<AreaName>.<NameOfClass>")]
The dead letter one is named after the main exchange with _dlx suffix.

Queues¶

Two Queues per service that listens to an event. One main, and then one for dead letters.
The queue is named as <NameOfExchange>.<NameOfServiceListener>, i.e., AccountEventRequest_Minority.Lithic.Service
The dead letter queue is named after the main queue with _dlq suffix (AccountEventRequest_Minority.Lithic.Service_dlq)

Future Topology¶

Exchanges¶

One Exchanges per Event
The exchange name is the eventName given in the MessageAttribute: [Message("ignored in Rabbit", "Ignored in Rabbit", "<AreaName>.<NameOfClass>")]

Queues¶

One Queue per service that listens to an event.
The queue is named as <NameOfExchange>.<NameOfServiceListener>, i.e., AccountEventRequest_Minority.Lithic.Service

Deadletters¶

One deadletter exchange Platform.DeadLetter, that all normal queues routes to as their deadletter exchange.
One deadletter queue Platform.DeadLetterQueue, that receives the deadletters from the exchange.
One deadletter queue for the Platform.DeadLetterQueue queue, called Platform.DeadLetterQueue_dlq. This is a final queue in case the normal deadletter mechanism breaks.
Hydra listens on the Platform.DeadLetterQueue and puts the events into its database. These can then be managed via UI interface in the Hydra web.

EventStream / Auditing¶

One exchange, Platform.EventStream, that is automatically bound to all normal exchanges.
One queue, Platform.EventStream_Minority.Auditing.Service, that auditing uses to receive all events.

Diagram¶

The diagram below shows the future topology using a single event (AccountEventRequest) as an example. Every normal exchange is bound to the shared Platform.EventStream exchange, and every normal queue dead-letters to the shared Platform.DeadLetter exchange.

flowchart TD
    Publisher([Publisher]) -->|publishes event| EX

    subgraph Normal["Per event"]
        EX[/"Exchange<br/>AccountEventRequest"/]
        Q1["Queue<br/>AccountEventRequest_Minority.Lithic.Service"]
        Q2["Queue<br/>AccountEventRequest_Minority.Ledger.Service"]
        EX --> Q1
        EX --> Q2
    end

    Q1 --> S1([Lithic.Service])
    Q2 --> S2([Ledger.Service])

    subgraph Audit["EventStream / Auditing (shared)"]
        ESX[/"Exchange<br/>Platform.EventStream"/]
        ESQ["Queue<br/>Platform.EventStream_Minority.Auditing.Service"]
        ESX --> ESQ
    end

    subgraph DeadLetter["Dead letters (shared)"]
        DLX[/"Exchange<br/>Platform.DeadLetter"/]
        DLQ["Queue<br/>Platform.DeadLetterQueue"]
        DLQFinal["Queue<br/>Platform.DeadLetterQueue_dlq"]
        DLX --> DLQ
        DLQ -.->|fallback if handling breaks| DLQFinal
    end

    EX -. bound to .-> ESX
    ESQ --> Auditing([Auditing.Service])

    Q1 -. dead-letters to .-> DLX
    Q2 -. dead-letters to .-> DLX
    DLQ --> Hydra([Hydra<br/>managed via Hydra web UI])

RabbitMQ Scheduling¶

It is possible to schedule events. See Scheduling for an overview of the scheduling options.

Manually sending events¶

If you need to publish events to RabbitMQ, it's possible to use the MessageOutbox functionality. This option works great, especially when there are many events to publish.

The idea is to manually insert the events as scheduled events into the platform.MessageOutbox table. Then the service will publish them automatically.

This is a sample insert, that will publish a DatabaseDailyMaintenanceMessage as soon as possible.

insert into platform.MessageOutbox
    (MessageId, MessageName, ContentType, MessageData, MessageProperties, Ruid, ScheduledTime)
    select NEWID(), 'Platform.DatabaseDailyMaintenanceMessage', 'application/json', 
    '{"ScheduledOn":"2025-06-05T09:40:06.3345904Z"}', '{"X-Correlation-ID": "772b64e0-07d3-4b7c-a6c0-e24160d184f5"}', 
    '772b64e0-07d3-4b7c-a6c0-e24160d184f5', GETUTCDATE()

Columns:

MessageId can be a random GUID. Make sure to keep it unique for each event.
MessageName should be the RabbitMQ Exchange name (eventName in the MessageAttribute as described above).
MessageData is the JSON-serialized Event class, the body of the event. One easy way to get a starting point is to look at the EIN logs for the event.
MessageProperties contains a bunch of properties. It can be an empty object, but it's best to put X-Correlation-ID (which is the RUID) there to ensure the subscribing code uses the correct RUID.
Ruid should be the same RUID as used in MessageProperties.
ScheduledTime is the time the event should be published. If you insert 1000s of events, it can be a good idea to spread them out over time to flatten the load on the system.

Additional notes:

Any service which has the MessageOutboxModule loaded can publish any event regardless of whether it is the owner or normally publishes/subscribes to the event.
The event publishing code is not optimized to handle many events at the same time. Therefore, it's best to spread out the ScheduledTime over some time so that you do not block the table for any other events. E.g., if you put 50k events to be published at 13:00, and another event comes in to be published at 13:01, it will have to wait until all the 50k events are published.
When you publish events in this way, there will not be any EOUT logs, only EIN/EINR.
Column MessageOutboxId is a BIGINT and serves as the internal primary key for the MessageOutbox table. This MessageOutboxId is the value returned by the messagePublisher.ScheduleEvent call when scheduling an event programmatically.
Unless explicitly set, the Sender column will be populated with the current SQL login name. This is logged as EventSender in the EIN log.

Batch sending events¶

We've previously SendEventsToServiceBus job to send batches of events. This job is quite slow and requires using production credentials on dev machines.
With services running MessageOutBox we can us it to send messages much quicker. Testing indicates 5k/minute leaves some margin to avoid blocking regular message processing. Please test for your individual case as limits may be different, especially on the consumer end!
Example script to batch run events:

DECLARE
@CorrelationId VARCHAR(50) = '5fd8be7d-71d2-4389-8975-ac1889aa01ab',
@BatchSize INT = 5000,
@MaxBatch INT = 100, -- End batch, depends on source data, with 5k batch size this would be 100*5000 = 500 000 total items in this case. Test with a few first to determine performance in the specific case
@CurrentBatch INT = 0, -- Starting batch number, usually 0, but if you test a few manually you can change it to avoid sending duplicates
@CurrentDate DATETIME = GETUTCDATE() -- Starting date to schedule message, change to hard-coded to set a specific future date instead of beginning processing right away

WHILE @CurrentBatch <= @MaxBatch
BEGIN
  INSERT INTO platform.MessageOutbox (MessageId, MessageName, ContentType, MessageData, MessageProperties, Ruid, ScheduledTime)
  SELECT NEWID(), 'PhonePlan.UpdatePhonePlanAccessLevelMessage', 'application/json', 
  '{"UserId":"' + UserId + '","AccessLevel": "Full"}', '{"X-Correlation-ID": "' + @CorrelationId + '"}', 
  @CorrelationId, @CurrentDate FROM [temp_table] 
  ORDER BY UserId
  OFFSET @CurrentBatch * @BatchSize ROWS
  FETCH NEXT @BatchSize ROWS ONLY

  SET @CurrentDate = DATEADD(MINUTE, 1, @CurrentDate)
  SET @CurrentBatch = 1 + @CurrentBatch
END