Arch Forum 2025-06-05¶
Participants: Backend devs & Victor
Agenda¶
- Removal of Sessions
- Event sending by message outbox
- PR reviews
Summary¶
Removal of Sessions¶
Implementing sessions for RabbitMQ is a major effort. When investigating our usage, it turns out that actually
only a few places in the backend use the Service Bus session feature:
| Area | Handler | Message/Event |
|---|---|---|
| Transactions | TransactionCallbackMessageHandler | LedgerTransactionCallbackMessage |
| Transactions | TransactionCallbackMessageHandler | ProcessorCallbackEvent |
| Transactions | TransactionCallbackMessageHandler | ProcessorCallbackSettlementEvent |
| Transactions | TransactionCallbackMessageHandler | LedgerTransactionReconciledMessage |
| Transactions | TransactionCallbackMessageHandler | LedgerTransactionSettlementCallbackMessage |
| Transactions | TransactionCallbackMessageHandler | ZeroDollarTransactionMessage |
| Transactions | ReserveFundsMessageHandler | ReserveFundsMessage |
| Transactions | ReleaseReservedFundsMessageHandler | ReleaseReservedFundsMessage |
| Transactions | DebitReservedFundsMessageHandler | DebitReservedFundsMessage |
| Transactions | ReverseTransactionMessageHandler | ReverseTransactionMessage |
| Transactions | FirebaseTransactionUpdateMessageHandler | FirebaseTransactionUpdateMessage |
| Phoneplan | InvoiceEventHandler | PhonePlanInvoiceUpdatedEvent |
| Phoneplan | InvoiceEventHandler | PhonePlanSubscriptionDebitFeeEvent |
| Phoneplan | WebhookEventHandler | GigsWebhookReceivedEvent |
| Fees & promo | TransactionEventHandler | TransactionCompletedEvent |
| Fees & promo | TransactionEventHandler | TransactionReconciledEvent |
| CardData | ReEncryptCardDataMessageHandler | ReEncryptCardDataMessage |
| CardData | UserSetReEncryptCardDataMessageHandler | UserSetReEncryptCardDataMessage |
Because of this, we won't implement Sessions in Rabbit, and instead the full platform feature is now deprecated and
will be removed. To start with, the SendEvent/ScheduleEvent with sessionId methods are now marked with the
[Obsolete] attribute.
Instead, the plan is to work around sessions with the DistributedLock in the platform.
Note that:
- DistributedLock is not the preferred way to handle concurrency issues in EventHandlers. However, in this case,
it is a quick solution to move away from Sessions without having to rewrite too much logic.
- We should discuss options for how to handle concurrency issues in a separate Arch Forum. There is more than one way,
and it very much depends on the case.
Ling noted that our current distributed lock might throw on Dispose, which makes the event retry, which is not optimal.
It's unclear if it's on purpose or a bug in the 3rd party library we use. To clarify, Victor raised an issue at
madelson/DistributedLock#245. If not fixed upstream, an alternative would be to wrap the
AsyncDispose in our own exception-safe dispose.
Event sending by message outbox¶
A demo of how to publish events manually with RabbitMQ, using the MessageOutbox table. Details at
Events And Messages
Example code
insert into platform.MessageOutbox
(MessageId, MessageName, ContentType, MessageData, MessageProperties, Ruid, ScheduledTime)
select NEWID(), 'Platform.DatabaseDailyMaintenanceMessage', 'application/json',
'{"ScheduledOn":"2025-06-05T09:40:06.3345904Z"}', '{"X-Correlation-ID": "772b64e0-07d3-4b7c-a6c0-e24160d184f5"}',
'772b64e0-07d3-4b7c-a6c0-e24160d184f5', GETUTCDATE()
PR reviews¶
We discussed a couple of questions/scenarios around Pull Requests:
Question: Who is responsible for checking for broken renovates and other leftover PRs?
It's the area's owner(s) who are responsible for checking these and other "leftover" PRs.
However, to help with this, a reminder like the daily dead letter count could be useful.
(David brought up an alternative idea that each team would have a weekly "maintainer" assigned, who would manage these and
related tasks for the week. If we want to change our ownership model in the future, it could be something to consider, but
we don't feel it's necessary now.)
Question: Who should fix broken renovates?
It is the area's owner(s) who are responsible for driving a fix, but they might not be the ones who actually implement it, and can ask for help/delegate instead.
- Ideally, the developer who creates a broken renovate (i.e. by committing a breaking API change) should also fix it.
- If possible, if you commit a breaking change, you can trigger renovate in affected downstream areas before the scheduled weekly renovate runs. This way the change can be handled with minimal disturbance.
Question: Do we want to review "scripted changes" that are made against all/most repos? For example, the PRs that update launchSettings.json?
Several voices were raised that it is a good idea to review changes even if they are done with a script.