Skip to content

Arch Forum 2023-06-29

Participants: Liangxiong, Shakib, Shoaib, Thani, Victor and Zak

Agenda

  • Azure Service Bus & MassTransit followup
  • Oncall
  • DbUp DB migrations of stored procedures
  • EventCatalog demo
  • Sendtab area

Notes

Azure Service Bus (ASB) & MassTransit (MT) followup: First topic of the day was a follow-up on the ASB / MT work discussed last time now that Shoaib and Shakib already worked on it a while. They first worked with replacing the ASB library with MT in platform. However, during this they discovered a serious limitation, namely that the subscription cannot be changed after initial setup.
- This becomes very complicated with our use of filters.
- No solution for how new code could be rolled out gradually without losing messages
- Would need changes in many areas
- We could start restricting a single event type per topic, but this would require rethinking critical uses since we intentionally use multiple events on a single topic to help with message ordering, i.e. a single card topic for various card events.

In contrast, after they instead started to "just" upgrade to the new ASB client libraries, everything have gotten easier and simple. This looks like a very promising path, which is already half way.

In the end, that means we will go with "just" an upgrade of the ASB libraries for now and postpone MT (or something else) to the future.

Oncall: Zak brought up an issue from the a OnCall meeting earlier. The Remittance DB used up too much CPU causing alerts. However, this is possible to improve, with a ticket already in jira.

DbUp DB migrations of stored procedures: Several weeks ago we identified that it would be very nice if stored procedures were not handled by change scripts like other db changes, but instead a stored procedure were in the same file always, so it would be easy to see its history, for example when doing a code review. Victor had made some investigation into how the DbUp platform code works and could conclude that we could update which underlying library we use (https://evolve-db.netlify.app/ looks promising) to have support for running these files only if the SP code changed. If, on the other hand, we are fine with re-running all the SPs on each deploy, then our current library can already do it with only slight modifications.

EventCatalog demo: Victor had done some investigation into how we could better and easier document our platform, and found a tool called EventCatalog that looks promising. We looked at a small demo of how it could look, with 3 services and 6 events documented using this tool. The most promising approach would be to generate AsyncAPI definitions, similar to how we today generate Swagger definitions, and then input them into the EventCatalog docs.

We could all agree that this looks like a promising tool that would provide real benefits both to developers and others to get an overview of our services and the events they publish/subscribe to.

Sendtab area: In the Moneyflow team they had a new feature, a "send tab" in the app which should display several different kinds of data like remittance, MTU, recently used beneficiaries, marketing offers for these etc. based on the user's country. The solution were to add a new area whose only purpose is to aggregate together data from these different existing areas, sort of like how Hydra area does for some APIs. There's a background written here https://dev.azure.com/MAJORITY/Documentation/_wiki/wikis/Main/233/20230621-SendTab-Area

This naturally raises the question if we need a more generic API aggregation layer, for example like you could do with GraphQL with Apollo, or a home built solution with .NET proxy Ocelot. It's not really needed now, but could be relevant in the future, if more needs like this occurs, for example to simplify app startup which calls many backend services.