Arch Forum 2023-04-27¶
Participants: Andy, Johan D, Liangxiong, Thani, Victor and Zakarya
Agenda¶
- Subscription / Scheduling of for MTU, Remittance
- Azure Service Bus / RabbitMQ
Notes¶
Subscription / Scheduling of for MTU, Remittance: The Moneyflow team has an upcoming feature where a User can subscribe / schedule MTUs (and in future also remittances etc), i.e. every month make a MTU to this msisdn of this amount. The question and our discussion was around how to design this?
Options
- Each area implement it independently.
- The logic is put into Fees & Promotions (F&P).
- A new scheduling area is created.
The basic scheduling logic will be similar between the different areas, and a user might want to see all their subscriptions / schedules in one place in app, and we might want to show all of them on a user in Hydra etc. At the same time there might be advanced features such as do a remittance only when FX rate is above X. There will also be complicated business logic for edge cases, i.e. when a user doesn't have enough money, or the MTU target is not valid anymore etc.
Given this it felt like F&P should be the best place. However, F&P has quite broad and mixed responsibilities so it might still be better with a new "clean" scheduling system that only does scheduling. More investigation is needed.
We also noted that the API needs careful design, to be both generic and support specific uses like the FX rate.
Azure Service Bus / RabbitMQ: Currently we use Azure Service Bus (ASB) for Bank, RabbitMQ for MF/Risk and Kinesis for Risk & DW. This is too many services to provide a single use case (events/messaging). In addition, they each have downsides and issues. The question and our discussion was around what should our vision of eventing look like?
We have thought for a long time that ASB is not the optimal choice. It is expensive, high average and 99% latency and we experience a number of issues locally & testing as well. On top of that we use a deprecated library to talk to ASB.
However, ASB also provides a couple of non-standard features that we use, mainly message scheduling and sessions.
For RabbitMQ we use (.Net library) MassTransit. We have previously tried to implement something to replace scheduling of messages, both by a RabbitMQ plugin and by implementing Hangfire. Neither option worked out.
Finally there is a question if we can take a broader view, for example replace everything with Kafka. JD was hesitant since we have not managed to consolidate the ASB & the two modes that RabbitMQ is deployed in, so it would be easy to end up with even more services and a half-done migration.
During our discussions we could confirm that yes, we still think and agree that ASB is not the future and we should move away. However, we will need to do more research into exactly what we need (i.e. sessions), their problems and if RabbitMQ is the future. Victor will look at this.