Arch Forum 2026-08-27¶
Participants: Backend devs, Andy... ???
Agenda¶
- Platform news
- Single queue for Auditing / Data Warehouse
- Deadletter handling & Hydra
- Continous Delivery
- Unit Testing (Andy)
Summary¶
Platform news¶
- [BCP-3567] Discover be-configuration repo by walking up directories / Martin
- [BCP-3578] Dispose distributed lock connections / Found by Ling (fixed by Victor)
- [MIB-14293] Add shared Google Pub/Sub package
- [MIB-13760] Generate RestClient implementations in callers
- [MIB-14302] Remove cache interceptor registration API
More:
- [MIB-14295] Add Hydra dead-letter MCP tools / Martin
- MIB-14295 Add consuming service filter to deadletters
Single queue for Auditing / Data Warehouse¶
What?
- We will migrate to a single queue for forwarding events to auditing and Data Warehouse
Why?
- Most events are useuful in the Data Warehouse, and we often forget to forward them.
- For any event you want to forward you have to change and deploy Auditing.
- Each queue costs a little bit of resources in RabbitMq.
Details
- All events will be included, unless opt out by setting
ExcludeFromEventStreamto true on theMessageAttribute. See for example theRecordUpdateEventevent in Risk. - Currently we exclude the top 30 events based on event count, that we doesnt forward today.
- Ticket
Meeting Notes:
- In MF services we do some custom stuff for remittance, rate, mf ach. how will that work? - Victor will investigate
Deadletter handling & Hydra¶
- Filtering added
- MCP tools added
- Event publish feature in progress
Meeting Notes:
- The daily deadletter message could deeplink to hydra for easier handling
- It would be good to be able to send your own messages - Victor: This is already in progress.
- in remittance, it would be good to retry after some hours. how to do? - Victor will followup, there's no great solution currently.
Continous Delivery¶
Simon deploys to prod by itself today. For example this run in Github. The only guard is the PR review, once approved it will go all the way to prod.
We want to aim for this also in C#-backend. Is it possible?
Meeting Notes:
- We had a flag (devops) before that you could set that allowed things to deploy to prod (looks like only Rasmus every used it)
- When deploying, we often
- Check for TEKMs in kibana
- Can also check DD, i.e for failed transaction or other metrics relating to the specific service
- To think about if we deploy automatically:
- Dealing with config is a bit messy, since they have to be deployed first it can be dangerous to deploy the service.
- Ordering of deploy of service, i.e. bitso must be deployed before remittance
- Its difficult to regression test today
- Regression tests run only on stage, e.g. latest master. So its easy to break stage.
- You cannot write a regression test before you have deployed the feature you want to test to stage, meaning a typical feature have 2 PRs, one for the regr test, one for the feature itself
- Is it good to have a specific time to deploy? E.g. in mornings? Probably not, better to deploy as soon as change is merged, to keep deploys as small as possible
- What about canary? We can have a canary, but it is not straight forward (DB, events, etc)
Unit/Integration Testing¶
Andy showed our current test coverage stats.
Overall its quite high, with only a few outliers below 70%, and a a majority of repos above 80%.
Discussion followed.
Meeting Notes:
- Some new repos (especially services repos) use 90% code cov as the min limit. Reason for such a high number was "Internet Financial Services Best Practice", and that it nowadays is easy to have AI write tests.
- Most devs report they look at the tests AI writes, either in PR reviews, or when reviewing their own code.
- We are not against a limit, but exactly where to put it is open for debate.