Skip to content

Arch Forum 2023-09-07

Participants: JD, Liangxiong, Shakib, Victor, Zak

Agenda

  • Dynamic mapping of handlers in Auditing
  • Replace Automapper with Mapperly
  • Denied transactions in backend db

Notes

Dynamic mapping of handlers in Auditing: Liangxiong brought the topic of dynamic message handlers in Auditing. The basic problem is that its easy to miss updating Auditing when a new event is introduced, and it should be sent to the Data warehouse (DW). In addition, its a bit tedious to do the update.

The discussion went both broad and detailed.

On the big picture, JD thinks we should try to make Data into a product, and not just a side effect or "exhaust" from the backend. Then a discussion around where we should decide what should be sent to DW followed. Is it up to each area (then a marker attribute on the event would fit perfectly), or is it a dedicated microservice (like we have today with Auditing?) Auditing does have some internal logic that is unclear how each area would handle otherwise.

Semi-related the Data team is currently looking at using Big Query instead of Snowflake, so a bigger change of the data pipeline is not unlikely.

For the specific issue of Auditing, one solution would be to add a marker attribute on the events, and then let Auditing handle all events with this attribute. Auditing could do this in two ways, either by reflection on service startup by scanning the assemblies for events with markers, or statically on compile time by source generation.

The reflection solution would put some delay on app startup, unclear exactly how much. Shoaib was always against any runtime reflection because of this reason.

Source generation would be quick, but we would both need to introduce some new source generation library, and it's a bit unclear if it works when the event definitions are in referenced Nuget packages only.

With both solutions, we would still need some work. Renovate & deploy of Auditing would still need to happen, and for events in a new contract the Nuget reference would need to be added manually. All together, it's not clear that it would be a big enough improvement of the current situation, and we left it at that for now.

Replace Automapper with Mapperly: We have a small "proof-of-concept" of using Mapperly instead of Automapper in the Ledger area.

Outside the technical merits of Mapperly, JD had a strong comment on the process, "We must bring usage of new tech (like replacement of Nuget package) for discussion and decision before we start using them."

There will be a followup discussion with all the backend related to JDs comment, what/how to think about introduction of new dependencies.

Regarding the technical case: Automapper works by reflection in runtime, which comes with a performance cost and a chance of runtime errors. Mapperly (and other source generation based tools) works by generating mappings from class A to class B during compile time, meaning its possible to inspect the resulting mapping, and have no performance penalty during runtime.

Due to time we did not discuss it further, but it would be very useful to get comments for other developers!

Denied transactions in backend db: JD has a strong feeling that we should store failed / denied transactions somewhere in Backend, that it is the responsibility of Backend. A couple of use cases followed, where either our users or our support want to be able to see failed transactions (i.e. a remittance denied by risk or an MPay denied because user is blocked). It would be useful to see these in Hydra on the user. It could also be useful for the users to see these themselves in the app.

Both Roman and Shoiab have previously been strongly against storing this "useless" data.

Shakib first said it, and we could agree, that this is not really different from other Product requests. If Product has a reasonable feature like this, we should not say no, but we should think about what is the best way to do it and how much effort it is.

Related to visibility into risk decisions, Liangxiong had a useful idea. There should be a OwnerResourceId meta field sent in all risk decisions which contains the ID of the transaction / user / etc which the decision is about that would then be included in the event sent to DW. If we had this, it would be easier to know exactly what the decision concerned.