Skip to content

Arch Forum 2024-09-19

Participants: Backend devs, Magnus, Zak, Andy and Victor

Agenda

  • Removal of dev!
  • Idempotency for Transactions
  • TEKM, ERR, Warnings & Deadletters

Notes

Removal of dev! A quick check on the removal of the dev deploy and dev regression tests from the pipelines: No-one miss it so far.

Idempotency for Transactions: Transactions area have now implemented idempotency for managed transactions.

These are the guidelines:
1. A call to create a managed transaction should include a TransactionId (will be required in the future)
2. The TransactionId doubles as idempotency key. That means that it is safe to call Transactions with the same TransactionId multiple times:
a. A second call with the same TransactionId will return a successful result in case the transaction was created successfully on the first call.
b. A second call with the same TransactionId will return a successful result in case the first time failed, but now it succeeded.
c. In both these scenarios, a single transaction was created.
3. Note that each new transaction should have a new TransactionId. I.e. if a caller want to first debit and then credit in the same "flow" it can bind the transactions together with OwnerResourceId (and not TransactionId which must be unique).
4. To know the final result of a transaction that first returned a failed response, it's possible to listen to the TransactionCompletedEvent, which will be sent out when we know for sure the transaction happened.
5. Given the above it is up to the called to decide when a TransactionId is needed

In CashDeposit there's logic implemented with these things in mind, including a reverse in case the transaction did not happen successfully.

TEKM, ERR, Warnings & Deadletters:
An overview and discussion of the log levels are in the wiki, at Logging (was updated after this discussion)

Additional notes / questions:
- Would it be good with a TEKM summary in slack every day, like the deadletter one?
- Sometimes a process might be very important even when it's only run once. In such a case, only one TEKM might be produced on error. These cases will need special handling.
- Don't we also have a debug log level? Yes we do, but by default its disabled.
- Can we increase retention of the info logs? Since they are not a big part of the logs it would probably be ok to use the same retention for those as other logs.
- Could it be good with some alerts for high levels of WARN logs?
- Deadletters are not mainly intended for alerting. Meaning code that log TEKM does not need to also create a deadletter in case the deadletter is impossible to retry. In such cases the TEKM is enough to indicate the problem.