Non-invasive logging of database changes (almost)

Vacation time is passing. People are coming back from their holidays. Finally there's a chance to get an interesting task.

I've reached a point at work where the main tasks I receive are repetitive functionalities. I rarely get the chance to use something new. I'm not a fan of using new libraries. I don't want to build up dependencies.

Recently I was asked to build an internal system for logging changes to our users. Nothing surprising in itself, but it gave me the opportunity to use two things that, by some miracle, I hadn't had the chance to use yet.

I'm talking about DbContext interceptors and Channel. I already knew both of these before — it's just that none of the functionalities I'd written up to that point happened to call for them.

Moving on to the task: write a mechanism that sends user data to another system on every change. For various reasons, we didn't want to give another team access to our database. The idea was conscious but automatic data sending — a mechanism that, once implemented, wouldn't require any secret knowledge to know it needs to be used in specific places in the code.

This is where the first novelty in my code comes in: DbContext interceptors.

The final code was slightly larger since it also checked additional conditions that aren't relevant to the article.

The most important thing here, besides the interception mechanism itself, is that instead of sending the data right away, we decided to save it to a Channel. We wanted to avoid having a significant impact on saving data to the database. During testing of communication with the external service, we ran into Timeout cases that could seriously disrupt our application's operation if processing were synchronous. The mechanism had to be written in a way that wouldn't disturb any of the existing functionalities.

Using a Channel also has its downsides. Messages can always overflow and we could lose data. However, based on our estimates this is unlikely, and — most importantly — we accept that we might not receive all the changes.

There are many ways to configure a Channel. We chose the option where, once it's full, the newest messages won't be added. This is fully acceptable to us.

As for handling the items that were added to the queue:

In the real application, we had considerably more logic here. There was communication with Kafka, checking the configuration itself to see whether we should log changes, as well as checking certain entity dependencies in the database.

I didn't expect such a small piece of functionality to bring me so much enjoyment to implement. It's worth sometimes stepping away from familiar and predictable tasks.