Handling Failures in Message Driven Architecture

Great video! I have a question - What is the usual process of continuing with deadlettered messages? Are these messages scheduled by the queue after X amount of time? Would the consumer actively try to re-query these messages once it knows that is is now capable of processing them?

I was literally sitting down to create two queues for processing purchases in my on going project and this video dropped. What a timing! I always thought of just retrying or burying the message to the back of the queue or just creating a new queue for failed messages. Excellent video once again. 👌

Nice video!
Do you have some vision or any draft of how potentially Distributed Circuit Breaker could be?
Another service between the multiple instances and external services? Or any framework/library already doing such a thing?
Thanks!

Appreciate this video, thanks. Doing a lot more messaging in my latest role so finding your videos extremely useful 👍

I'm notice you use Kafka in many of your videos. Have you got a blog post/video on why or how you'd choose between Kafka/Service bus/Event grid? I'd be interested to learn about that

Very clear, I like the way you explain, keep it up 👍

Funny enough, I presented message driven architecture in my company's knowledge sharing session 2 days ago. Had you upload the video sooner, I could've just show your video and just eat popcorn.

A trick for an exponential back off that doesn't hang up your worker node that I did was using multiple queues as a sort of psuedo dead letter queues.

1. Create a few queues and set the working queue as their dead-letter queue.
2. Set different message TTL based on each back-off period on each queue. So, when a message is expired, they will be requeue in the workin queue.
3. When a fail message on a consumer node, the node will increment nuimber of retries in the message header, then throw the message to the psuedo dead letter queue. ie: retries = 1, send it to the queue 1.
4. After the message is published to a psuedo dead letter queue, the consumer will ACK message back to the working queue toT remove the message.

You need multiple queues with different TTL instead of setting TTL on each message, because TTL on each message will be taken into account only when the message is at the head of a queue. So, a message with short TTL have to wait for a prior message with a longer TTL to be expired first. A head of the line issue.

Nice! This kicks off the day beautifully here in Denmark :)
Especially: Emphacising business involvement in how we should handle errors and the hold-up caused by retries within handling of a specific message.
Please, please keep it up! I love it! :)

Really helpfull, as always! Thanks for the video!

Like before watching, thank you!

Glad i found your channel. Really advance concepts in such a nice and crisp video.

Thank you for your videos!

I like the idea of loosely coupled monolith but I think in real world too few projects starts with this approach. Could you share your thought about strategies how to move out from coupled monolith when it is live on production? Have you had such cases in your practice?

jesus man, you just keep making super interesting videos! keep it going! and thanks!

What would you do if you have a client that sent the initial command that started the flow asynchronously, but needs to know if a failure happened? would you have the client consuming from the deadletter queue to check if there's a matching message? or would you send a MessageFailed msg to the message broker yourself and have the client consuming those?

you have the best, most consumable videos on very complex topics. I love this channel. I'm shocked you are not at 1mil followers yet.

I can spin up multiple projections services (same projection but multiple instances in the kubenetes for mutiple processing) for one persistant subscription from the eventstore?

Hi, great content on this channel. I would like to know if there is a demo in code about how to build a dead letter queue mechanism, if I become a member could I find this demo?

Hi where I can see practicle implementation..?

Oh really I want to say thank you. You make the videos along with documents that very clear and more professional. The topics are very interesting nowadays. The big thanks for you pls keep it up!!!