Hi Raul! Thanks for sharing. I have a question though. Isn't relying on the front-end too error-prone? What if the browser just lose connection after the first api call to store the video. Shouldn't the storage in File Storage trigger the message to the work queue? Have you explored other options? Excited to hear about the whys about this design decision. Thanks again for such a great content
For this specific example, I used API-Based Workflow Management; after uploading the file, the API creates the task and pushes it to the queue. Mainly because I need the collected metadata to be sent as part of the message.
The File Storage trigger is another good solution to trigger the process. We just need to be careful since there’s a risk of a race condition where the file gets uploaded, and the event is triggered for processing before the backend has fully saved the metadata.
Great step-by-step description Raul.
Queues are such a versatile data structure. Also, with the competing consumer pattern, they can really help balance the load as well.
Also, thanks for the mention!
Excellent point, Saurabh.
I think the competing consumer pattern is one of the great benefits of using queues.
Hi Raul! Thanks for sharing. I have a question though. Isn't relying on the front-end too error-prone? What if the browser just lose connection after the first api call to store the video. Shouldn't the storage in File Storage trigger the message to the work queue? Have you explored other options? Excited to hear about the whys about this design decision. Thanks again for such a great content
Hi David, that's an excellent question.
For this specific example, I used API-Based Workflow Management; after uploading the file, the API creates the task and pushes it to the queue. Mainly because I need the collected metadata to be sent as part of the message.
The File Storage trigger is another good solution to trigger the process. We just need to be careful since there’s a risk of a race condition where the file gets uploaded, and the event is triggered for processing before the backend has fully saved the metadata.
Thanks!
Great article Raul!
Loved the Video Thumbnail generation example.
Thanks for the feedback, Ashish!!!
Always learn something new from your articles, Raul!
Thanks so much, Gina!!!