Curious to know experience of people using DBOS and Temporal.
I have used Temporal in the past, works really good, my only problem with it was some limits on request payload or event sizes, created some inconveniences to us when building solutions. It also enforces good engineering practices, but sometimes you don't want to write special logic if your CSV file is larger than 2Mb, upload it to S3, pass link, then download it in the workflow.
What is your experience with DBOS? How does it compare to Temporal in terms of operational complexity, feature parity and anything else
we're using dbos for ai gen workflows and processing video files. understanding how to migrate from celery took time, but for our case it was worth it.
They've just released an external storage approach to solve the large payload issue. I don't 100% love it (it's bolted on, not an intrinsic part), and it's an early release right now - but you can consider this effectively solved for now.
That's good because back in the day if you were putting entire documents in a message queue I would laugh people out the door, putting something in object storage + linking is much more useful (though the distributed system part/backup current state part can be annoying!)
I feel it's way too hand wavy on consistency and correctness. My opinion as someone who've implemented marketing workflows that breaks all the time (and tons of painful lessons).
Strong correctness guarantee is something that should not be undermine. Even more important than availability.
The examples on the website is simple but heavily undermines the importance of correctness. Anyone who implement similar pseudo-code directly will eventually suffer from data correctness issue in crashes.
Continuously amazed by what you can do with few tools, as long as Postgres is a part of your toolkit.
I recently developed a distributed queue and it works really great - benchmarks great too, with no race conditions or conflicts. I used SKIP LOCKED so that workers can compete safely.
You can also have multiple workers across nodes avoid conflict by using session wide mutexes i.e. pg advisory lock.
Having inherited a few of these - you tend to home-grow an ad-hoc version of many of the existing OSS tools, but with less of the patterns baked in.
Not sure where the NIH ends and where you're actually better off with a supported orchestration approach. I suppose if you expect your program to be around a while (or need advanced features), maybe think about using something a bit more battle tested?
Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable. It has been quite stable and extensible but of course you need to be very careful with the SQL implementations. Hope this is interesting to readers here.
The efforts we've undergone to make Oban (and Pro) work with CRDB have been ridiculous. Feature detection all over because of a lack of common operators and functions that can't be used in indexes. The worst is the rampant "serialization_failure" errors that force continual transaction retries. Not how I'd suggest scaling Postgres.
That said, as a predecessor to dbos in building durable workflows just using Postgres, I concur with the overall sentiment.
Curious to know experience of people using DBOS and Temporal.
I have used Temporal in the past, works really good, my only problem with it was some limits on request payload or event sizes, created some inconveniences to us when building solutions. It also enforces good engineering practices, but sometimes you don't want to write special logic if your CSV file is larger than 2Mb, upload it to S3, pass link, then download it in the workflow.
What is your experience with DBOS? How does it compare to Temporal in terms of operational complexity, feature parity and anything else
we're using dbos for ai gen workflows and processing video files. understanding how to migrate from celery took time, but for our case it was worth it.
They've just released an external storage approach to solve the large payload issue. I don't 100% love it (it's bolted on, not an intrinsic part), and it's an early release right now - but you can consider this effectively solved for now.
That's good because back in the day if you were putting entire documents in a message queue I would laugh people out the door, putting something in object storage + linking is much more useful (though the distributed system part/backup current state part can be annoying!)
I feel it's way too hand wavy on consistency and correctness. My opinion as someone who've implemented marketing workflows that breaks all the time (and tons of painful lessons).
Strong correctness guarantee is something that should not be undermine. Even more important than availability.
The examples on the website is simple but heavily undermines the importance of correctness. Anyone who implement similar pseudo-code directly will eventually suffer from data correctness issue in crashes.
Continuously amazed by what you can do with few tools, as long as Postgres is a part of your toolkit.
I recently developed a distributed queue and it works really great - benchmarks great too, with no race conditions or conflicts. I used SKIP LOCKED so that workers can compete safely.
You can also have multiple workers across nodes avoid conflict by using session wide mutexes i.e. pg advisory lock.
Having inherited a few of these - you tend to home-grow an ad-hoc version of many of the existing OSS tools, but with less of the patterns baked in.
Not sure where the NIH ends and where you're actually better off with a supported orchestration approach. I suppose if you expect your program to be around a while (or need advanced features), maybe think about using something a bit more battle tested?
Since DBOS doesn't support Rust, we implemented a very minimal Rust version of this at https://github.com/tensorzero/durable. It has been quite stable and extensible but of course you need to be very careful with the SQL implementations. Hope this is interesting to readers here.
How do you incorporate secrets in this kind of implementation? Stored in db?
Citing CockroachDB as an example of scaling Postgres made me spit out coffee. Was this LLM-written?
The efforts we've undergone to make Oban (and Pro) work with CRDB have been ridiculous. Feature detection all over because of a lack of common operators and functions that can't be used in indexes. The worst is the rampant "serialization_failure" errors that force continual transaction retries. Not how I'd suggest scaling Postgres.
That said, as a predecessor to dbos in building durable workflows just using Postgres, I concur with the overall sentiment.
Temporal is an insane piece of software, always surprised people dont know about it. You could replace almost youre whole AWS stack with temporal
PgFlow is pretty awesome for DAG workflows - it's built on pgmq (which does the heavy lifting).
Typescript: https://www.pgflow.dev
Elixir: https://github.com/agoodway/pgflow/blob/main/docs/COMPARISON...