Hacker Newsnew | past | comments | ask | show | jobs | submit | citguru's commentslogin

Nothing is built yet. Before I spend time building it, I want the idea roasted.

LaunchPort: launch your side project and get real customers without incorporating first. You'd keep the code, IP, and customers. Infra, payments, sales/marketing and legal handled for you; you "graduate" into your own company whenever you're ready.

Do you get what this does? Would you actually sign up? What would stop you?

Landing page for context: https://launchport.lovable.app/


I needed to back up terabytes of Postgres RDS data to R2 without intermediate local dumps or AWS egress fees, I could not find any ideal solution, so I built this.


Hi,

DuckLake is great for the lakehouse layer and it's what we use in production. But there's a gap and thats what I'm trying to address with OpenDuck. DuckLake do solve concurrent access at the lakehouse/catalog level and table management.

But the moment you need to fall back to DuckDB's own compute for things DuckLake doesn't support yet, you're back to a single .duckdb file with exclusive locking. One process writes, nobody else reads.

OpenDuck sits at a different layer. It intercepts DuckDB's file I/O and replaces it with a differential storage engine which is append-only layers with snapshot isolation.


The project is still fairly new and not close to production tbh.

I'd actually recommend the simplest option: just write them to Parquet on S3 and query with plain DuckDB. Or you could use Ducklake - https://ducklake.select/


Thanks for this, really enjoyed reading this and helps validate some of my personal thoughts


OT but: You joined in 2019, barely post anything, then suddenly in 2026 your comments are copy pasted LLM output. Why? Why don't you use your own voice and type with your own hands? Notice how all those copy pasta posts were nuked - for good reason - we don't like being insulted.


You joined in 2017, barely post anything, then suddenly in 2025/2026 2/3 of your posts are copy pasted links, 1 of which is dead and another is 10 years old. Why? Why don’t you use your own voice and type with your own hands? Why don’t you post something new and relevant that you made instead of attacking people who are posting entire code repos of interesting technology?


I call it suspicious activity.


touché


Yes, this is actually one of the core problems OpenDuck's architecture addresses.

The short version: OpenDuck interposes a differential storage layer between DuckDB and the underlying file. DuckDB still sees a normal file (via FUSE on Linux or an in-process FileSystem on any platform), but underneath, writes go to append-only layers and reads are resolved by overlaying those layers newest-first. Sealing a layer creates an immutable snapshot.

This gives you:

Many concurrent readers: each reader opens a snapshot, which is a frozen, consistent view of the database. They don't touch the writer's active layer at all. No locks contended.

One serialized write path: multiple clients can submit writes, but they're ordered through a single gateway/primary rather than racing on the same file. This is intentional: DuckDB's storage engine was never designed for multi-process byte-level writes, and pretending otherwise leads to corruption. Instead, OpenDuck serializes mutations at a higher level and gives you safe concurrency via snapshots.

So for your specific scenario — one process writing while you want to quickly inspect or query the DB from the CLI — you'd be able to open a read-only snapshot mount (or attach with ?snapshot=<uuid>) from a second process and query freely. The writer keeps going, new snapshots appear as checkpoints seal, and readers can pick up the latest snapshot whenever they're ready.

It's not unconstrained multi-writer OLTP (that's an explicit non-goal), but it does solve the "I literally cannot even read the database while another process has it open" problem that makes DuckDB painful in practice.


This is an attempt to replicate MotherDucks differential storage and implement hybrid query execution on DuckDB


As someone working in the field I have to admit that I'm not familiar with the terms differential storage nor do I really understand what hybrid execution means. Maybe you could describe it both from a simple technical point of view and what benefits it has to me as a user?


from the post:

Differential storage

Append-only layers with PostgreSQL metadata. DuckDB sees a normal file; OpenDuck persists data as immutable sealed layers addressable from object storage. Snapshots give you consistent reads. One serialized write path, many concurrent readers.

Hybrid (dual) execution

A single query can run partly on your machine and partly on a remote worker. The gateway splits the plan, labels each operator LOCAL or REMOTE, and inserts bridge operators at the boundaries. Only intermediate results cross the wire.


Already started porting some of the services, but alot are tightly coupled with AWS services


10 hours is too much time. Time is money, we deal with time sensitive business. If people are not able to process transactions how are their users able to do their daily activities. Money was definitely lost once the Finance team analyse the situation


You're not paying for the support you're expecting. This was an oversight on your part to vent on HN instead of upgrading to the Business tier the past however many hours your systems have been down. You can face the same suspension issue with any cloud provider. A painful, but necessary lesson it seems.


Developer Plan


Well I mean.... there's your problem right there. You're paying for a 12 hour response time, on a plan the AWS has told you fairly explicitly is NOT for production or mission critical workloads.

I get that it's frustrating that they try and nickel and dime you on support costs, but at the end of the day you do get what you pay for. I've used Business and Enterprise support in the past and always had very good support from them delivered very quickly.


I'd start by going to business plan and trying again.


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: