How would horizontal partition work by storing different rows in different tables? Did they write code to do that? Or is it something postgres offer? How does read work if data has to joined or filtered or sorted?
How would horizontal partition work by storing different rows in different tables? Did they write code to do that? Or is it something postgres offer? How does read work if data has to joined or filtered or sorted?
The proxy server is a Golang service. It's where the partitioning logic is implemented, as I understood it. (Postgres seems to have native support for partitioning, along with some extensions available.)
They use the 'scatter-gather pattern' if the data is spread across many databases.
So many databases will be queried in parallel, and the result is combined at the app layer.
There's a performance tradeoff by querying many databases, but it happens rarely with Figma's query patterns.
How would horizontal partition work by storing different rows in different tables? Did they write code to do that? Or is it something postgres offer? How does read work if data has to joined or filtered or sorted?
The proxy server is a Golang service. It's where the partitioning logic is implemented, as I understood it. (Postgres seems to have native support for partitioning, along with some extensions available.)
They use the 'scatter-gather pattern' if the data is spread across many databases.
So many databases will be queried in parallel, and the result is combined at the app layer.
There's a performance tradeoff by querying many databases, but it happens rarely with Figma's query patterns.
As long as it is designed in such a way that requires fewer joins then performance won't be a bottleneck :)
Did they do manual sharding in code or sharding using citus which is extension on postgres?