Discussion about this post

User's avatar
Jade Wilson's avatar

Great walkthrough. I really like the visuals and the way you structure you sentences to have breaks between them.

Just reading between the lines, the way they do data redundancy seems costly, I know they say it's a cheap solution but would be good if you have any insights on how much that costs to have such high durability this way?

Expand full comment
Rushik's avatar

"Besides they use the HTTP trailer to send checksum.

Because it allows sending extra data at the end of chunked data.

Thus avoiding the need to scan data twice and check data integrity at scale."

What do we mean by 'avoiding the need to scan data twice'? Why do need to scan data twice to calculate checksum? I don't quite get it. Can someone explain please?

Expand full comment
12 more comments...