Moving a file between two devices of your own should be the easiest thing a computer does. With Glide you open a page on one, scan its code with the other, and the file goes across: no cable, no account on the second device, no upload to wait for first. This post is about what happens between the scan and the file landing, and in particular how the receiving device knows that what it wrote is what was sent.
The short version: the file is cut into blocks, every block has a fingerprint, and each one is checked the moment it arrives. The longer version is worth reading for the places where a browser gets in the way, because those shaped the design more than anything else.
Where does the file actually go?
From one browser to the other, and nowhere else. Nemi's server introduces the two devices, which is what scanning the code sets in motion, and after that it is out of the picture. The file is read from the sending device a little at a time and written on the receiving one as it arrives, which is why a live transfer has no size limit: there is no server in the middle for the file to be too large for.
Some networks refuse a direct connection between two devices. Then a relay passes the traffic along. It forwards encrypted data it cannot read and keeps none of it. The connection between the two browsers is encrypted either way, by the browsers themselves.
Why cut a file into blocks at all?
Because a check that only happens at the end is a check that fails late. The simple way to send a file is as one long stream, in pieces of 16 KiB, with a single fingerprint of the whole thing compared when the last piece is in. Glide still does exactly that with any device that does not speak the newer way, and it works. But if one piece was damaged on the way, the only thing the receiver learns is that the file as a whole is wrong, after all of it has been sent.
So before the first byte moves, the sender walks through the file and cuts it into blocks of at least 64 KiB and at most 1 MiB, 256 KiB on average, which makes about 4,096 blocks for a gigabyte. Each block gets a fingerprint (a BLAKE3 hash), and the list of them goes to the receiver first. From then on the receiver knows what every part of the file must look like before it has any of it.
One damaged block costs one block
Seven blocks, none the same size, because the content chose the cuts. Six matched their fingerprint on arrival. The darker one did not, so it alone is asked for again.
Why does the content decide where the cuts fall?
The obvious cut is every so many bytes. The trouble with it is that it depends on where data sits rather than on what it is: put one extra byte at the front of a file and every block after it is different. Glide cuts by content instead. It slides over the bytes and cuts where a pattern in the data itself says so, so the same stretch of data is cut into the same blocks wherever it appears.
That pays off straight away within a single transfer. Two files that share a long stretch, or one file that repeats itself, produce blocks with the same fingerprint, and a block the receiver has already been sent is not sent a second time. The receiver writes it in both places from the one copy.
Who decides what is sent next?
The receiver does. It holds the list, so it asks for blocks by number, 32 at a time, and the sender answers. Each block that arrives is hashed and compared with its entry in the list. A match is written to the file. A mismatch is thrown away and asked for again, and only that block: everything already written stays written. A block that arrives damaged 3 times ends the transfer with an error, because by then something is wrong that asking again will not fix.
A file that finishes is a file whose every block was checked on the way in.
256 KiB
the average block
1 MiB
the largest a block can be
32
blocks asked for at a time
3
tries for one block before giving up
What does a browser stop it from doing?
Two things, and both are worth being plain about. A browser writes a download from front to back and cannot jump around inside it. So the receiver asks for blocks in the order of the file, and the few that arrive ahead of their turn wait in memory, 64 MiB of them at most, until the ones before them are in.
And a browser cannot read back a download it has partly written. That rules out picking up a transfer that was cut off, because there is nothing on the receiving side to compare with when it starts again. A damaged block is repaired while the connection is up; a transfer that loses its connection starts over. A program installed on a computer has neither limit, and that is the other half of this story.
What if one device does not know the new way?
Then nothing changes for it. Every batch of files starts with an offer: the sender says it can send in blocks, and the receiver either takes the offer or answers the way it always has. A plain answer means the plain stream, with its one fingerprint at the end. Neither side has to be up to date for a transfer to work, and nobody is asked to choose.
If you have never used it: open Glide, choose to send from this device, drop your files and scan the code with the other one. The steps are in moving a file between two devices. The identifiers in this post, such as BLAKE3, are the names of public algorithms, and the numbers are the engine's own constants.
Questions people ask
How does Glide know a file arrived intact?
The sender lists every block of the file with its fingerprint before the first byte moves. The receiver computes the fingerprint of each block as it lands and compares the two. A block that does not match is thrown away and asked for again, so a file that finishes is a file whose every block was checked.
Does a file sent with Glide pass through Nemi's servers?
No. The two browsers connect directly and the file travels between them. Nemi's server only introduces the two devices to each other. Where a network refuses a direct connection, a relay forwards the encrypted traffic, which it cannot read and does not keep.
What is content defined chunking?
A way of cutting a file into blocks where the content decides where each cut falls, instead of a fixed count of bytes. The same stretch of data is then cut the same way wherever it sits in a file, so identical blocks can be recognised and sent only once.
Can Glide resume a transfer that was interrupted?
Not in a browser. A browser cannot read back a download it has partly written, so there is nothing to compare against when the transfer starts again. A block that arrives damaged while the connection is up is fetched again by itself, but a transfer that is cut off starts over.
Is there a size limit on a Glide transfer?
A live transfer between two devices has none, because the file never rests on a server. It is read from the sending device a little at a time and written to the receiving one as it arrives, so memory use stays small whatever the size of the file.
Where the numbers come from
- The Glide engine: block sizes, request window, retries
glide-web/src/lib.rs - The largest block and the block encryption
github.com/GuusLab/Glide, crates/glide-core - The engine against the native program, both directions
glide-web/tests/interop.rs - How a transfer chooses between the two ways of sending
lib/glide/transfer.ts - The stream every version understands, and its chunk size
lib/glide/protocol.ts - Where the browser writes a download
lib/glide/storage.ts, public/glide-download-sw.js