Read the whole document
on the phone. Copy from it.
SRW Pocket is the Sophia Reader-Writer. A large reader takes in up to 35K tokens of context once. A small writer produces the answer and can copy tokens only from spans of that context. The whole model is 431M parameters and fits in 291 MB on a phone with int4 experts.
The lease starts on 1 March 2027 and runs for 36 months. Rent is $4,850 a month, due on the first. The deposit is held by Harlow Property Group.
The lease runs
Most of the model
is the reader.
The reader is a 16-layer mixture of experts, and its experts are stored at int4, which is how the phone footprint reaches 291 MB. The writer is four dense layers with cross-attention into the reader's output, so it writes with the document in view and does not carry it.
The copy head is restricted to context spans. A name, a date or an amount in the answer is either generated from the vocabulary or copied, and a copy can only point at a span that exists in the context.
SRW Pocket is in build. There are no benchmark numbers for it on this site yet, and none will be printed before there is a run to point at.