Edge assistant
Compact private assistant that does not send sensitive prompts to a third-party API.
Open Release · Apache 2.0
Cassandra T1
A masked-diffusion language model that generates the sequence as a field, not a line.
Cassandra T1 is a real, open-released SophiaXT prototype. The repository ships a verified 2.66 GB epoch-5 FP16 checkpoint, the tokenizer, the PyTorch architecture, training utilities, and a Flask chat server — under Apache 2.0. It is a proof-of-concept release: the architecture and training path work; output quality still needs more training and evaluation before production use.
Repository · Chorozion/Casandra-t1-diffusion-edge-model · License: Apache 2.0
28
Transformer layers
2,048
Hidden size
32,768
Vocabulary
2.66 GB
EP-5 FP16 weights
v1.5 · continued pretrain · anchor-token-masked
Cassandra T1.559× corpus-overlap improvement over base v1.
T1.5 is a continued-pretrain on top of T1 epoch-5, running 500 steps of anchor-token-masked training over LTMi-XT bundles (the validated methodology behind our 1.67× OOD generalization advantage). On the held-out diagnostic eval, pooled corpus overlap rises from 0.008 → 0.481 — base v1 emits English-shaped gibberish; v1.5 emits coherent on-corpus content.
base · v1 epoch-5
0.008
corpus overlap (pooled)
v1.5 · continued step-500
0.481
corpus overlap (pooled)
step-1000 (overtrained)
0.440
slight regression — diminishing returns
Use T1.5 from lens-xt
# pip install lens-xt[local]
from lens_xt import LensX
lens = LensX("specs/medical_basic.lensx") # backend=cassandra-t1.5 by default
print(lens.run(user_input="What's the standard dose?"))SHA-256 · d75d404cec050931ca75362e50229a668b69a7bf1cf4dd84c90ec3e7babd8618 · source: cassandra-t1.5-eval.json · methodology: anchor-token-masking paper
Released artifacts
Real weights. Real tokenizer. Verified by hash.
Both checkpoints are split into numbered Git LFS parts because GitHub rejects single LFS objects above 2 GB. Reassembled hashes match weights/checksums.sha256.
cassandra_ep5_fp16.ptVerified epoch-5 Cassandra T1 FP16 checkpoint
Reassembled size
2,659,500,664 bytes
2 LFS parts
SHA-256
D70C813C513F5232A25313FA60338F862020BA942ED26D54F62511766FA5F044v2_scratch_epoch2_82002.ptNewer v2 scratch checkpoint (step 82,002) — included for transparency
Reassembled size
16,665,974,950 bytes
9 LFS parts
SHA-256
8BFB5644209AB8FD241D85A725781495B29922811A2BECF8260AAAAD0A26DA6Frelease/tokenizer.jsonCassandra tokenizer used by released scripts
Reassembled size
2,253,607 bytes
Single file
SHA-256
376A9537FCE79B7004237845E6B2C9991661E6BAEEA0B76AF9C9A3C1EB405C4DArchitecture
T1-Base — compact, GQA, masked-diffusion-trained.
Cassandra T1 is a 28-layer transformer with grouped-query attention (16 query / 4 KV heads), SwiGLU feed-forwards, RoPE position encoding, and RMSNorm. Default inference uses 12 denoising steps over a masked output span — instead of committing each token strictly left-to-right, the model reveals tokens across the whole span in parallel passes.
The architecture is declared explicitly in src/model/config.py. The reference forward pass and decoder live in src/model/sophia_t1.py. The PDE-inspired mask schedule lives in src/scheduler/pde_lattice.py.
T1-Base · default configuration
| Vocabulary | 32,768 tokens |
| Hidden size | 2,048 |
| Transformer layers | 28 |
| Query heads | 16 |
| KV heads | 4 |
| FFN intermediate size | 5,632 |
| Position encoding | RoPE |
| Normalization | RMSNorm |
| Feed-forward | SwiGLU |
| Mask token id | 32,766 |
| Pad token id | 32,767 |
| Default denoise steps | 12 |
Parameter count, from the table above
Twenty-eight layers of grouped-query attention and a SwiGLU feed-forward at these widths come to 45,092,864 parameters each, or 1,262,602,240 for the stack. A tied 32,768 by 2,048 embedding adds 67,108,864, giving 1,329,711,104. The released half-precision checkpoint is 2,659,500,664 bytes, and two bytes to the parameter makes that 1,329,750,332. The two disagree by 39,228, which is the serialization header, so the model is 1.33 billion parameters and both published artifacts say so.
Training measurements
Five recorded epochs. 3.78 → 2.26 loss.
Loss values are recorded directly from the training notes for the epoch-5 line. They show the trajectory; they are not a public benchmark, and we do not invent benchmark numbers we have not measured publicly.
Qualitative notes from the source project: epoch 5 produces better short factual answers than earlier checkpoints, while long-form generation remains unstable. The newer v2 scratch checkpoint is larger by file size but currently shows worse coherence under the captured 10-prompt test.
Captured evaluation
We publish what the model actually does — including where it fails.
The repo includes eval/v2-ep2-vs-t1-ep5-coherence-raw.txt with the raw output captured under a fixed 10-prompt set. This is a development comparison, not a benchmark — but it is a real one.
Captured comparison · 10 prompts
T1 epoch-5 vs v2 scratch epoch-2
T1 epoch-5
Outputs are still fragmented and contain corrupted token pieces — but the model samples a wider vocabulary and surfaces domain-relevant fragments (appliance, technical, architecture, denoising, Sophia). Distributional structure is forming; instruction alignment is not yet stable.
V2 scratch epoch-2
Severe repetition collapse. Output dominated by left right left right scaffolding tokens across most prompts. Newer by timestamp does not mean better — checkpoint selection should be evaluation-driven.
Prompt set used in the captured test
01 · Who are you? 02 · Are you ChatGPT? 03 · What company built you? 04 · How do you generate text? 05 · Refrigerator humming loudly — what could be wrong? 06 · Washing machine won't drain, what should I check? 07 · Dryer won't heat up, what's wrong? 08 · Explain quantum entanglement simply. 09 · Write a short paragraph about the ocean. 10 · What is your architecture?
What's in the repo
Source tree, one line per file.
src/model/sophia_t1.py PyTorch transformer architecture (28 layers, GQA, SwiGLU, RoPE, RMSNorm).src/model/config.py Typed configuration object — every architecture parameter declared explicitly.src/model/spatial_tokens.py Spatial-token utilities for the next training cycle.src/scheduler/pde_lattice.py PDE-inspired lattice for masked-token revealing order.src/scheduler/pde_scheduler.py Confidence-and-schedule decoder for parallel diffusion sampling.src/train/pretrain_from_scratch.py Scratch pre-training loop with the masked-diffusion objective.src/train/continue_from_ep5.py Continuation training entry-point — resume from the verified epoch-5 checkpoint.src/train/train_diffusion.py Diffusion-style training utilities used by both scratch and continuation runs.src/train/train_qlora.py QLoRA fine-tuning experiment harness.src/train/merge_lora.py Merge LoRA adapters back into base weights.scripts/run_cassandra.py Local interactive CUDA inference for the epoch-5 checkpoint.scripts/chunk_gen.py Chunk-based generation experiment.scripts/serve_ep5.py Flask server exposing an OpenAI-style chat completion endpoint.src/eval/canary_identity.py Canary-style identity / regression check helper.eval/v2-ep2-vs-t1-ep5-coherence-raw.txt Captured 10-prompt comparison output between the two checkpoints. quickstart.sh
# Reassemble the verified epoch-5 checkpoint from the LFS parts
cat weights/cassandra_ep5_fp16.pt.part00? > cassandra_ep5_fp16.pt
sha256sum -c weights/checksums.sha256
# Local interactive inference (CUDA)
python scripts/run_cassandra.py \
--checkpoint cassandra_ep5_fp16.pt \
--tokenizer release/tokenizer.json \
--denoise-steps 12
# Or stand up the Flask chat endpoint
python scripts/serve_ep5.py --port 8080Heads up
Inference scripts retain a few original development-machine path assumptions. They are useful for technical review and local adaptation. The release is not yet packaged as a clean installable runtime, and there is no quantized GGUF artifact yet — the custom architecture isn't currently supported by the standard GGUF conversion path.
Direction of travel
What the architecture is built toward.
Cassandra T1 is the first checkpoint in a SophiaXT model line aimed at compact, edge-deployable diffusion-style generation for technical service operations. These are intended use cases — they describe direction, not a current product claim.
Document QA
Spatial-token aware so document position, tables, receipts, and page structure are preserved.
Routing logic
Summarize logs, classify next action, route operational cases to the right workflow.
Private code help
Explain and repair code locally so proprietary snippets stay on the machine.
Limitations · stated up front
What this release is not.
Not production-ready. Output quality still requires more training and structured evaluation.
Long-form generation is unstable — short factual answers are more reliable.
No formal public benchmark suite is included yet. We refuse to publish numbers we have not measured.
Training datasets are excluded for safety and licensing reasons.
No quantized GGUF artifact yet — the custom architecture isn't supported by the standard conversion path.
Inference scripts retain dev-machine path assumptions; clean packaged runtime is future work.
Roadmap
What the next training cycle is for.
Stability, identity consistency, instruction following, and evaluation — not just adding more checkpoint files.
01 Continue from the most stable checkpoint, not automatically from the newest one.
02 Increase identity and instruction-following data quality.
03 Add structured evaluation prompts: appliance/service reasoning, general QA, code, spatial layout, refusal/safety.
04 Integrate spatial tokens into the active data path before claiming native spatial behavior.
05 Track loss, output samples, latency, memory, and regression prompts per checkpoint.
06 Publish a per-checkpoint model card with hashes, training-data summary, decoding settings, known failure modes.
Cassandra T1 is part of the SOPHIA XT model family.
Inspect the public repo, verify the checksums, run the inference scripts, or read the architecture rationale. Everything is real, everything is licensed Apache 2.0, and we are honest about what works and what doesn't.