Learner 1.0 learns continually without catastrophic forgetting.
Evolution found a learning algorithm built on biological neurons that can continually learn without catastrophic forgetting. Learner 1.0, a ~28B dense foundation model, achieves the same on silicon. We believe this solves a ~40-year-old problem of online continual learning in the field. Sign up for early API access and read our demo reports.
01 / The problem
Current AI struggles to learn from few examples without forgetting.
Humans learn incrementally from experience. Training today’s AI models on a narrow stream of new data can overwrite earlier knowledge and capabilities.
Most deployed models therefore keep their weights frozen. They can learn within a context window, but that learning is lost when the context resets.
Why continual learning matters02 / Our solution
Learner 1.0 learns from few examples while retaining prior capabilities.
Learner 1.0 is built on new architectural primitives that we believe do for continual adaptation what attention does for long sequence modeling. This allows it to keep learning incrementally and efficiently.
It learns from a handful of examples in our agent experiments, with the learning carried in its weights. Earlier knowledge is remeasured as new material is taught.
The architecture is inspired by a decade spent studying cellular systems and how memory works within them.
See the ten-skill experiment03 / Capabilities & reproducible evidence
Continual learning is demonstrated across skills, facts and agent tasks.
01Continual learning is built into Learner 1.0’s architecture.
Task label free · incremental · replay free · fixed capacity
Continual learning is built into Learner 1.0’s architecture.
Task label free · incremental · replay free · fixed capacity
Learner 1.0 takes an architectural approach to continual learning. The mechanism is proprietary. The measurements are available to inspect.
These requirements must hold together. Our published experiments test them across sequential skills, text domains, factual knowledge and an agent learning from its own experience.
Learning needs no task labels
The model sees the data. It receives no task ID, domain label or signal that a new task has begun.
Each example produces an incremental update
One example, one update, in arrival order. New skills are taught in consecutive blocks.
Earlier examples are not replayed
Earlier examples are not mixed back into later training to preserve what was learned.
Base capability is checked alongside new learning
We evaluate pretrained capabilities alongside acquisition and retention, rather than testing new skills alone.
Trainable capacity stays fixed
Trainable capacity stays fixed through the stream. Published runs use a single GPU. Larger scales remain to be tested.
The complete continual-learning standard
Continual learning claim also requires genuine acquisition, retention measured after later teaches, fixed capacity and efficient use of pretrained priors. Not forgetting is only useful if the model also learns the new material.
The demonstrations use supervised next-token prediction. Teaching narrow streams without replay puts retention under pressure, making this a test of the learning architecture.
02New skills are learned without task labels while earlier capabilities are retained.
10 sequential skills · earlier skills and base capability re-evaluated
New skills are learned without task labels while earlier capabilities are retained.
10 sequential skills · earlier skills and base capability re-evaluated
Ten unrelated skills. One continuous stream. No task labels or replay.
We taught 63,282 examples one at a time, in contiguous skill blocks, and re-evaluated earlier skills after each later teach. Acquired skills remained within two correct answers of their own-end score on 225-question panels.
Methods and measured results
Training uses ordinary supervised next-token prediction: batch size 1, no gradient accumulation, one pass through the skill data and no optimizer resets between tasks. It deliberately avoids interleaving earlier domains. This is a demanding test of retention.
Retention is measured in two complementary ways: exact-match accuracy in the ten-skill report, and held-out next-token loss in nats in the four-domain study. Oriel, a skill that was weakly acquired, showed a small decline. The full report exposes that result.
The base was checked at every checkpoint with a 498-item sentinel covering MMLU, ARC-Challenge, HellaSwag and WinoGrande. At checkpoints 0, 5 and 10, a 16,481-item likelihood battery covered all 57 MMLU subjects, ARC-Challenge and WinoGrande. No material degradation was observed on those evaluations.
LoRA forgot earlier material in both comparisons. In the two invented languages study, its first-language loss rose from 2.22 to 6.00 nats after learning the second language. Learner 1.0 kept that language’s held-out loss near its own-end level, at 2.8871 versus 2.8880 nats. The text and language order matched, but training schedules and retention probes differed. In the four-domain study, capacity-matched LoRA lost 0.03–0.07 nats on earlier domains. Learner 1.0 kept them within ±0.004 nats of their own-end loss on the same windows and domain order.
In the four-domain experiment, some earlier domains improved slightly as later ones trained: positive backward transfer. The effects were small (roughly 0.001–0.004 nats) and each condition was a single run.
One example. One update.
Ten skills → one model → fixed trainable capacity
03Agents learn from a handful of examples and apply that learning to later tasks.
Few-example lessons · schema and procedure reuse · fresh context per question
Agents learn from a handful of examples and apply that learning to later tasks.
Few-example lessons · schema and procedure reuse · fresh context per question
Small lessons carry learning forward in the weights, even when context starts over.
Continual Learning Bench, released by UC Berkeley’s Sky Computing Lab and collaborators, tests whether agents improve across related real-world tasks, using earlier experience to help with later ones. In database exploration, reward measures correct answers and query efficiency. Gain compares that reward with the same model with learning switched off.
In our comparison with its published leaderboard, Learner 1.0 is the only weight-updating method. The compared systems carry experience through context or external memory. Our learner trains after each attempt and starts the next question with fresh context.
The learning loop
- 01
The learner attempts each question.
The learner acts, receives results and submits its answer.
- 02
A teacher turns the attempt into examples.
A teacher reads the attempt and feedback, then writes a handful of grounded training examples.
- 03
Weight updates carry learning forward.
Learner 1.0 trains once on each example before the next question.
No earlier questions, notes or retrieved examples enter the next prompt.
The teacher prepares training data after the attempt. It does not answer benchmark questions for the learner.
Continual Learning Bench / Database exploration / 3 runs
Small lessons teach schema facts and procedures that help with later questions.
A recorded lesson and its later use · Run 0
A small lesson teaches a query procedure used successfully nine questions later.
- 01 · Question 24
The old brand column leads to a wrong answer.
The schema has changed. The learner groups instruments by
items_g3.str_nm, but that column is empty. It guesses Yamaha. The correct answer is Fender. - 02 · Lesson from the attempt
Find the new brand attributes before computing.
The teacher identifies the fact and procedure to learn: inspect the instrument attribute tables for brand information, then calculate the answer using those attributes.
Read the recorded procedure
Musical-instrument brand ranking (brands with at least N distinct products, highest average list price, alphabetical tie-break). The items_g3.str_nm brand column gives no brand with N or more products, so find where brand is recorded (the g3 key-value attribute tables) before computing or answering. Do not switch to other categories' tables or answer with an uncomputed guess.
- 03 · Incremental training
Six examples teach the new-table query.
6 examples · 522 supervised tokensFor this specific query
Six examples in the update after question 24 teach this query. Each is trained once.
See an actual training target
Given the attempt and its query results, predict this next query:
SELECT 'attrs_g3_legacy' AS src, attr_key, COUNT(*) AS n_rows, COUNT(DISTINCT ref_id) AS n_items FROM attrs_g3_legacy GROUP BY attr_key UNION ALL SELECT 'product_attributes_g3', attr_key, COUNT(*), COUNT(DISTINCT ref_id) FROM product_attributes_g3 GROUP BY attr_key ORDER BY n_items DESC LIMIT 50 - 04 · Between questions
The context resets. The weights carry the learning.
Every question starts with fresh context. Eight intervening questions and their updates follow before question 33 tests use of the new attribute table.
- 05 · Question 33
The learner uses the table and answers correctly.
Asked how many instruments have brand information in the current feed but not the old export, it inspects the schema and queries the new table.
1,145 · correct in 4 queriesNine questions after the lesson
Read the successful query
SELECT COUNT(*) AS n FROM product_attributes_g3 WHERE attr_key = 'Brand' AND ref_id NOT IN (SELECT ref_id FROM attrs_g3_legacy WHERE attr_key = 'Brand')
Learning improves later answers and query efficiency.
Swipe the chart to read every label →
All three runs finish with positive gain.
Swipe the chart to read every label →
6.0–13.5 examples per question. Across the three runs, average training dose was 514–1,127 supervised tokens per question. Each example was used once. Context reset before every question, so learning carried between questions had to remain in the weights.
The three runs answered 21, 15 and 16 of 40 questions correctly. The recorded learning-off run answered 7. The example above shows how a small lesson was successfully applied nine questions later.
Training protocol & the other benchmark tasks
One pass, batch size 1, no replay and no optimizer resets between questions. The learner trains on next actions grounded in its own trajectory and actual tool results. Questions are not simply paired with their final answers. The same reasoning-when-stuck rule is used for both the learner and its stateless baseline.
On cohort studies, all three learning runs exceeded their baseline, with mean normalized gain of +4.35%. On sales forecasting (n = 1), Learner scored 8.44 versus 5.76 with learning off and exceeded the baseline in 10 of the 11 years after the first.
04Learner 1.0 learns new facts in its weights and retains them as later lessons are taught.
15/16 handbook answers · no previously correct answers lost across four lessons
Learner 1.0 learns new facts in its weights and retains them as later lessons are taught.
15/16 handbook answers · no previously correct answers lost across four lessons
We teach the learner facts from a document by turning them into practice questions and training on the answers. We then ask questions without supplying the document, retrieved passages or examples. The learner must recall what it learnt from its weights.
Learning new facts. After learning a 706-word fictional company handbook, it answered 15 of 16 questions correctly. The base model answered none.
Retaining earlier lessons. We taught four lessons into one learner, one after another, without replaying earlier lessons. After each lesson, we tested earlier knowledge again. No previously correct answer was lost on either the published quiz or a separate 108-question retention panel.
Preserving base capabilities. A separate 150-question check across MMLU, ARC-Challenge and WinoGrande returned the same accuracy as the base model on each benchmark.
Methods and measured results
Recipe 2 extracts facts using a rule pass and a language-model pass, generates practice questions, and trains over two passes. Examples within a lesson may repeat. Earlier lessons are not replayed during later teaches.
We also taught a fresh learner facts about an invented planet that contradicted what its base model believed. It recalled 6 of the 8 taught facts its base did not hold. All three control questions about Earth stayed correct.
Teach a document / Recipe 2
questions answered correctly
without the document in context
Measured after every lesson on the published quiz and a 108-question panel.
05The learning mechanism works across Qwen, Llama and GPT model families.
Published Qwen results · internal GPT-2 and Llama tests
The learning mechanism works across Qwen, Llama and GPT model families.
Published Qwen results · internal GPT-2 and Llama tests
Designed to compose with a pretrained transformer and its existing strengths, including attention.
The published experiments use an architecturally modified Qwen 3.6-27B. We have also tested the architecture internally in the GPT-2 and Llama 3.1 8B families.
What is demonstrated. What comes next
Those internal tests are a portability observation, not a published head-to-head scaling study. We expect the mechanism to extend to larger models and mixture-of-experts architectures, but that remains work to validate.
Trainable capacity is fixed before each experiment. The reported configurations add 1.14B or 2.28B parameters. This capacity does not grow as the stream continues.
Same learning idea / different foundations
06Pretrained models gain continual learning without full retraining.
25 updates achieved 82–125% of LoRA’s 300-update loss reduction
Pretrained models gain continual learning without full retraining.
25 updates achieved 82–125% of LoRA’s 300-update loss reduction
Retrofitting a pretrained model means adaptation can start from its existing representations, without retraining a foundation model from scratch.
In the four-domain comparison, Learner 1.0 achieved 82–125% of the parameter-matched LoRA’s 300-update loss reduction after just 25 updates, measured from each condition’s recorded base reference.
Methods and measured results
Twenty-five updates is 8.3% of 300. This is a comparison at measured checkpoints, not a claim that every domain reached an identical loss in one-tenth the time.
Learner 1.0 trained stably at a learning rate of 1×10⁻³ in this stream. The tested LoRA configuration ran at 2×10⁻⁵ after a 2×10⁻⁴ trial diverged. Two LoRA rates were tested, not an exhaustive hyperparameter search.
The observed data efficiency is consistent with useful pretrained priors. It is not, by itself, an isolation of which internal representations account for the effect.
Four-domain experiment
at the earlier checkpoint
Bars show update counts, not accuracy. Each condition is measured from its own recorded base reference.