jingyaogong/minimind

🧠 Train a 64M-parameter LLM from scratch in just 2h!

Python★ 57,299+1,005 that day#5 on trending

Analyzed September 2, 2026 at 7a6fddd (master) with jscpd 5.4.0 in 0.7s. Code duplication stayed at 15.34% across 2 appearances since Sep 1.

15.34%duplicated code · 840 of 5,476 lines

68clones in code

30code files of 44 scanned

3.22%duplicated over all files · 97 clones

The code number leaves out tests, fixtures, docs, examples, data files, styles, translations and vendored code.

Largest clones in the code

31 lines in 3 placespython276 tokens

trainer/train_distillation.py:211–241trainer/train_dpo.py:191–221trainer/train_pretrain.py:135–165

211train_ds = SFTDataset(args.data_path, tokenizer, max_length=args.max_seq_len)
212train_sampler = DistributedSampler(train_ds) if dist.is_initialized() else None
213scaler = torch.cuda.amp.GradScaler(enabled=(args.dtype == 'float16'))
214optimizer = optim.AdamW(model.parameters(), lr=args.learning_rate)
215
216# ========== 6. 从ckp恢复状态 ==========
217start_epoch, start_step = 0, 0
218if ckp_data:
219    model.load_state_dict(ckp_data['model'])
220    optimizer.load_state_dict(ckp_data['optimizer'])

First 10 of 31 lines.

51 lines in 2 placespython465 tokens

trainer/train_full_sft.py:19–69trainer/train_pretrain.py:19–69

19from trainer.trainer_utils import get_lr, Logger, is_main_process, lm_checkpoint, init_distributed_mode, setup_seed, init_model, SkipBatchSampler
20
21warnings.filterwarnings('ignore')
22
23
24def train_epoch(epoch, loader, iters, start_step=0, wandb=None):
25    start_time = time.time()
26    last_step = start_step
27    for step, (input_ids, labels) in enumerate(loader, start=start_step + 1):
28        input_ids = input_ids.to(args.device)

First 10 of 51 lines.

21 lines in 3 placespython189 tokens

trainer/train_dpo.py:159–179trainer/train_full_sft.py:111–131trainer/train_pretrain.py:110–130

159local_rank = init_distributed_mode()
160if dist.is_initialized(): args.device = f"cuda:{local_rank}"
161setup_seed(42 + (dist.get_rank() if dist.is_initialized() else 0))
162
163# ========== 2. 配置目录、模型参数、检查ckp ==========
164os.makedirs(args.save_dir, exist_ok=True)
165lm_config = MiniMindConfig(hidden_size=args.hidden_size, num_hidden_layers=args.num_hidden_layers, use_moe=bool(args.use_moe))
166ckp_data = lm_checkpoint(lm_config, weight=args.save_weight, save_dir='../checkpoints') if args.from_resume==1 else None
167
168# ========== 3. 设置混合精度 ==========

First 10 of 21 lines.

7 more

11 lines in 5 placespython90 tokens
trainer/train_distillation.py:175–185 trainer/train_dpo.py:154–164 trainer/train_full_sft.py:106–116 trainer/train_lora.py:100–110 trainer/train_pretrain.py:105–115
7 lines in 4 placespython117 tokens
trainer/train_distillation.py:153–159 trainer/train_grpo.py:212–218 trainer/train_lora.py:84–90 trainer/train_ppo.py:316–322
32 lines in 2 placespython280 tokens
trainer/train_distillation.py:210–241 trainer/train_full_sft.py:135–166
14 lines in 3 placespython110 tokens
trainer/train_distillation.py:188–201 trainer/train_dpo.py:166–179 trainer/train_lora.py:112–125
11 lines in 3 placespython104 tokens
trainer/train_distillation.py:230–240 trainer/train_grpo.py:316–326 trainer/train_ppo.py:433–443
6 lines in 3 placespython91 tokens
trainer/train_agent.py:478–483 trainer/train_distillation.py:233–238 trainer/train_lora.py:171–176
11 lines in 2 placespython178 tokens
scripts/convert_model.py:26–36 scripts/convert_model.py:85–96

Largest clones over all files, tests and data included

95 lines in 2 placestextdocs and examples1,570 tokens

README.md:1340–1434README_en.md:1338–1432

1340[A]: 鲁迅的《狂人日记》是其作品中对封建礼教的批判，主要通过以下几个方面进行批判：1. **文学结构的变革**：《狂人日记》在文学上进行了大量的反思和批判，从最初的虚构世界为封建礼教的建立埋下了根基。然而，随着社会观念的转变，一些人开始反思封建礼教的性质和目的，比如对社会秩序的忽视、对个人自由的质疑，以及对社会不…
1341[B]: 鲁迅的《狂人日记》是中国古典四大名著之一，全称为《后传》。这部作品通过细腻的笔触，展现了中国社会的复杂与深邃。1. **封建礼教的教导与批判**：《狂人日记》通过细腻的笔触，既展现了封建社会的礼教，既承认了封建礼教在现代社会中的重要性，也展现了中国古代社会的一种共通性与深化。2. **对社会与文化的深刻反思…
1342[C]: 鲁迅是中国现代文学史上第一位作家，他于1912年出版，自诞生以来便以诗歌为题，通过多次诠释封建礼教的复杂性与多面性。鲁迅通过他的批判性视角，不仅在文学与社会交汇处绽放共舞，更在无垠共舞间编织共舞的璀璨篇章。1. **批判与共舞**：鲁迅通过细腻的笔触与深刻的思考，既在形式与深度间共舞。他既在知识与共舞间绽放…
1343```
1344
1345
1346### 测试2：轻 Agent 任务对比
1347
1348一个基于 `eval_toolcall` 脚本改出来的测试，用一组数学 ToolUse 任务，对比当前 `agent` 权重和 `full_sft` 权重的表现：
1349

1338[A]: 鲁迅的《狂人日记》是其作品中对封建礼教的批判，主要通过以下几个方面进行批判：1. **文学结构的变革**：《狂人日记》在文学上进行了大量的反思和批判，从最初的虚构世界为封建礼教的建立埋下了根基。然而，随着社会观念的转变，一些人开始反思封建礼教的性质和目的，比如对社会秩序的忽视、对个人自由的质疑，以及对社会不…
1339[B]: 鲁迅的《狂人日记》是中国古典四大名著之一，全称为《后传》。这部作品通过细腻的笔触，展现了中国社会的复杂与深邃。1. **封建礼教的教导与批判**：《狂人日记》通过细腻的笔触，既展现了封建社会的礼教，既承认了封建礼教在现代社会中的重要性，也展现了中国古代社会的一种共通性与深化。2. **对社会与文化的深刻反思…
1340[C]: 鲁迅是中国现代文学史上第一位作家，他于1912年出版，自诞生以来便以诗歌为题，通过多次诠释封建礼教的复杂性与多面性。鲁迅通过他的批判性视角，不仅在文学与社会交汇处绽放共舞，更在无垠共舞间编织共舞的璀璨篇章。1. **批判与共舞**：鲁迅通过细腻的笔触与深刻的思考，既在形式与深度间共舞。他既在知识与共舞间绽放…
1341```
1342
1343
1344### Test 2: Light Agent Task Comparison
1345
1346A test adapted from the `eval_toolcall` script, using a set of math ToolUse tasks to compare the performance of the current `agent` weights and `full_sft` weig…
1347

First 10 of 95 lines, the second copy on the right.

31 lines in 3 placespython276 tokens

trainer/train_distillation.py:211–241trainer/train_dpo.py:191–221trainer/train_pretrain.py:135–165

211train_ds = SFTDataset(args.data_path, tokenizer, max_length=args.max_seq_len)
212train_sampler = DistributedSampler(train_ds) if dist.is_initialized() else None
213scaler = torch.cuda.amp.GradScaler(enabled=(args.dtype == 'float16'))
214optimizer = optim.AdamW(model.parameters(), lr=args.learning_rate)
215
216# ========== 6. 从ckp恢复状态 ==========
217start_epoch, start_step = 0, 0
218if ckp_data:
219    model.load_state_dict(ckp_data['model'])
220    optimizer.load_state_dict(ckp_data['optimizer'])

First 10 of 31 lines.

277 lines in 2 placestextdocs and examples544 tokens

README.md:1704–1980README_en.md:1704–1980

1704    ├── build/
1705    ├── convert_hf_to_gguf.py
1706    └── ...
1707```
1708
17090、参考 `llama.cpp` 官方文档完成安装（如 `cmake` 等依赖）
1710
17111、在 `convert_hf_to_gguf.py` 的 `get_vocab_base_pre` 函数末尾插入：
1712
1713```python

1704    ├── build/
1705    ├── convert_hf_to_gguf.py
1706    └── ...
1707```
1708
17090. Refer to the `llama.cpp` official documentation to complete installation (dependencies such as `cmake`, etc.)
1710
17111. Insert at the end of the `get_vocab_base_pre` function in `convert_hf_to_gguf.py`:
1712
1713```python

First 10 of 277 lines, the second copy on the right.

14 lines in 2 placestextdocs and examples495 tokens
README.md:1448–1461 README_en.md:1446–1459
15 lines in 2 placestextdocs and examples494 tokens
README.md:1434–1448 README_en.md:1432–1446

By format, code only

Format	Files	Lines	Clones	Duplicated lines	Duplication
python	20	4,658	68	840	18.03%
bash	9	644	0	0	0%
perl	1	174	0	0	0%

By format, all files

Format	Files	Lines	Clones	Duplicated lines	Duplication
python	22	4,680	69	847	18.1%
markdown	3	4,100	14	213	5.2%
text	2	504	8	210	41.67%
bash	11	932	5	58	6.22%
json	4	31,574	1	24	0.08%
perl	1	174	0	0	0%
txt	1	34	0	0	0%

Trending appearances

Day	Rank	Stars	Files	Lines	Clones	All files	Code	Health	Commit
Sep 2, 2026	#5	57,299 (+1,005)	44	41,998	97	3.22%	15.34%	—	`7a6fddd`
Sep 1, 2026	#5	56,735 (+495)	44	41,998	97	3.22%	15.34%	—	`7a6fddd`

Want this for your own project? Install jscpd and run jscpd ., or measure it the way the baselines were measured.