DeepSeek-R1-Distill-Qwen-32B — Model Documentation
Published by Nextbit256 S.L. | Last updated: May 2026 Compliance reference: AI Act Regulation (EU) 2024/1689, Art. 53.1.c and 53.1.d
Model Identification
| Field | Detail |
|---|---|
| Full model name | DeepSeek-R1-Distill-Qwen-32B |
| Developed by | DeepSeek AI (distillation fine-tune); base model: Qwen Team, Alibaba Cloud (Qwen2.5-32B) |
| Release date | January 2025 |
| Model card | huggingface.co/deepseek-ai/DeepSeek-R1-Distill-Qwen-32B |
| Technical report | arXiv:2501.12948 — DeepSeek-R1 |
| Developer blog | deepseek.com |
Important note on base model: The Nextbit catalog identifier for this model is qwen3-r1-32b, which may suggest a Qwen3 base. This model is in fact based on Qwen2.5-32B (not Qwen3). The distillation fine-tune was performed by DeepSeek AI on top of the Qwen2.5-32B pretrained weights.
Architecture and Parameters
| Field | Detail |
|---|---|
| Architecture | Dense (decoder-only transformer) — Qwen2.5 architecture |
| Total parameters | 32.8B (published as "33B" rounded) |
| Active parameters per token | N/A — dense model (all parameters active) |
| Non-embedding parameters | Not published by DeepSeek AI |
| Layers | Not published by DeepSeek AI for this distilled variant |
| Attention | Not published by DeepSeek AI for this distilled variant |
| Native context length | 32,768 tokens (matches Qwen2.5-32B base) |
| Extended context | Not applicable |
| Precision | BF16 |
License and Commercial Use
License: MIT License
The MIT License is a permissive open-source license that allows free commercial use, distribution, and modification with minimal restrictions (attribution and license text inclusion required).
Nextbit's verification: Nextbit has reviewed the license terms applicable to DeepSeek-R1-Distill-Qwen-32B and confirmed that:
- The distilled model is released by DeepSeek AI under the MIT License, which permits commercial API use without restriction.
- The base model (Qwen2.5-32B) is released under Apache 2.0 by the Qwen Team. Apache 2.0 is also fully permissive for commercial use.
- Both licenses are compatible and no additional commercial agreement is required for serving this model via API.
Restrictions relevant to users: None beyond standard MIT and Apache 2.0 terms. Users of Nextbit's API who integrate model outputs into their own products remain responsible for their own compliance with applicable law and any downstream licensing requirements.
Training Data Summary
This model has two distinct training data layers: the Qwen2.5-32B base model pretraining, and the DeepSeek-R1 distillation fine-tune.
(a) Base model pretraining data — Qwen2.5-32B
DeepSeek-R1-Distill-Qwen-32B is built on the Qwen2.5-32B pretrained model. The pretraining data for Qwen2.5-32B is documented by the Qwen Team / Alibaba Cloud. Nextbit does not reproduce that documentation here; for details see the Qwen2.5 model documentation and the Qwen2.5 Technical Report.
Key known facts about Qwen2.5-32B pretraining:
- Approximately 18 trillion tokens used for pretraining
- Data includes web text, code, mathematics, and synthetic data
- Covers 29+ languages
- Knowledge cutoff: approximately early 2024
(b) Distillation fine-tune data — DeepSeek-R1
DeepSeek AI fine-tuned Qwen2.5-32B using approximately 800,000 reasoning samples generated by the DeepSeek-R1 teacher model (a 671B parameter MoE model). These samples consist of long chain-of-thought reasoning traces across mathematics, science, coding, and logical reasoning domains.
The distillation process does not add new factual knowledge to the model beyond what the base Qwen2.5-32B already contains. Its purpose is to transfer the reasoning patterns and extended thinking capabilities of the larger teacher model to the smaller 32B student.
DeepSeek AI has not published:
- The exact composition or sources of the 800K distillation samples beyond the description above
- The data filtering methodology applied to the distillation dataset
Primary reference: arXiv:2501.12948.
Knowledge cutoff
Inherited from Qwen2.5-32B base: approximately early 2024. The distillation fine-tune does not extend the knowledge cutoff.
Languages Supported
Primarily the languages supported by Qwen2.5-32B: 29+ languages including Arabic, Chinese (Simplified and Traditional), English, French, German, Hindi, Indonesian, Italian, Japanese, Korean, Malay, Portuguese, Russian, Spanish, Thai, Turkish, Ukrainian, Vietnamese, and others. The distillation fine-tune did not change language support. Full language list: see the Qwen2.5 documentation.
Intended Uses
This model is designed for:
- Advanced mathematical reasoning (AIME, MATH-500 benchmarks)
- Scientific and logical problem-solving with extended chain-of-thought
- Code generation with reasoning verification
- Tasks requiring step-by-step reasoning where intermediate thinking is made visible
The model is optimized for reasoning-intensive tasks. It generates extended <think>...</think> reasoning traces before producing final answers. Mistral AI recommends:
- Temperature: 0.5–0.7 (0.6 recommended)
- Avoiding system prompts; placing all instructions in the user turn
- Prefilling responses with
<think>\nto enforce reasoning
This model is not a general-purpose conversational model; it is best suited to structured reasoning tasks.
Systemic Risk Assessment
| Field | Detail |
|---|---|
| Training FLOPs | Not published by DeepSeek AI or Qwen Team for this distilled variant |
| Relevant base for threshold assessment | Qwen2.5-32B (the pretrained base model); the distillation fine-tune FLOPs are negligible by comparison |
| Estimated FLOPs (Qwen2.5-32B pretraining) | ~3.5 × 10²⁴ (estimate based on 32.8B parameters × ~18T tokens; formula: 6 × N × D) |
| Exceeds 10²⁵ FLOPs threshold? | No — estimated to be approximately 3× below the systemic risk threshold |
| Note on dense architecture | Qwen2.5-32B and this distilled variant are dense models; all parameters are active per token. |
| AI Office designation | Not designated as a systemic risk model as of May 2026 |
| Art. 55 obligations apply? | No |
The FLOPs relevant for systemic risk threshold assessment are those of the base model (Qwen2.5-32B pretraining), not the distillation fine-tune. This estimate is based on publicly available information (parameter count from model card; token count from Qwen2.5 documentation). If official compute figures are published, this section will be updated.
This documentation is published by Nextbit256 S.L. in accordance with Article 53(1)(c) and 53(1)(d) of Regulation (EU) 2024/1689 (AI Act). Nextbit256 S.L. serves this model via its inference API but did not develop or train it. The distillation fine-tune was performed by DeepSeek AI; the base model was developed by the Qwen Team / Alibaba Cloud. All information is sourced from public documentation published by DeepSeek AI and the Qwen Team.
For questions: [email protected]