Developer converts dense models into Mixture-of-Experts, releases Qwen2.5-1.5B-MoE

An independent developer continued work on converting dense LLMs into Mixture-of-Experts models, releasing Qwen2.5-1.5B-MoE-A0.9B along with GGUF versions for local use.

Name

Qwen2.5-1.5B-MoE-A0.9B

Params

1.5B total, 0.9B active

Read it at Reddit →

Coverage: 1 sources, in order

1. 13 Oct 11:50 Reddit: Update: Converting dense models into Mixture-of-Experts

← Front page