vLLM v0.33.0 adds Reflection Beam-1.0 and moves to PyTorch 2.14
vLLM v0.33.0 brings 927 commits from 389 contributors, 122 of them new. Its new model support includes Reflection Beam-1.0, a 501B mixture-of-experts model with 23B active parameters, along with its DFlash2 drafter. The release moves to PyTorch 2.14.0 and ships wheels and images for CUDA 13.0 only.
Our view: Support in a widely used open serving engine puts a very large open-weight model within reach of anyone with the hardware to host it themselves.
Ollama adds Liquid AI's LFM3, small models built for local agents
Ollama v0.40.12 adds LFM3, Liquid AI's new family of small models built for local agents, tool calling and structured output. The family has a 1.7B text model with a 256K-token context window, plus image, audio and all-in-one versions, and the audio model covers speech recognition, translation and text-to-speech in 18 languages. In this release the models run on Apple hardware, with Linux and Windows support listed as coming soon.
Our view: Small models tuned for tool calling are what make agents practical on an ordinary laptop, so this is worth a test if you run AI offline.
Study: multiple-choice safety tests hide how models refuse
A new paper explains why multiple-choice safety benchmarks see fewer refusals, testing Llama-3.1-8B and Qwen3-8B on the WMDP dataset. In multiple-choice format the models commit to an answer letter early, and their activations sit about three times further from the refusal direction than with open-ended prompts. The authors conclude that multiple-choice safety scores mostly measure knowledge, not safety mechanisms.
Our view: A model judged safe or unsafe on multiple-choice tests alone may behave quite differently in open conversation, so read those scores with care.
Study: fine-tuned medical models get worse at saying they don't know
An audit compared 25 biomedical fine-tuned models with the 11 base checkpoints they were built from, using the MedAbstain and ClinDet-Bench benchmarks. It measures whether a model holds back when it lacks the knowledge, which the authors separate from refusing for value-based reasons. Appropriate abstention was already low in the base models and fell after fine-tuning in 21 of 25 pairs.
Our view: Anyone fine-tuning an open model for medicine should test whether it still admits what it doesn't know, not only whether it answers correctly.