Rendered at 06:46:01 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
NitpickLawyer 16 hours ago [-]
This links to the nvfp4 version of the model, so they only compare it to the bf16 in this model card. If you're looking for other similar model comparisons they are in the model card of the bf16 version here [1].
While it looks "behind" the qwen equivalent model on most benchmarks, a few personal notes:
- nemotron models feel to me a bit less benchmaxxed / "stubborn". That means that they generalise a bit better, or can be tasked to solve similar but not quite identical task types to the training data (something that's hard to do w/ qwen/ds models)
- nemotron series are also open training (w/ open training recipes and some training data public)
- nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models. Whatever other incentives 3rd party labs have (i.e. meta, goog w/ gemma, the chinese labs that IPOd, etc) nvda will always want to sell hardware so their incentive to keep pushing open models is evident and will likely continue "forever".
this "DFlash" model should be used together with one of the previous two "for lower-latency speculative decoding deployments tuned for low-concurrency data center and workstation workflows".
> nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models
This! It's literally in their best interest for open-weights models to succeed
happycube 14 hours ago [-]
Commoditize your compliments.
bearjaws 17 hours ago [-]
Crazy to see how well Qwen3.6 35b-a3b is holding up, sure it is ~20% larger but it's scores are also ~20% higher with the same number of active params (excluding the IFBench).
Hopefully Qwen follows up their 3.8 launch with a new 35b-a3b
WalterGR 16 hours ago [-]
What’s expected in an updated 3.6 release?
hadlock 13 hours ago [-]
4 months of progress. Qwen 3.8 Max is, depending on who you ask, what you're doing, standard disclaimers, really really close to Opus 4.8. That is a huge dump from 3.6 Max. Presumably improvements in their training librar(y/ies), training improvements, etc will "trickle down" to Qwen 3.8. We are using 3.6 35B internally and it is in most cases better than Sonnet 4.6 which already makes it "good enough" for most data enrichment tasks. I would be a lot more excited about a MoE Qwen 3.8 35B-A3B than the "announced" 27B dense model. I have 50 million records I need to keep up to date, an A3B MoE is waaaay more performant than a 27B dense model.
I tried that exact model, I get about 50 tokens/sec on an 64GB M1 Max MBP.
It writes pretty good code; it does seem to second-guess itself in thinking traces and I wonder if it just needs a reasoning budget and message.
Each time I test a model quickly in LM Studio, I ask it:
- to write a tiny little wordpress "last login" tracker plugin, asking me clarifying questions first. I ask it to use old PHP, break the WP coding guidelines to use inline anonymous functions in the hooks, avoid custom SQL, I see if it can write something useful in a singleton class, and what questions it thinks to ask. Qwen 3.6 does this very well, Nemotron has done OK, though it's a little less effective at reading between the lines, maybe.
- to offer an answer for a SQL puzzle about finding max score per category on old MySQL (5.0) without using subquery/derived tables -- it did a good job, picked up the nuances in the prompt that allow a particular solution, didn't go on a tangent about how it would be nice to have window functions or use subqueries, did a tool call to check like I asked. With this puzzle, if the model doesn't offer up an index for performance optimisation, I nudge it; this time it didn't volunteer one but when prompted about performance it offered an index and a bunch of other nice solutions, and only there did it round up the options for subqueries and derived tables, which is fair game.
(It did badly fail the car wash test, though, even on repeated nudging, where it gets more and more insane, doubling down and never getting the point, whereas Muse Glimmer solved it and well, with a thinking trace that didn't particularly suggest it had been post-trained)
I need to test it in Pi or opencode. I've been trying to motivate my brain to move to pi, but this model supports a longer context window so maybe opencode's overlong system prompt is less of an issue.
Of the 30B models in the last 24 hours (!) I think I prefer working with Muse Glimmer, which is slow but very good, and writes rather well with just a hint of being a bit of a cheeky monkey. Not tested that in an agentic coding setting yet.
Interesting times.
EchoVoicy 9 hours ago [-]
>~100 tokens/second
That sounds great to me!
>Runs fast on my Mac
What specifications (CPU/RAM)?
simonw 7 hours ago [-]
M5 Max MacBook Pro, 128GB of RAM.
gaodean 17 hours ago [-]
Developing on the Mamba 2 architecture is a really interesting point to note. It seems to be catching up to “regular” transformer architectures.
kamranjon 16 hours ago [-]
You might look at this and and be a bit disappointed by the performance against qwen and gemma models - but this is an entirely open source training pipeline, this is quite impressive and I don't think another model this performant exists with fully open source data and recipes alongside the weights.
hadlock 13 hours ago [-]
>I don't think another model this performant exists with fully open source data and recipes alongside the weights
If you're already selling shovels, you might as well provide maps to the gold mine with every purchase.
I know very little about this but why isn't it mamba-3? Isn't it... uh, more gooder?
bcatanzaro 13 hours ago [-]
The Nemotron 3 family uses Mamba-2 because that is what we had validated when we started building the family.
Changing the SSM backbone is a major revision, not a dot release. At least for us.
dofm 5 hours ago [-]
Car wash test was amusing with the LM Studio version of the model simonw mentions below.
Bad, bad failure and no amount of nudging helps it see the point. Comical.
But it did OK on other mini tests.
judem 15 hours ago [-]
Interesting to see in the Agentic Coding Benchmarks that the codex harness is so demonstrably lower than any of the other harnesses, even Claude Code another lab harness. Does anyone know why that would be?
e2e4 14 hours ago [-]
Could you please provide the link.
killix 5 hours ago [-]
[flagged]
piyh 15 hours ago [-]
That SWEBench drop going from bf16 to fp4 is massive
ggcr 16 hours ago [-]
Nice cadence of releases by the Nemotron team :)
lostmsu 5 hours ago [-]
> nvfp4
Still trying to lock in, huh?
239127 17 hours ago [-]
[flagged]
varispeed 16 hours ago [-]
Nvidia just throwing something "for peasants" to stay relevant. Where is competition spirit? More importantly why Nvidia is gatekeeping computing for everyday people?
I find these releases are bad taste.
Make 1TB DGX priced affordably, not some crap model for people to waste time on.
tasty_freeze 16 hours ago [-]
First convince the DRAM suppliers to drop their prices for you and then maybe nvidia will drop the DGX price for you too.
nicman23 15 hours ago [-]
they are selling the shovels not the gold
timmmmmmay 8 hours ago [-]
you wouldn't be happy then either
jazzyjackson 14 hours ago [-]
imo a 512GB DGX cluster for $20k is not a bad price for an enterprise that wants to do on prem LLMs. Gatekeeping is a weird way to put “running a business”
While it looks "behind" the qwen equivalent model on most benchmarks, a few personal notes:
- nemotron models feel to me a bit less benchmaxxed / "stubborn". That means that they generalise a bit better, or can be tasked to solve similar but not quite identical task types to the training data (something that's hard to do w/ qwen/ds models)
- nemotron series are also open training (w/ open training recipes and some training data public)
- nvda will have an incentive to continue this kind of releases, even if other parties slowly abandon the open release of models. Whatever other incentives 3rd party labs have (i.e. meta, goog w/ gemma, the chinese labs that IPOd, etc) nvda will always want to sell hardware so their incentive to keep pushing open models is evident and will likely continue "forever".
[1] - https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
to be used for further training/fine-tuning.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
main model.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
quantized version of the previous.
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
this "DFlash" model should be used together with one of the previous two "for lower-latency speculative decoding deployments tuned for low-concurrency data center and workstation workflows".
https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Lightning-...
like DFlash, the previous model above, but optimized for DGX Spark.
I actually copied the link from NVIDIA's Technical Blog post:
- https://developer.nvidia.com/blog/nvidia-nemotron-3-5-lightn...
You can also try the model via a free API endpoint from Openrouter, would be interesting to see if it's the BF16 or NVFP4 version:
- https://openrouter.ai/nvidia/nemotron-3.5-lightning:free
This! It's literally in their best interest for open-weights models to succeed
Hopefully Qwen follows up their 3.8 launch with a new 35b-a3b
I used LM Studio and this model file: https://lmstudio.ai/models/nvidia/nemotron-3.5-lightning
I got a pretty solid (and reasonably fast) result for this simply coding agent activity (with simonw/llm-coding-agent):
https://gist.github.com/simonw/a8741f79280cedc86bcb8d90edfb4... - took just under a minute.It writes pretty good code; it does seem to second-guess itself in thinking traces and I wonder if it just needs a reasoning budget and message.
Each time I test a model quickly in LM Studio, I ask it:
- to write a tiny little wordpress "last login" tracker plugin, asking me clarifying questions first. I ask it to use old PHP, break the WP coding guidelines to use inline anonymous functions in the hooks, avoid custom SQL, I see if it can write something useful in a singleton class, and what questions it thinks to ask. Qwen 3.6 does this very well, Nemotron has done OK, though it's a little less effective at reading between the lines, maybe.
- to offer an answer for a SQL puzzle about finding max score per category on old MySQL (5.0) without using subquery/derived tables -- it did a good job, picked up the nuances in the prompt that allow a particular solution, didn't go on a tangent about how it would be nice to have window functions or use subqueries, did a tool call to check like I asked. With this puzzle, if the model doesn't offer up an index for performance optimisation, I nudge it; this time it didn't volunteer one but when prompted about performance it offered an index and a bunch of other nice solutions, and only there did it round up the options for subqueries and derived tables, which is fair game.
(It did badly fail the car wash test, though, even on repeated nudging, where it gets more and more insane, doubling down and never getting the point, whereas Muse Glimmer solved it and well, with a thinking trace that didn't particularly suggest it had been post-trained)
I need to test it in Pi or opencode. I've been trying to motivate my brain to move to pi, but this model supports a longer context window so maybe opencode's overlong system prompt is less of an issue.
Of the 30B models in the last 24 hours (!) I think I prefer working with Muse Glimmer, which is slow but very good, and writes rather well with just a hint of being a bit of a cheeky monkey. Not tested that in an agentic coding setting yet.
Interesting times.
>Runs fast on my Mac What specifications (CPU/RAM)?
If you're already selling shovels, you might as well provide maps to the gold mine with every purchase.
https://github.com/NVIDIA-NeMo/Nemotron/blob/main/docs/nemot...
Changing the SSM backbone is a major revision, not a dot release. At least for us.
Bad, bad failure and no amount of nudging helps it see the point. Comical.
But it did OK on other mini tests.
Still trying to lock in, huh?
I find these releases are bad taste.
Make 1TB DGX priced affordably, not some crap model for people to waste time on.