Hardware is the exact same as what used to be available for $2K last year (and is still $1K cheaper from Chinese OEMs).
LTT Lab's LLM testing is getting more sophisticated, which is great - I think it's worth noting that ROCm/Vulkan versions and llama.cpp build versions are going to have some big differences for numbers.
For those wanting to get the most out of their Strix Halos, there's both kernel tweaks and utilities like ryzenadj that can help you get the most out of it. ( http://strixhalo.wiki/ has most of that documented). Also, if you're running for coding or agentic work, if you model supports MTP, that's mature and should give you a decent (30%?) decode boost.
Seems not really worth it? About the same cost as DGX, same amount of memory and yet the bandwidth is actually slightly lower. And also the DGX is CUDA being an Nvidia device which is a big compatibility advantage
15 square inch box? Wow. Are there similar size, but less powered (and cheaper) workstations? I need a box that can build chromium reasonably fast and I would rather have something portable like this than a PC tower, but this is an overkill at $4k.
In case it saves anyone some time (from the article):
"The AMD Ryzen AI Max+ 395(Strix Halo) processor has been available since Spring 2025 and the Halo doesn’t offer anything new on that front."
It has the same 256 GB/s memory bandwidth limit as every board previously, not sure why this is even being released right now as if it's some new fangled thing - you can go get a Framework Desktop for roughly the same price or a GMKtec EVO-X2 for a bit cheaper.
It's being released right now because it's massively profitable and in high demand and has actually gone up in price over the past year so obviously AMD wants to cash in on that instead of selling these units to PC manufacturers at a lower price.
I really want a 128gb+ machine but it's brutal to be at only 256 GB/s for $4k (especially with the drawbacks of both ARM and AMD).
I fear that by the time the RTX Spark comes out it'd have to be $6k, and by the time a 128gb or more machine with 700+
GB/s comes out it'd be at $10k, way out of most consumers' hands.
A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now and it’s probably better to wait for the market to be less insane.
I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks.
Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
Late last year I was debating a Framework desktop vs waiting for a M5 studio. I went with the former in December 2025, glad I did now as everything has gone up in price and if I had put off the decision I’d probably still be putting it off.
For anyone considering these devices, the only reason I would recommend against them is if you plan on getting multiple to link together - the DGX Spark has a much, much faster interconnect bandwidth ceiling than the AMD devices do.
Yep, the only reason I bought mine (in late 2025, before hardware prices went totally crazy) was because it was half the price of a Spark. I spent a while fiddling around with the right Linux kernel, kernel firmware, ROCm installs, etc.
32 Gb DDR4 RAM module has a bandwidth of 25 Gb/s and costs $160. If you buy 8 of these, you get 256 Gb RAM with 200 Gb/s bandwidth at $1280. And if you buy 16 x 16 Gb modules (each at $60) then you can get 400 Gb/s of bandwidth for $960.
The only problem, you need 8 or 16 memory controllers. Memory controllers are not that expensive: Intel Core i3-14100F has 2 channel controller and costs $110, so we can estimate that 16-channel controller should cost not more than $880, and 8-channel controller should cost $440.
So isn't it better to make a cheap CPU with 16 DRAM controllers instead of this $4K gear having only 128 Gb? Or maybe 2 CPUs each having 8 RAM channels?
DDR5 costs 2 times more ($360 for 32 Gb) while not even having 2 times the bandwidth so it is not worth buying. It is more reasonable to make more RAM channels and stuff them with DDR4.
So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.
In the alternate reality where this happened, wouldn't the price of DDR4 still be sky high? We'll ignore any costs for CPU, chip set, and motherboard redesign. You're just pushing the demand somewhere else.
Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket.
This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double the number of pins and so much bigger sockets as compared to consumer desktop CPUs.
Well to be honest, there are a lot of NOOP pins on CPUs, but using them basically means fabbing a new die altogether, which is basically making a whole new CPU altogether.
Any performance gains caused by the internal bandwidth of the card will evaporate once you spill into system RAM, because now your bottleneck is probably a slow PCI lane.
And if your jobs do fit onto a 24GB card, then you are not the target user for the "AI mini PC" niche that these guys are trying to carve out
Are you sure about that? High memory speed is great for dense models, or when serving at high concurrency.
However for local single-user setups, it's often better to have access to more capable/bigger MoE models at reasonable speeds and lower concurrences, which is enabled by these platforms.
If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use.
I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s
While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX.
The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have.
That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself.
Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.
I run DSv4 Flash at home on 2 DGX Sparks and am pretty sure there is no more cost effective way for me to do so. I'm not interested in running smaller models.
what matters is how much memory it has; with the new MTP models, Qwen3.6 with 35B MOE, it's pumping out tokens up to ~80k context with little slow down.
It's great to get lots of tokens, but being able to handle and extent context is why it'll continue to be a great machine compared to any of the small graphics cards.
All the gpu makers make all their profit selling datacenter products. They don’t want consumer/home lab stuff with lower margins to replace their data center products so they handicap the vram in those products to make them less enticing for datacenter use.
> A shame, really, as the Ryzen 7640U, 7840U, 7840HS, and 7940HS all support 256GB of RAM.
To be fair, those platforms support dual dimms per channel, which Strix Halo would not, at least not at it's high speeds.
But reciprocally Gorgon Halo 400 just launched and it supports... 192GB. And is the exact same APU.
Memory chips did finally have their first big doubling per chip semi recently (available last February), with 48 & 64GB dimms becoming available. There is some reasonable lag here, that Strix Halo & Gorgon Halonuse lpddr5x, which perhaps had some lag, that 32GB (x4) was the best available. But now with Gorgon Halo being 192GB capable but not 256GB, it sure feels looks & seems like this is just bad spirited fuckery from AMD.
https://forum.level1techs.com/t/where-are-the-ddr5-unbuffere...
I assume they validated certain DRAM chips when the 395 first came out and they're just not going to validate any more. So newer DRAM is validated for the 495. We can't compare DDR5 and LPDDR5 since they are completely different; if 256 GB DDR5 is possible that doesn't mean anything.
Was “only” $2k in its previous form but even in this updated box the mem bandwidth is woefully inadequate.
There’s a few models with space for a dedicated GPU for hybrid inference but imo not worth it.
Save your money for a Xeon or EPYC build
I recently bought a few sparks from Micro Center for the exact same price and it comes with ConnectX-7 200Gbps inter-connectivity. Not sure how AMD feels it can charge exactly the same for less.
The differences are basically, sparks require ARM and sparks allow interconnects; so if you do have dreams of electric sheep to chain them together, you're not gonna get the AMD halo units.
But if you just want to putz around with a dev machine and do other things, not sure you'd want a spark.
Wow the prices on these have really come up.. Got my Framework desktop mainboard (Just the motherboard + CPU + soldered 128gb RAM) in Dec 2025 for ~1900 EUR
$4k is pretty darn spendy. I recently purchased a refurbished Corsair AI Workstation with almost the same hardware (same chip, same 128GB RAM, but only 1TB storage) for $2160. Pretty good deal! Codex and I wrote a Linux driver to report the power mode of the device:
I have another strix halo that I got for half the price (before this price increase world wide). AMD making lemonade is one of the best reasons to get a strix halo. Lemonade + qwen3.6 35B MTP @ Q8_0 + anythingLLM (in docker) replaced 90%+ of my AI usage. And it’s fully local! Setting everything up took less than 3 hours total, including installing the OS
It would be really nice if they included clustering support like a blueprint on how to buy several of these and cluster them to run the really large models in the best way possible.
How much are we going to pay for "AI kits" once the DRAM shortage is over? Will we be able to run a local model equivalent to the current AI frontier in sub $1000 hardware, even if dedicated, in 5 years?
Yeeeeep. There is no moat at the moment. AI companies are trying to dig one as fast as they possibly can. Either through passing laws to prevent local inference ("It's too dangerous! We need to control it") or by creating/limiting possible integrations (locking down OS/hardware, APIs/MCPs that only work with Claude/ChatGPT, etc).
Good luck trying to enforce those laws outside of the USA. And in the future China will be happy to sell local inference hardware at competitive prices.
No it won't because Chinese DRAM manufacturers have relatively low capacity and it's already being used. And in an auction, prices from different suppliers converge.
When this hardware was announced, it was expected to be in the $1200-1400 range new... so, maybe. The real question is will the powers that be let this bubble burst, and how painful will the fallout be... I have a feeling it will be worse than 2001-2002.
i wish there was a system like strix halo, but with enough lanes for a dedicated PCIe 5.0 x16 slot so you can have the best of both worlds: large sparse models on CPU with unified memory, dense models on GPU with real tensors and higher bandwidth memory.
I had hoped this was about Medusa Halo, but unfortunately, it's about 2025 technology. It's the same as Framework Desktop was at the end of last summer, which would have been a slightly silly but fun buy at $2k... I'd hope Mark Cerny / Sony launch PS6 sooner rather than later, as together with the upcoming LPDDR6 standard, it should trickle down to us in the local LLM mud eventually?
True, this is the new reality though. My main gripe with Strix Halo is memory bandwidth and compute performance. Gaming performance sits squarely in base PS5 territory just as is the case with Steam Machine AFAIR; yet due to economies of scale "cheap" 2020 era PS5 still has higher memory bandwidth by quite a bit last time I checked.
Perhaps if less spending went towards their private aviation interests LTT labs could review a piece of hardware that was released _this_ year, or maybe extend their narrow testing process to cover real-world use metrics like TTFT. Not to mention the lack of real value-perf comparison to CUDA
The repeated claim that all these different forms are not directly comparable is a very strange aspect.
Only thing that separates them is the build quality and the extra 20W of boost the framework desktop and this variant support.
They have a note on the thermals but no measurement of noise. Doesn't matter if it's stricly a whoosh or a whine, only if they bother people in the same room. And the small ones like Bosgame get a consistent complaint about the noise in in-depth youtube videos.
The one thing that's new/worth pointing out are the https://developer.amd.com/playbooks/ (https://github.com/amd/playbooks) - this is AMD's answer to Nvidia's playbooks (https://build.nvidia.com/spark / https://github.com/NVIDIA/dgx-spark-playbooks ) - I think it's great that they're actually taking this more seriously.
Hardware is the exact same as what used to be available for $2K last year (and is still $1K cheaper from Chinese OEMs).
LTT Lab's LLM testing is getting more sophisticated, which is great - I think it's worth noting that ROCm/Vulkan versions and llama.cpp build versions are going to have some big differences for numbers.
For those wanting to get the most out of their Strix Halos, there's both kernel tweaks and utilities like ryzenadj that can help you get the most out of it. ( http://strixhalo.wiki/ has most of that documented). Also, if you're running for coding or agentic work, if you model supports MTP, that's mature and should give you a decent (30%?) decode boost.
Seems not really worth it? About the same cost as DGX, same amount of memory and yet the bandwidth is actually slightly lower. And also the DGX is CUDA being an Nvidia device which is a big compatibility advantage
15 square inch box? Wow. Are there similar size, but less powered (and cheaper) workstations? I need a box that can build chromium reasonably fast and I would rather have something portable like this than a PC tower, but this is an overkill at $4k.
In case it saves anyone some time (from the article): "The AMD Ryzen AI Max+ 395(Strix Halo) processor has been available since Spring 2025 and the Halo doesn’t offer anything new on that front."
It has the same 256 GB/s memory bandwidth limit as every board previously, not sure why this is even being released right now as if it's some new fangled thing - you can go get a Framework Desktop for roughly the same price or a GMKtec EVO-X2 for a bit cheaper.
It's being released right now because it's massively profitable and in high demand and has actually gone up in price over the past year so obviously AMD wants to cash in on that instead of selling these units to PC manufacturers at a lower price.
This. I bought Framework Desktop in November 2025 with almost these exact specs for ~$2.5k
I really want a 128gb+ machine but it's brutal to be at only 256 GB/s for $4k (especially with the drawbacks of both ARM and AMD).
I fear that by the time the RTX Spark comes out it'd have to be $6k, and by the time a 128gb or more machine with 700+ GB/s comes out it'd be at $10k, way out of most consumers' hands.
Edit: capitalized gb/s to GB/s.
A Mac Studio is a much better buy in terms of memory bandwidth, but impossible to buy in a 128 GB configuration. Honestly there aren’t great options right now and it’s probably better to wait for the market to be less insane.
I looked for one and it's impossible to find, let alone at a reasonable price + it does suffer from being harder to train/use less common models and workflows (e.g. arbitrary comfyui ones). Spark at least doesnt have that drawback, while AMD has both drawbacks.
Waiting for the market to be less insane is somewhat akin to waiting for the s&p500 to drop a decent amount so you can buy in.
Late last year I was debating a Framework desktop vs waiting for a M5 studio. I went with the former in December 2025, glad I did now as everything has gone up in price and if I had put off the decision I’d probably still be putting it off.
To be clear though that's GB/s. Which is 2 terabits/sec
And 4000 USD is over 1.2 million Hungarian forints.
These devices were great when they were cheaper than the DGX Spark.
But when they cost the same price (unless the Spark has shot up too), there's no reason to buy this over a Spark.
The Spark is literally a faster version of this, with better software support.
Edit: And I say that as an owner of a Ryzen AI Max 395 device.
Ability to run any OS is a pretty nice benefit versus the spark.
As far as I know, you can use other OSes once the Spark's firmware is updated with LVFS.
You'll need a custom-built distro image, but that goes for like 90% of ARM hardware on Linux.
I dropped a Framework mainboard in rack mount case and use it as a speedy low power x86 homelab as well as an inference server.
Cheapest I've been able to get a DGX Spark FE is now around $4700 just FYI. This is from multiple vendors in higher-ed.
Ah ya then that's a bit of a gap.
For anyone considering these devices, the only reason I would recommend against them is if you plan on getting multiple to link together - the DGX Spark has a much, much faster interconnect bandwidth ceiling than the AMD devices do.
Otherwise, they're great!
Yep, the only reason I bought mine (in late 2025, before hardware prices went totally crazy) was because it was half the price of a Spark. I spent a while fiddling around with the right Linux kernel, kernel firmware, ROCm installs, etc.
yeah, if only there wasn't some global hegemony that immediately drove up the price of all memory everywhere...
That's the rub... this platform was supposed to be around $1200-1400 maxed out, but now it's $4k, which is insane.
32 Gb DDR4 RAM module has a bandwidth of 25 Gb/s and costs $160. If you buy 8 of these, you get 256 Gb RAM with 200 Gb/s bandwidth at $1280. And if you buy 16 x 16 Gb modules (each at $60) then you can get 400 Gb/s of bandwidth for $960.
The only problem, you need 8 or 16 memory controllers. Memory controllers are not that expensive: Intel Core i3-14100F has 2 channel controller and costs $110, so we can estimate that 16-channel controller should cost not more than $880, and 8-channel controller should cost $440.
So isn't it better to make a cheap CPU with 16 DRAM controllers instead of this $4K gear having only 128 Gb? Or maybe 2 CPUs each having 8 RAM channels?
DDR5 costs 2 times more ($360 for 32 Gb) while not even having 2 times the bandwidth so it is not worth buying. It is more reasonable to make more RAM channels and stuff them with DDR4.
If you want Epyc go for it. The motherboards can be quite expensive though.
So what I am trying to say, industry took a wrong turn. Instead of moving to over-priced DDR5, they should just make even cheapest CPUs support 8/16 DDR4 channels. Because a 32Gb DDR5-4800 module costs $360, and two 32Gb DDR4-3200 modules cost $320, so you get twice more size, more bandwidth and it costs you less. DDR5 is just a rip off.
In the alternate reality where this happened, wouldn't the price of DDR4 still be sky high? We'll ignore any costs for CPU, chip set, and motherboard redesign. You're just pushing the demand somewhere else.
Each memory controller interface is a not-insignificant number of PCB traces. Increasing the number of memory controllers may dramatically increase the number of PCB layers (or may not, it really depends on the CPU pinout) but it definitely will increase the number of pins on the CPU socket.
This is one of the main reasons (the other is the number of PCIe lanes) why high end desktop and server CPUs have like double the number of pins and so much bigger sockets as compared to consumer desktop CPUs.
> just make even cheapest CPUs support 8/16 DDR4 channels
Isn't adding pins kind of expensive?
Well to be honest, there are a lot of NOOP pins on CPUs, but using them basically means fabbing a new die altogether, which is basically making a whole new CPU altogether.
256gbs memory bandwidth is about 1/4 that of a 3090. It would be a better buy with half the memory at 4x the speed.
Any performance gains caused by the internal bandwidth of the card will evaporate once you spill into system RAM, because now your bottleneck is probably a slow PCI lane.
And if your jobs do fit onto a 24GB card, then you are not the target user for the "AI mini PC" niche that these guys are trying to carve out
Are you sure about that? High memory speed is great for dense models, or when serving at high concurrency.
However for local single-user setups, it's often better to have access to more capable/bigger MoE models at reasonable speeds and lower concurrences, which is enabled by these platforms.
If you're using a MoE model, then why do you care about the larger RAM offered by these devices? That's the main problem with low bandwidth devices: they limit the effective ram you can make use.
I do (and have historically done) quite a work with both local LLMs and local diffusion models. I have an M3 Max MBP at 400 GB/s and also a desktop with a RTX 4090 with 1,008 GB/s
While the M3 Max MBP can serve up MoE reasonably fast (~60 token/sec)the RTX 4090 is an entirely different experience (~170 token/sec). I also do a fair bit of experimentation and am currently running a custom decoder that requires expensive look-ahead, but I'm still able to get a usable 25 token/s on the RTX.
The raison d'etre for the DGX spark is not practical home inference, but rather offering the same fundamental architecture as data center cards for a affordable CUDA prototyping. If you want to build software to run on H100s, you probably can't justify buying (and running) a single card. The DGX spark solves this by having the same fundamental setup as what those cards have.
That makes these non-NVIDIA DGX-like devices confusing to me. The entire benefit of the DGX series is the NVIDIA architecture itself.
Anyone interested in home LLMs should decide whether a Mac or a dedicated GPU is the more sensible path based on their budget and other computer use. Each has their own benefits.
I run DSv4 Flash at home on 2 DGX Sparks and am pretty sure there is no more cost effective way for me to do so. I'm not interested in running smaller models.
it depends
it allows you to run smaller models much better
imo 3090s make the most sense if you can buy at least 2x ideally 4x but of course we're talking about a completely different budget at that point
what matters is how much memory it has; with the new MTP models, Qwen3.6 with 35B MOE, it's pumping out tokens up to ~80k context with little slow down.
It's great to get lots of tokens, but being able to handle and extent context is why it'll continue to be a great machine compared to any of the small graphics cards.
I was considering getting an AI Max+ machine last year when the price was around $2K. Crazy to see the same specs now going for double the price.
Why do all similar products have a hard limit on the 128 GB VRAM part? For that price, I hoped to get at least 224 GB VRAM
The 495 is going to support 192 GB. It depends on the memory bus.
128 bit: 96 GB?
256 bit: 192 GB
512 bit: 384 GB?
1024 bit: 768 GB?
All the gpu makers make all their profit selling datacenter products. They don’t want consumer/home lab stuff with lower margins to replace their data center products so they handicap the vram in those products to make them less enticing for datacenter use.
Because it’s a limit of the platform
https://community.frame.work/t/was-there-no-possible-way-to-...
From the replies,
> A shame, really, as the Ryzen 7640U, 7840U, 7840HS, and 7940HS all support 256GB of RAM.
To be fair, those platforms support dual dimms per channel, which Strix Halo would not, at least not at it's high speeds.
But reciprocally Gorgon Halo 400 just launched and it supports... 192GB. And is the exact same APU.
Memory chips did finally have their first big doubling per chip semi recently (available last February), with 48 & 64GB dimms becoming available. There is some reasonable lag here, that Strix Halo & Gorgon Halonuse lpddr5x, which perhaps had some lag, that 32GB (x4) was the best available. But now with Gorgon Halo being 192GB capable but not 256GB, it sure feels looks & seems like this is just bad spirited fuckery from AMD. https://forum.level1techs.com/t/where-are-the-ddr5-unbuffere...
I assume they validated certain DRAM chips when the 395 first came out and they're just not going to validate any more. So newer DRAM is validated for the 495. We can't compare DDR5 and LPDDR5 since they are completely different; if 256 GB DDR5 is possible that doesn't mean anything.
Was “only” $2k in its previous form but even in this updated box the mem bandwidth is woefully inadequate. There’s a few models with space for a dedicated GPU for hybrid inference but imo not worth it. Save your money for a Xeon or EPYC build
I recently bought a few sparks from Micro Center for the exact same price and it comes with ConnectX-7 200Gbps inter-connectivity. Not sure how AMD feels it can charge exactly the same for less.
it's 2026.07 and 128 GB of VRAM costs a firstborn.
The spark also has 128GB VRAM (same type) and by recently I mean I bought them last week for $3999 each.
yeah, so $500 spread https://www.microcenter.com/product/699008/nvidia-dgx-spark is what the current price appears to be.
The differences are basically, sparks require ARM and sparks allow interconnects; so if you do have dreams of electric sheep to chain them together, you're not gonna get the AMD halo units.
But if you just want to putz around with a dev machine and do other things, not sure you'd want a spark.
Please note that 8 x 16 Gb DDR4 modules cost $480, and have a bandwidth of 200 Gb/s, the only problem you need 8 DRAM controllers.
> the only problem you need 8 DRAM controllers.
Which is a huge problem? Even using 2x memory controllers in typical consumer motherboard can make system very unstable.
Wow the prices on these have really come up.. Got my Framework desktop mainboard (Just the motherboard + CPU + soldered 128gb RAM) in Dec 2025 for ~1900 EUR
Indeed December 2025 was the best time to buy.
$4k is pretty darn spendy. I recently purchased a refurbished Corsair AI Workstation with almost the same hardware (same chip, same 128GB RAM, but only 1TB storage) for $2160. Pretty good deal! Codex and I wrote a Linux driver to report the power mode of the device:
https://github.com/pettijohn/corsair-ai-workstation-performa...
Is that price still available?
I have another strix halo that I got for half the price (before this price increase world wide). AMD making lemonade is one of the best reasons to get a strix halo. Lemonade + qwen3.6 35B MTP @ Q8_0 + anythingLLM (in docker) replaced 90%+ of my AI usage. And it’s fully local! Setting everything up took less than 3 hours total, including installing the OS
https://lemonade-server.ai/
I want to play with openclaw for continuous workflows without burning my cloud credits. Do I want this?
no, you openclaw is too vibecoded, use Hermes
Even a two-year-old Mac Studio outperforms this kit. A used unit with sufficient memory currently seems to offer the best price-to-performance ratio
"The Apple Silicon Mac Studios outperform the AMD Ryzen AI Max+ 395 machines"
A 2-year-old Mac Studio 128GB also sells for more.
It would be really nice if they included clustering support like a blueprint on how to buy several of these and cluster them to run the really large models in the best way possible.
How much are we going to pay for "AI kits" once the DRAM shortage is over? Will we be able to run a local model equivalent to the current AI frontier in sub $1000 hardware, even if dedicated, in 5 years?
Yeeeeep. There is no moat at the moment. AI companies are trying to dig one as fast as they possibly can. Either through passing laws to prevent local inference ("It's too dangerous! We need to control it") or by creating/limiting possible integrations (locking down OS/hardware, APIs/MCPs that only work with Claude/ChatGPT, etc).
Good luck trying to enforce those laws outside of the USA. And in the future China will be happy to sell local inference hardware at competitive prices.
Open, cheap & good enough will win the race.
frontier to laptop runnable open weight so far seems to be ~2 years latency, so maybe there's some hope
For that matter, just getting Chinese DRAM into the market could cool pricing down a lot to oppose the cartel.
No it won't because Chinese DRAM manufacturers have relatively low capacity and it's already being used. And in an auction, prices from different suppliers converge.
When this hardware was announced, it was expected to be in the $1200-1400 range new... so, maybe. The real question is will the powers that be let this bubble burst, and how painful will the fallout be... I have a feeling it will be worse than 2001-2002.
i wish there was a system like strix halo, but with enough lanes for a dedicated PCIe 5.0 x16 slot so you can have the best of both worlds: large sparse models on CPU with unified memory, dense models on GPU with real tensors and higher bandwidth memory.
Does this have the same memory bandwidth problems as the spark?
Yes. And the same not-enough-memory problems too
What's the Spark's memory bandwidth?
273 GB/s. Same ballpark as M4 Pro and Strix Halo.
Ah yeah, bummer. It's fine for building something that you know needs to run faster in the next generation.
I had hoped this was about Medusa Halo, but unfortunately, it's about 2025 technology. It's the same as Framework Desktop was at the end of last summer, which would have been a slightly silly but fun buy at $2k... I'd hope Mark Cerny / Sony launch PS6 sooner rather than later, as together with the upcoming LPDDR6 standard, it should trickle down to us in the local LLM mud eventually?
> I hope Mark Cerny launches his PS6 sooner rather than later
With the current RAM and SSD prices... I rather a bit later.
True, this is the new reality though. My main gripe with Strix Halo is memory bandwidth and compute performance. Gaming performance sits squarely in base PS5 territory just as is the case with Steam Machine AFAIR; yet due to economies of scale "cheap" 2020 era PS5 still has higher memory bandwidth by quite a bit last time I checked.
PS6 "undertaker of physical media" will supposedly be priced >$1k: https://youtu.be/-F1JS-4Abjo
Are the likes of Dell and Lenovo not going to be annoyed that AMD are cutting them out?
As traditionally AMD was a supplier of parts.
Both Dell and Lenovo have tended to favor Intel first... even more recently on business laptops.
Do they care that Microsoft is selling the Surface, or that Intel used to sell the NUC?
Intel = Fair as they sell both Intel and AMD so no loyalty on either side.
Microsoft = yes, they care enormously, as Surface has taken away many sales. Albeit they sold some ChromeBooks
Perhaps if less spending went towards their private aviation interests LTT labs could review a piece of hardware that was released _this_ year, or maybe extend their narrow testing process to cover real-world use metrics like TTFT. Not to mention the lack of real value-perf comparison to CUDA
Bosgame is $2799 does the same thing if you plan to run only 1 of them
The repeated claim that all these different forms are not directly comparable is a very strange aspect.
Only thing that separates them is the build quality and the extra 20W of boost the framework desktop and this variant support.
They have a note on the thermals but no measurement of noise. Doesn't matter if it's stricly a whoosh or a whine, only if they bother people in the same room. And the small ones like Bosgame get a consistent complaint about the noise in in-depth youtube videos.
Drastically slower than Macs and NVIDIA unified memory boxes while not being any cheaper.
I love how these dev!kits are for devs in silicon valley making 300k+ a year and not any other dev in any other part of the world.