Intel's next-gen Arrow Lake CPUs might come without hyperthreaded cores(tomshardware.com)
tomshardware.com
Intel's next-gen Arrow Lake CPUs might come without hyperthreaded cores
https://www.tomshardware.com/pc-components/cpus/intels-next-gen-arrow-lake-cpus-might-come-without-hyperthreaded-cores-leak-points-to-24-cpu-cores-ddr5-6400-support-and-a-new-800-series-chipset
49 comments
Not having to worry about if hyperthreading helps or hurts should make it less confusing. If your threads are cpu-busy, run one thread per core and you're done.
If your threads are io-busy, pick thread counts based on io limits (depending on where on the throughput/latency spectrum you fall)
If you've got P and E cores, you might need to do some perhaps new stuff to balance threads --- the old way where an e core was about half the perf of a P core, so two threads on P and one on E was kind of even won't work without hyperthreads, but if you don't cpu pin your threads, maybe it works out. (Maybe the OS needs to do more work to track usage though)
If your threads are io-busy, pick thread counts based on io limits (depending on where on the throughput/latency spectrum you fall)
If you've got P and E cores, you might need to do some perhaps new stuff to balance threads --- the old way where an e core was about half the perf of a P core, so two threads on P and one on E was kind of even won't work without hyperthreads, but if you don't cpu pin your threads, maybe it works out. (Maybe the OS needs to do more work to track usage though)
Except that in this case a P core being able to run two threads, but registering as one "core", means you'll need to account for that to take full use of them, otherwise they're only half loaded.
Maybe the CPU can report how many threads it's capable of executing, regardless of how the cores are structured and their various capabilities, that'd be nice. I can imagine Intel might push this further, having 4 threads per core some day.
Maybe the CPU can report how many threads it's capable of executing, regardless of how the cores are structured and their various capabilities, that'd be nice. I can imagine Intel might push this further, having 4 threads per core some day.
In this case, the rumor says this core design no longer has hyperthreading, so a P core can't run two threads. If that's true, it seems more likely that Intel is heading towards ending multithreaded cores than going towards 4 threads per core. IMHO, more threads per core helps with interactivity at very low core counts (like 1 core), and can help with throughput by scheduling other work during memory loads, but at higher core counts, memory bandwidth is oversubscribed and you end up with both threads waiting for memory. Die space wise, it may make more sense to put in full e-cores or c cores rather than hyperthreads. And security wise, hyperthreads are a nightmare.
In the case where a core can run multiple threads, you have to benchmark to see if it makes more sense to run one thread per core (and possibly disable hyperthreading) or one OS thread per cpu thread. Sometimes there's a big difference, usually it's fairly small; and it can go either way. It's not unreasonable to run one thread per cpu-thread as a default, for cpu-busy code.
It's definitely not the case that if you have a core capable of running two threads that you lose half your throughput by only running a single thread. Depending on core design, you may or may not have a penalty by running single threaded and configured for two threads, if the core does a static partition of resources such as rename registers. But even in that case, losing half the rename registers is unlikely to reduce throughput by half.
In the case where a core can run multiple threads, you have to benchmark to see if it makes more sense to run one thread per core (and possibly disable hyperthreading) or one OS thread per cpu thread. Sometimes there's a big difference, usually it's fairly small; and it can go either way. It's not unreasonable to run one thread per cpu-thread as a default, for cpu-busy code.
It's definitely not the case that if you have a core capable of running two threads that you lose half your throughput by only running a single thread. Depending on core design, you may or may not have a penalty by running single threaded and configured for two threads, if the core does a static partition of resources such as rename registers. But even in that case, losing half the rename registers is unlikely to reduce throughput by half.
Isn't that still just a matter of asking the operating system how many cores it has? My understanding is that hyperthreading shows up as CPU cores, so if the CPU doesn't have hyperthreading it just won't show up as having that many cores.
Except in this case 1 P core can handle 2 threads, while before 1 P core shows up as 2 "cores", so the math is easier.
> Also, the next-generation desktop CPU is set to support 16 PCIe lanes for graphics cards and two x4 lanes for solid-state drives. Meanwhile, the whole platform supports a DisplayPort output at a UHB20 rate and Thunderbolt 4 connectors, assuming that the motherboard has an appropriate Hayden Bridge retimer (which will not be cheap).
Really curious to see how the connectivity plays out. Raptor Lake has 16x PCIe 5.0 + 4x 4.0 on cpu, then 8x lanes to the chipset. But so far no USB4 on chipset.
Here it says Thunderbolt 4, which is 40Gb/s. Will those be through the DMI link too? They're also saying it has 80Gb/s, which probably is tied to the cpu. Now with USB/Thunderbolt you need more than some line mixing, so it seems more likely for the USB to be on cpu than chipset.
Really curious to see how the connectivity plays out. Raptor Lake has 16x PCIe 5.0 + 4x 4.0 on cpu, then 8x lanes to the chipset. But so far no USB4 on chipset.
Here it says Thunderbolt 4, which is 40Gb/s. Will those be through the DMI link too? They're also saying it has 80Gb/s, which probably is tied to the cpu. Now with USB/Thunderbolt you need more than some line mixing, so it seems more likely for the USB to be on cpu than chipset.
Might not be that bad, based on experience in parallel code, hyperthreading is actually harmfull as the shared CPU units increase the pressure on the code, and available caches, thus actually reducing the performance, versus having each thread running on dedicated cores.
Then there is the whole security attacks that hyperthreading gives an helping hand.
Then there is the whole security attacks that hyperthreading gives an helping hand.
Well! That's actually an effective defense against most if not all of the Spectre-family attacks, right?
Probably makes verification easier because you don't have to worry about the case of "sharing resources between hardware threads" - instead you can focus on worrying about the only-slightly-less-complicated "sharing resources between privilege domains on a single hardware thread".
edit: Instead running two hardware threads interleaved and sharing the resources on a single core, the implication is that you're turning it into a problem of scheduling work on separate cores with some provably separate resources. That probably makes it easier to have certain guarantees, and leaves you more time to worry about resources that you really do need to share but cannot afford to duplicate/distribute across the machine. Sometimes you really want two threads to share memory!
edit: Instead running two hardware threads interleaved and sharing the resources on a single core, the implication is that you're turning it into a problem of scheduling work on separate cores with some provably separate resources. That probably makes it easier to have certain guarantees, and leaves you more time to worry about resources that you really do need to share but cannot afford to duplicate/distribute across the machine. Sometimes you really want two threads to share memory!
Sure, though most workloads will take a hit from this. Probably the right move but painful regardless.
TBH a lot of consumer stuff doesn't really benefit from hyperthreading. And thats if the scheduling is working perfectly.
You mostly want it for, say, low IPC server database workloads where the cores are largely just sitting there, twiddling their thumbs waiting for something from DRAM, where the very fancy branch predictors and cache hierarchies we have now can't help much.
You mostly want it for, say, low IPC server database workloads where the cores are largely just sitting there, twiddling their thumbs waiting for something from DRAM, where the very fancy branch predictors and cache hierarchies we have now can't help much.
I read a paper recently that basically said 'about a dozen threads' is optimal for most workloads. If they can deliver 12 'real' threads then the hyperthreads are redundant.
Also, last time I checked (late penryn) the branch prediction and other elements necessary for good HT take up egregious amounts of die space -- which can probably be put to better use.
Also, last time I checked (late penryn) the branch prediction and other elements necessary for good HT take up egregious amounts of die space -- which can probably be put to better use.
Yeah that is the other big thing, what is the opportunity cost of HT?
In the past Intel has claimed its not very much area, but (as you said) I am not so sure.
In the past Intel has claimed its not very much area, but (as you said) I am not so sure.
Surely it would be a question of how parallel the workload is? I was under the impression that hyperthreading actually reduced performance per core, it just gives you sort of twice as many cores to work with.
Relatively few workloads live in tight-loops, they tend to jump around (busting the pipeline and idling for relative eons). Those that do (and are important) tend to get run on HPC clusters. Media encoding is a good example but even that is being HW offloaded these days.
Look up some of the Jim Keller interviews on YouTube if you're interested in learning more.
Look up some of the Jim Keller interviews on YouTube if you're interested in learning more.
I am more interested in the (old) rumors of a big 384 EU graphics processor:
https://www.notebookcheck.net/More-details-on-the-2024-Intel...
I am not up-to-date on Arc's architecture, but I am hoping that is in M Pro/256 bit memory bus territory. Big enough to run/finetune generative AI (especially with Intel's many own contributions to inference engines) with sufficient speed and plenty of RAM.
https://www.notebookcheck.net/More-details-on-the-2024-Intel...
I am not up-to-date on Arc's architecture, but I am hoping that is in M Pro/256 bit memory bus territory. Big enough to run/finetune generative AI (especially with Intel's many own contributions to inference engines) with sufficient speed and plenty of RAM.
It would make sense to a point. It reduces complexity and certain attacks. With core counts skyrocketing it's mattering far less than it used to.
> However, a replacement for the traditional HT is likely to come in the form of Rentable Units, a more efficient pseudo-multi-threaded solution that splits the first thread of incoming instructions into two partitions, assigning them to different cores based on complexity. Rentable Units will use timers and counters to measure P/E core utilization and send parts of the thread to each core for processing.
https://www.hardwaretimes.com/intel-15th-gen-cpus-to-get-ren...
https://www.hardwaretimes.com/intel-15th-gen-cpus-to-get-ren...
I'm a novice but doesn't this bring back the spectre (ha) of process (or cache?) security concerns that have cropped up with multithreaded approaches in the past years?
I'm not sure "bring back" is the right way to put it: it's been haunting us this whole time.
Getting rid of SMT gives you guarantees that certain resources are separated between software threads. That probably makes it easier for them to focus on making sure that other resources can be shared correctly.
Getting rid of SMT gives you guarantees that certain resources are separated between software threads. That probably makes it easier for them to focus on making sure that other resources can be shared correctly.
Yeah sorry I didn't mean "bring back" in that its a solved problem, only that I've heard of moving away from SMT (as Intel may be doing here) as one of the best, if hardline, solutions for, at least, some of those concerns. But then the approach the GP quoted seems to step back towards that?
I don’t think SMT makes much sense if you have enough cores to more than max out the power envelope of the socket when they all run at max frequency. You can then dynamically tune the system to get as close to the power envelope as possible with the available threads by adjusting the power usage of each core. Also the cores would be simpler. Meaning more power efficient and potentially eliminate some attack vectors as mentioned earlier in this thread.
Yeah, I've been surprised that hyperthreading was still around for quite a while now. It's from an era when the goal was to squeeze as many instructions per time unit as possible through a given transistor count. Or perhaps more correctly, through a given chip area. But that's in the past, now the goal is achieving as many instructions as possible per watt. And if that requires a larger chip (or even better: more chiplets), so be it.
I think two cores that are 50% utilized burn through less power than one core that is 100% utilized but that also contains all the extra stuff it needs to pretend it's two cores. And that's the best case scenario. Quite possible that in the days HT was introduced, idle units weren't half as good at not consuming power as they are now.
I think two cores that are 50% utilized burn through less power than one core that is 100% utilized but that also contains all the extra stuff it needs to pretend it's two cores. And that's the best case scenario. Quite possible that in the days HT was introduced, idle units weren't half as good at not consuming power as they are now.
It's just anecdotal but I really feel like Intel hyper threading helped their cores not pipeline stall more than anything so it may have helped the feel and benchmarks until their core count rose or their thread scheduler improved.
> max frequency
There is no such thing, basically core speed is a curve where they get more and more inefficient the faster they run, and the hard frequency limit is just some arbitrary point on that curve.
If a particular HT friendly workload gets more done at a specific clockspeed, that is a significant boost to efficiency even if it uses more power, as clockspeed power usage scales very non linearly towards the top.
That being said, I dont think you are wrong. In consumer devices, HT seems like a bad tradeoff for a bigger, more cache heavy core.
There is no such thing, basically core speed is a curve where they get more and more inefficient the faster they run, and the hard frequency limit is just some arbitrary point on that curve.
If a particular HT friendly workload gets more done at a specific clockspeed, that is a significant boost to efficiency even if it uses more power, as clockspeed power usage scales very non linearly towards the top.
That being said, I dont think you are wrong. In consumer devices, HT seems like a bad tradeoff for a bigger, more cache heavy core.
Ditching hyperthreading in server processors would create an interesting pricing/communication challenge for cloud providers. Currently, they tend to sell single hyperthreaded cores as if they were two cores.
Not really; they sell each x86 core as two vCPUs. There could be some confusion if a new generation has fewer vCPUs but better performance. Customers should do their own benchmarking but I'm sure many of them don't bother and just choose VM types based on specs.
That is the current graviton model right?
They beat on price and energy with no SMT.
This should be ok I assume in a similar vein if you can stuff as many cores as possible.
They beat on price and energy with no SMT.
This should be ok I assume in a similar vein if you can stuff as many cores as possible.
No. Due to spectre and other similar issues, all of them (at least all the sane ones) sell cores, but will count each core as either one "core" or more commonly two "vcpus".
Could someone here with experience tell me about how important hyper-threading is in whatever code they have experience with? In my prehistoric experience we got support from Intel to put hyper-threading code in a game around 2001-- that would have been with a Xeon processor. Our mandate was clear but unwritten, have a particular level of the game perform much better on an Intel CPU then any other x86 and then talk about it. As I recall we really had to jump through hoops to get a meaningful performance improvement. It was more like we slightly crippled non hyper-threading cpus on a handful of assets. We wanted to sell the game everywhere. It was like, if you looked at a particular rock in a particular level you really could measure the difference.
It probably got a lot better with the i7, but experiences like that make me take new CPU features with a grain of salt. Hopefully someone can share more contemporary experience with hyper-threading.
It probably got a lot better with the i7, but experiences like that make me take new CPU features with a grain of salt. Hopefully someone can share more contemporary experience with hyper-threading.
In general:
* hyperthreading is useless for purely-numerical code (the inner loop of math/science software), and for this kind of code it is also often harmful per the next point
* hyperthreading is harmful for code that is highly cache-sensitive within the scheduling quantum (though multiple cores may share caches too, and for the degenerate case of RAM-as-cache-for-swap people generally think in terms of "too many threads means more thrashing"; hyperthreading really isn't special there). This is more likely for "busy" tasks, but "background" tasks are likely to have no problem here.
* hyperthreading is beneficial for OO-like code that does a lot of pointer-chasing (which almost all software is, at least in part)
* hyperthreading might beat threads in separate cores for contended atomics
If you're trying to optimize particular code, it's generally better to eliminate the horrible memory patterns in the first place than expect a handwaved hardware "solution". Where it wins is for broader code that is too spread out for you to focus your attention on.
And of course, there is still a lot of code that is fundamentally single-threaded (at least in the critical path), and if allowed to hog the CPU will not see any benefit from hyperthreading.
For a long time hyperthreading was any easy win on desktop for all the various single-threaded background tasks. Now, with increasing core counts, specialized E- vs P-cores (and focus on power management in general), and Spectre/Meltdown, I'm not sure.
* hyperthreading is useless for purely-numerical code (the inner loop of math/science software), and for this kind of code it is also often harmful per the next point
* hyperthreading is harmful for code that is highly cache-sensitive within the scheduling quantum (though multiple cores may share caches too, and for the degenerate case of RAM-as-cache-for-swap people generally think in terms of "too many threads means more thrashing"; hyperthreading really isn't special there). This is more likely for "busy" tasks, but "background" tasks are likely to have no problem here.
* hyperthreading is beneficial for OO-like code that does a lot of pointer-chasing (which almost all software is, at least in part)
* hyperthreading might beat threads in separate cores for contended atomics
If you're trying to optimize particular code, it's generally better to eliminate the horrible memory patterns in the first place than expect a handwaved hardware "solution". Where it wins is for broader code that is too spread out for you to focus your attention on.
And of course, there is still a lot of code that is fundamentally single-threaded (at least in the critical path), and if allowed to hog the CPU will not see any benefit from hyperthreading.
For a long time hyperthreading was any easy win on desktop for all the various single-threaded background tasks. Now, with increasing core counts, specialized E- vs P-cores (and focus on power management in general), and Spectre/Meltdown, I'm not sure.
This is oversimplifying. Pipeline stalls aren't the same thing as "horrible memory patterns". Not all workloads can be optimized like that at all, some very legitimate workloads are inherently random access (think RAM caching and databases!) and we still want to run them on CPUs tuned for wide issue of in-register instructions.
And when your very legitimate workload does stall, inevitably, it's nice if the CPU has another thread able to issue while it waits. In fact hyperthreading benefits almost all but the most heavily tuned workloads. Pretty much everyone stalls on a routine enough basis to make it worthwhile.
Now, in the post-spectre world where you need to spend transistors on firewalling/tagging to prevent information leaks, maybe it's not as big a benefit. But it's mischaracterizing history to claim that it was never worthwhile.
And when your very legitimate workload does stall, inevitably, it's nice if the CPU has another thread able to issue while it waits. In fact hyperthreading benefits almost all but the most heavily tuned workloads. Pretty much everyone stalls on a routine enough basis to make it worthwhile.
Now, in the post-spectre world where you need to spend transistors on firewalling/tagging to prevent information leaks, maybe it's not as big a benefit. But it's mischaracterizing history to claim that it was never worthwhile.
Yet GPUs which take HT to the next level by having thousands of "hyper-threads" work very well for scientific computing.
Their design is completly different, it isn't shared execution units like it happens on CPUs.
But it is. GPUs have many more threads in flight than execution units.
Threads groups get exclusive resources in SIMT execution pipelines.
And at memory stall they are exchanged with other waiting thread groups.
Just like HT.
Just like HT.
Scheduling algorithm is different.
CPUs target low latency (they switch often). GPUs target high troughput (they switch rarely, only when needed).
High troughput algorithms dont have problem with a lot of threads. Low latency algorithms have problem with a lot of threads (they need lot of cache memory because of constant switching).
CPUs target low latency (they switch often). GPUs target high troughput (they switch rarely, only when needed).
High troughput algorithms dont have problem with a lot of threads. Low latency algorithms have problem with a lot of threads (they need lot of cache memory because of constant switching).
Hyperthreading works really well when the same code is running on each core, but the data dependencies are different. Databases and ray tracing are probably the two best examples of this, but any parallel code that does a lot of random memory access or "pointer chasing" will benefit a lot.
With typical database servers, hyperthreading provides about 1.4x the performance "for free" because it allows execution units to do something useful while another thread is waiting for memory.
Where it isn't useful is typical game engines, except in some corner-cases as you've noticed.
Typical game engines like to have dedicated cores with the highest possible performance. Hyperthreading improves overall throughput at the expense of single-threaded throughput.
Similarly, game engines tend to run task-per-thread with different code, which tends to pollute the L1 code cache too much. So if two different threads are running on the same core but are doing different things, they'll fight over this precious resource.
Intel measured something like a +20% overall improvement averaged out across a wide range of workloads. A few had regressions, and a few (like ray tracing) had huge benefits.
IMHO, hyperthreading makes the most sense for servers, or laptops with a small number of cores (2-4). For high-end gaming, 16 cores with HT off will likely have the best performance.
With typical database servers, hyperthreading provides about 1.4x the performance "for free" because it allows execution units to do something useful while another thread is waiting for memory.
Where it isn't useful is typical game engines, except in some corner-cases as you've noticed.
Typical game engines like to have dedicated cores with the highest possible performance. Hyperthreading improves overall throughput at the expense of single-threaded throughput.
Similarly, game engines tend to run task-per-thread with different code, which tends to pollute the L1 code cache too much. So if two different threads are running on the same core but are doing different things, they'll fight over this precious resource.
Intel measured something like a +20% overall improvement averaged out across a wide range of workloads. A few had regressions, and a few (like ray tracing) had huge benefits.
IMHO, hyperthreading makes the most sense for servers, or laptops with a small number of cores (2-4). For high-end gaming, 16 cores with HT off will likely have the best performance.
AMD chips with hyperthreading are the reality of console gaming. It works pretty well for couple of generations and anything from before that should be viewed as ancient history not reflecting current state of the art of games and engines.
I dimly recall 10-15 years ago that we saw approximately 10% faster running texture filtering on HT cores in addition to real cores -- I'm hazy on the number and I don't remember which CPU we tested with but it seemed as though for a fetch-heavy workload SMT can help hide a cache-miss on one thread by doing ALU on the twin core. Since it didn't cost anything and we already scheduled work across real cores we didn't complain about the gains being marginal from HT cores and left it enabled.
More recently we did some DirectX 12 MT-rendering tests; since there's more considerable overhead for breaking up the work into more batches so we wanted to be sure so we checked carefully.
On a i7-12700K we found that distributing Dx12 work to SMT or E-Cores had no benefit, even in a contrived rendering stress-test.
However, on an unnamed game console, we found that scheduling additional work to the SMT cores improved draw performance by maybe 7%.
YMMV but my impression is the workload needs stall on memory enough to get signfican gains by interleaving the ALU from another thread and this might benefit low-power systems like laptops or mobile phones more than workstations.
More recently we did some DirectX 12 MT-rendering tests; since there's more considerable overhead for breaking up the work into more batches so we wanted to be sure so we checked carefully.
On a i7-12700K we found that distributing Dx12 work to SMT or E-Cores had no benefit, even in a contrived rendering stress-test.
However, on an unnamed game console, we found that scheduling additional work to the SMT cores improved draw performance by maybe 7%.
YMMV but my impression is the workload needs stall on memory enough to get signfican gains by interleaving the ALU from another thread and this might benefit low-power systems like laptops or mobile phones more than workstations.
Xeon was/is their workstation line, isn't it? Just curious how you got into a situation where you wanted to optimize a game for that type of hardware. Do gamers actually have Xeon hardware? (Why?)
Mainly ECC ram and I believe they're better at floating point calculations, but I'm not an expert. You can have multiple processors on a board too. Here's a high level explanation.
https://www.hp.com/us-en/shop/tech-takes/why-should-i-upgrad...
Here's Reddit's take
https://www.reddit.com/r/explainlikeimfive/comments/66vy2r/e...
https://www.hp.com/us-en/shop/tech-takes/why-should-i-upgrad...
Here's Reddit's take
https://www.reddit.com/r/explainlikeimfive/comments/66vy2r/e...
It seems disabling SMT on my AMD Zen2 solved the issue of the CPU being locked into low power/low frequency.
Great brand name that also helped the cloud providers to increase their margins by selling crippled threads as full CPU cores.
The sysadmin of my university's HPC turned off AMD's equivalent of hyperthreading as he claimed that if you run huge loads, running a single thread on each core gets you more compute. I found that fairly peculiar. I think the CPUs are Zen3.
as with many things HPC the answer is... it depends.
another poster here commented on a 10% performance boost overall with HT on vs it off, that about matches up with my experience in an HPC-adjacent environment.
another poster here commented on a 10% performance boost overall with HT on vs it off, that about matches up with my experience in an HPC-adjacent environment.
The silicon required to implement SMT takes up space, and Intel clearly thinks that they have a better use for it:
The P-core tile is on the smallest, most expensive node, right? So Intel is probably getting more P-core tiles per wafer by removing SMT, and making up for it by giving you more of “other stuff” that comes from cheaper wafers with higher yield. As long as they are honest and you are getting performance that matches your expectations, this is fine.
The P-core tile is on the smallest, most expensive node, right? So Intel is probably getting more P-core tiles per wafer by removing SMT, and making up for it by giving you more of “other stuff” that comes from cheaper wafers with higher yield. As long as they are honest and you are getting performance that matches your expectations, this is fine.
This makes figuring out how many threads to run for optimal performance even more confusing.