Any EEs that can comment on at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory lives on a PCIe slot? It seems like that would also have some power delivery advantages.
mcdeltat 12 minutes ago [-]
Personally I hope this point comes after we realise we don't need 1kW GPUs doing a whole lot of not much useful
kvemkon 17 hours ago [-]
> at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory
Actually the RapsberryPi (appeared 2012) was based on a SoC with a big and powerful GPU and small weak supporting CPU. The board booted the GPU first.
verall 18 hours ago [-]
If you look at a any of the nvidia DGX boards it's already pretty close.
PCIe is a standard/commodity so that multiple vendors can compete and customers can save money. But at 8.0 speeds I'm not sure how many vendors will really be supplying, there's already only a few doing serdes this fast...
bgnn 6 hours ago [-]
These days you can buy any standard as a soft IP from Synopsys or Cadence. They take their previous serdes and modify it to meet the new standard. They have thousands of employees across the globe just doing that.
y1n0 8 hours ago [-]
There are companies that specialize in memory controller ip that every one else uses, including large semi companies like Intel.
The ip companies are the first to support new standards, make their money selling to intel etc. Allowing intel or whomever to take their time to build higher performance ip.
Melatonic 13 hours ago [-]
Isnt it about latency as well with DGX boards? Vs PCI-E. You can only fit so much RAM on a board that will realistically be plugging into a slot
Good to see I’m not the only person that’s been thinking about this. Wedging gargantuan GPUs onto boards and into cases, sometimes needing support struts even, and pumping hundreds of watts through a power cable makes little sense to me. The CPU, RAM, these should be modules or cards on the GPU. Imagine that! CPU cards might be back..
ksec 4 days ago [-]
It is not like CPU aren't getting higher wattage as well. Both AMD and Intel have roadmap for 800W CPU.
At 50-100W for IO, this only leaves 11W per Core on a 64 Core CPU.
linotype 18 hours ago [-]
800 watt CPU with a 600 watt GPU, I mean at a certain point people are going to need different wiring for outlets right?
0manrho 12 hours ago [-]
This is a legitimate problem in datacenters. They're getting to the point where a single 40(ish)OU/RU rack can pull a megawatt in some hyperdense cases. The talk of GPU/AI datacenters consuming inordinate amounts of energy isn't just because the DC's are yuge, (although some are), but because the power draw per rack unit space is going through the roof as well.
On the consumer side of things where the CPU's are branded Ryzen or Core instead of Epyc or Xeon, a significant chunk of that power consumption is from the boosting behavior they implement to pseudo-artificially[0] inflate their performance numbers. You can save huge (easily 10%, often closer to 30%, but really depends on exact build/generation) on energy by doing a very mild undervolt and limiting boosting behavior on these cpus and keeping the same base clocks. Intel 11th through 14th gen CPU's are especially guilty of this, as are most Threadripper CPU's. you can often trade single digit or even negligible performance losses (depends on what you're using it for and how much you undervolt/underclock/restrict boosting) for double digit reductions in power usage. This phenomenon is also true for GPU's when compared across the enterprise/consumer divide, but not quite to the significant extent in most cases.
Point being, yeah, it's a problem in data centers, but honestly there's a lot of headroom still even if you only have your common American 15A@120VAC outlets available before you need to call your electrician and upgrade your panel and/or install 240VAC outlets or what have you.
0: I say pseudo-artificial because the performance advantages are real, but unless you're doing some intensive/extreme cooling, they aren't sustainable or indicative of nominal performance, just a brief bit of extra headroom before your cooling solution heat-soaks and the CPU/GPU's throttle themselves back down. But it lets them put the "Bigger number means better" on the box for marketing.
Panzer04 8 hours ago [-]
It's not just about better numbers. Getting high clocks for a short period helps in a lot of use cases - say random things like a search. If I'm looking for some specific phrase in my codebase in vscode, everything spins up for the second or two it takes to process that.
Boosting from 4 to 5,5.5 ghz for that brief period shaves a fraction of a second - repeat that for any similar operation and it adds up.
spacedcowboy 5 hours ago [-]
unless it’s an Apple data center, populated by the server version of the latest ultra chips…
jchw 18 hours ago [-]
At least with U.S. wiring we have 15 amps at 120 volts. For continuous power draw I know you'd want an 80% margin of safety, so let's say you have 1440 Watts of AC power you can safely draw continuously. Power supplies built on MOSFETs seem to peak at around 90% efficiency, but you could consider something like the Corsair AX1600i using gallium nitride transistors, which supposedly can handle up to 1600 watts at 94% efficiency.
Apparently we still have room, as long as you don't run anything else on the same circuit. :)
atonse 17 hours ago [-]
You can always have an electrician install a larger breaker for a particular circuit. I did that with my "server" area in my study, which was overkill cuz I barely pull 100w on it. But it cost nearly zero extra since he was doing a bunch of other things around the house anyway.
jacquesm 5 hours ago [-]
You need to increase the wire diameter as well if you go that route. Running larger breakers on 10A or 15A wiring is a recipe for bad stuff.
viraptor 15 hours ago [-]
> You can always have an electrician install ...
If you own the house, sure. Many people don't.
mrweasel 6 hours ago [-]
In older houses, made from brick and concrete, that can be tricky to do. The only reason I can have my computer on a separate circuit is because we could repurpose the old three phase wiring for a sauna we ripped out. If that had not been the case, getting the wires to the fuse board would have been tricky at best.
New homes are probably worse than old homes through. The wires a just chucked in the space been the outer and inner walls, there's basically no chance of replacing them of pulling new ones. Old houses at least frequently have piping in which the wires run.
glitchc 9 hours ago [-]
Without upgrsding the wiring to a thicker gauge? That's not code compliant and is likely to cause a fire.
davrosthedalek 17 hours ago [-]
Larger breaker and thicker wires!
atonse 17 hours ago [-]
I thought you only needed thicker wires for higher amps? Should go without saying, but I am not a certified electrician :-)
I only have a PhD from YouTube (Electroboom)
jchw 16 hours ago [-]
The voltage is always going to be the same because the voltage is determined by the transformers leading to your service panel. The breakers break when you hit a certain amperage for a certain amount of time, so by installing a bigger breaker, you allow more amperage.
If you actually had an electrician do it, I doubt they would've installed a breaker if they thought the wiring wasn't sufficient. Truth is that you can indeed get away with a 20A circuit on 14 AWG wire if the run is short enough, though 12 AWG is recommended. The reason for this is voltage drop; the thinner gauge wire has more resistance, which causes more heat and voltage drop across the wire over the length of it, which can cause a fire if it gets sufficiently hot. I'm not sure how much risk you would put yourself in if you were out-of-spec a bit, but I wouldn't chance it personally.
bangaladore 15 hours ago [-]
Could you not just run a 240 volt outlet on existing wiring built for 110v? Just send l1 and l2 on the existing hot/neutral?
bri3d 14 hours ago [-]
You can, 240V on normal 12/2 Romex is fine. The neutral needs to be "re-labeled" with tape at all junctions to signify that it's hot, and then this practice is (generally) even code compliant.
However! This strategy only works if the outlet was the only one on the circuit, and _that_ isn't particularly common.
jchw 12 hours ago [-]
Although this exists, as a layperson, I've rarely seen it. There is the NEMA 6-15R receptacle type, but I have literally none of those in my entire house, and I've really never seen them. Apparently they're sometimes used for air conditioners. Aside from the very common 5-15R, I see 5-20R (especially in businesses/hospitals), and 14-30R/14-50R for ranges and dryers. (I have one for my range, but here in the midwest electric dryers and ranges aren't as common, so you don't always come across these here. We have LNG ran to most properties.) So basically, I just really don't see a whole lot of NEMA 6 receptacles. The NEMA 14 receptacles, though, require both hots and the neutral, so in a typical U.S. service panel it requires a special breaker and to take up two slots, so definitely not as simple of a retrofit.
(Another outlet type I've seen: I saw a NEMA 7 277V receptacle before. I think you get this from one phase of a 480V three-phase system, which I understand is ran to many businesses.)
bryanlarsen 12 hours ago [-]
If you drive an electric car in a rural area you might want to carry around 6-30 and 6-50 adapters because most farms have welders plugged into those and that can give you a quick charge. And also TT-30 and 14-50 adapters to plug in at campgrounds.
bri3d 11 hours ago [-]
NEMA 6s are extremely common in barns and garages for welders. 6-50 is more common for bigger welders but I’ve also seen 6-20s on repurposed 12/2 Romex as the parent post was discussing used for cheap EV retrofits, compressors, and welders.
esseph 10 hours ago [-]
5-20R/6-20R is also somewhat commonly used by larger consumer UPS for your computer, router, etc.
wat10000 4 hours ago [-]
NEMA 6 is limiting because there’s no neutral, so everything in the device has to run on 240V. Your oven and dryer want 120V to run lights and electronics, so they use a 14 (or 10 for older installs) which lets them get 120V between a hot and the neutral.
Oddly, 14-50 has become the most common receptacle for non-hardwired EV charging, which is rather wasteful since EV charging doesn’t need the neutral at all. 6-50 would make more sense there.
cosmic_cheese 17 hours ago [-]
Where things get hairy are old houses with wiring that’s somewhere between shaky and a housefire waiting to happen, which are numerous.
jchw 16 hours ago [-]
As an old house owner, I can attest to that for sure. In fairness though, I suspect most of the atrocities occur in wall and work boxes, as long as your house is new enough to at least have NM sheathed wiring instead of ancient weird stuff like knob and tube. That's still bad but it's a solvable problem.
I've definitely seen my share of scary things. I have a lighting circuit that is incomprehensibly wired and seems to kill LED bulbs randomly during a power outage; I have zero clue what is going on with that one. Also, often times opening up wall boxes I will see backstabs that were not properly inserted or wire nuts that are just covering hand-twisted wires and not actually threaded at all (and not even the right size in some cases...) Needless to say, I should really get an electrician in here, but at least with a thermal camera you can look for signs of serious problems.
kube-system 17 hours ago [-]
Yeah, but it ain't nothing that microwaves, space heaters, and hair dryers haven't already given a run for their money.
jchw 16 hours ago [-]
Hair dryers and microwaves only run for a few minutes, so even if you do have too much resistance this probably won't immediately reveal a problem. A space heater might, but most space heaters I've come across actually seem to draw not much over 1,000 watts.
And even then, even if you do run something 24/7 at max wattage, it's definitely not guaranteed to start a fire even if the wiring is bad. Like, as long as it's not egregiously bad, I'd expect that there's enough margin to cover up less severe issues in most cases. I'm guessing the most danger would come when it's particularly hot outside (especially since then you'll probably have a lot of heat exchangers running.)
chronogram 18 hours ago [-]
That's still not much for wiring in most countries. A small IKEA consumer oven is only 230V16A=3860W. Those GPUs and CPUs only consume that much at max usage anyway. And those CPUs are uninteresting for consumers, you only need a few Watts for a single good core, like a Mac Mini has.
rbanffy 15 hours ago [-]
> And those CPUs are uninteresting for consumers, you only need a few Watts for a single good core, like a Mac Mini has.
Speak for yourself. I’d love to have that much computer at my disposal. Not sure what I’d do with it. Probably open Slack and Teams at the same time.
ThunderSizzle 12 hours ago [-]
> Probably open Slack and Teams at the same time.
Too bad it feels like both might as well be single threaded applications somehow
dv_dt 18 hours ago [-]
So Europe ends up with an incidental/accidental advantage in the AI race?
atonse 17 hours ago [-]
All American households get mains power at 240v (I'm missing some nuance here about poles and phases, so the electrical people can correct my terminology).
It's often used for things like ACs, Clothes Dryers, Stoves, EV Chargers.
So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
kube-system 17 hours ago [-]
To get technical -- US homes get two phases of 120v that are 180 degrees out of phase with the neutral. Using either phase and the neutral gives you 120v. Using the two out of phase 120v phases together gives you a difference of 240v.
Even more technical, we don't have two phases, we have 1-phase that's split in half. I hate it cause it makes it confusing.
Two phase power is not the same as split phase (There's basically only weird older installations of 2 phase in use anymore).
kube-system 17 hours ago [-]
Yeah that's right. The grid is three phases (as it is basically everywhere in the world), and the transformer at the pole splits one of those in half. Although, what are technically half-phases are usually just called "phases" when they're inside of a home.
That's such a great video, like most of his stuff.
the8472 15 hours ago [-]
If we're counting all the phases then european homes get 400V 3-phase, not 240V split-phase.
Not that typical residential connections matter to highend servers.
bonzini 15 hours ago [-]
It depends on the country, in many places independent houses get a single 230V phase only.
ender341341 17 hours ago [-]
> So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
It'd be all new wire run (120 is split at the panel, we aren't running 240v all over the house) and currently electricians are at a premium so it'd likely end up costing a thousand+ to run that if you're using an electrician, more if there's not clear access from an attic/basement/crawlspace.
Though I think it's unlikely we'll see an actual need for it at home, I imaging a 800w cpu is going to be for server class CPUs and rare-ish to see in home environments.
com2kid 15 hours ago [-]
> and currently electricians are at a premium so it'd likely end up costing a thousand+
I got a quote for over 2 thousand to run a 24v line literally 9 feet from my electrical panel across my garage to put a EV charger in.
Opening up an actual wall and running it to another room? I can only imagine the insane quotes that'd get.
Marsymars 10 hours ago [-]
I kinda suspect there’s a premium once you mention “EV vehicle”, since you’re signalling that you’re affluent enough to afford an EV and have committed to spending the money required to get EV charging at home working, etc. (Kinda like getting a quote for anything wedding related.)
I’m getting some wiring run about the same distance (to my attic, fished up a wall, with moderately poor access) for non-EV purposes next week and the quote was a few hundred dollars.
qotgalaxy 9 hours ago [-]
[dead]
tguvot 8 hours ago [-]
the trick is to request 240v outlet for welder. it brings price down to 400 or so.
running to another room will be done usually (at least in usa) through attic or crawlspace. i got it done a few months ago to have dedicated 20A circuit (for my rack) in my work room. cost was around 300-400 as well
com2kid 6 hours ago [-]
Labor chargers alone are going to be higher than that in Seattle. Just to have someone come out on a call is going to be 150-200. If it is an independent electrician who owns their own business, maybe 100-150/hr, if they are part of a larger company, I'd expect even more than that.
Honestly I wouldn't expect to pay less than $1000 for the job w/o any markups.
tguvot 6 hours ago [-]
i live in bay area. i have some doubts that seattle going to be more expensive.
com2kid 6 hours ago [-]
Handy man prices around here are $65 to $100/hr, and there is a huge wait list for the good ones.
I've gotten multiple quotes on running the 240v line, the labor breakdown was always over $400 alone. Just having someone show up to do a job is going to be almost $200 before any work is done.
When I got quotes from unlicensed people, those came in around $1000 even.
vel0city 15 hours ago [-]
I don't think many people would want some 2kW+ system sitting on their desk at home anyways. That's quite a space heater to sit next to.
Tor3 8 hours ago [-]
I should look at the label (or check with a meter..), but when I run my SGI Octane with its additional XIO SCSI board in active use, the little "office" room gets very hot indeed.
bonzini 15 hours ago [-]
Also the noise from the fans.
dv_dt 17 hours ago [-]
Well yes its possible but often $500-1000 to run a new 240v outlet, and that's to a garage for an ev charger. If you want an outlet in the house I dont know how much wall people want to tear up and extra time and cost.
atonse 17 hours ago [-]
Sure yeah, I was just clarifying that if the issue is 240v, etc, US houses have the feed coming in. Infrastructure-wise it's not an issue at all.
kube-system 17 hours ago [-]
Consumers with desktop computers are not winning any AI race anywhere.
buckle8017 17 hours ago [-]
In residential power delivery? yes
In power cost? no
I'm literally any other way? also no
carlhjerpe 15 hours ago [-]
In the Nordics we're on 10A for standard wall outlets so we're stuck on 2300W without rewiring (or verifying wiring) to 2.5mm2.
We rarely use 16A but it exists. All buildings are connected to three phases so we can get the real juice when needed (apartments are often single phase).
I'm confident personal computers won't reach 2300W anytime soon though
Tor3 8 hours ago [-]
In the Nordics (I'm assuming you mean Nordic countries) 10A is _not_ standard. Used to be, some forty years ago. Since then 16A is standard. My house has a few 10A leftovers from when the house was built, and after the change to TN which happened a couple of decades ago, and with new "modern" breakers, a single microwave oven on a 10A circuit is enough to trip the breaker (when the microwave pulses). Had to get the breakers changed to slow ones, but even those can get tripped by a microwave oven if there's something else (say, a kettle) on the same circuit.
16A is fine, for most things. 10A used to be kind of ok, with the old IT net and old-style fuses. Nowadays anything under 16A is useless for actual appliances. For the rest it's either 25A and a different plug, or 400V.
carlhjerpe 5 hours ago [-]
Let's rephrase: 10A is the effective standard that's been in use for a long long time, if you walk into a building you can assume it has 10A breakers.
On new installations you can choose 10A or 16A so if you're forward thinking you'd go 16 since it gives you another 1300 watts to play with.
bonzini 15 hours ago [-]
In Italy we also have 10A and 16A (single phase). In practice however almost all wires running in the walls are 2.5 mm^2, so that you can use them for either one 16A plug or two adjacent 10A plugs.
nordcikmgsdf 3 hours ago [-]
[dead]
jacquesm 5 hours ago [-]
You can up the voltage to 240 and re-use the wiring (with some minor mods to the ends), for double the power. Insulation class should be sufficient. That makes good sense anyway. You may still have an issue if the powersupply can't handle 240/60 but for most of the ones that I've used that would have worked. Better check with the manufacturer to be sure though. It's a lot easier and faster than rewiring.
tracker1 18 hours ago [-]
There already are different outlets for these higher power draw beasts in data centers. The amount of energy used in a 4u "AI" box is what an entire rack used to draw. Data centers themselves are having to rework/rewire areas in order to support these higher power systems.
t0mas88 18 hours ago [-]
A simple kitchen top water cooker is 2000W, so a 1500W PC sounds like no big deal.
kube-system 17 hours ago [-]
Kettles in the US are usually 1500W, as the smallest branch circuits in US homes support 15A at 120V and the general rule for continuous loads is to be 80% of the maximum.
linotype 17 hours ago [-]
True but kettles rarely run for very long.
kube-system 17 hours ago [-]
But computers do, which was why I included that context. You don't really want to build consumer PC >1500W in the US or you'd need to start changing the plug to patterns that require larger branch circuits.
CyberDildonics 16 hours ago [-]
Kettles and microwaves are usually 1100 watts and lower, but space heaters and car chargers can be 1500 watts and run for long periods of time.
Tor3 8 hours ago [-]
Microwave ovens have a different issue, which I found when I upgraded my breaker board to a modern one in my house. The startup pulse gives a type of load which trips a standard A-type 10A breaker (230V). Had to get those changed to a "slow" type, but even that will trip every blue moon, and if there's something else significant on the same circuit the microwave oven will trip even so, every two weeks or so (for the record, I have several different types of microwave ovens around the house, and this happens everywhere there's a 10A circuit).
The newer circuits in the house are all 16A, but the old ones (very old) are 10A. A real pain, with new TN nets and modern breakers.
wtallis 12 hours ago [-]
Microwave ovens top out around 1100-1250W output from a ~1500W input from the wall. Apparently there's a fair bit of energy lost in the power supply and magnetron that doesn't make it into the box where the food is.
It is mostly an issue in countries with 120V mains (I know that in the US 240V outlets exist though).
In France for example it is required that standard plugs must be able to deliver at least 16A on each outlet, at the 230V used here, we get 3600W of power, that’s more than enough.
esseph 10 hours ago [-]
Yes and this is something I've been thinking about for awhile.
A computer is becoming a Home Appliance in the it will need 20A wiring and plugs soon, but should move to 220/240v soon anyway (and change the jumper on your standard power supply).
orra 18 hours ago [-]
Laughs in 230V (sorry).
AnthonBerg 5 hours ago [-]
ʰᵉₕₑheʰᵉₕₑhe in 400V
derefr 17 hours ago [-]
But all of the most-ridiculous hyperscale deployments, where bandwidth + latency most matter, have multiple GPUs per CPU, with the CPU responsible for splitting/packing/scheduling models and inference workloads across its own direct-attached GPUs, providing the network the abstraction of a single GPU with more (NUMA) VRAM than is possible for any single physical GPU to have.
How do you do that, if each GPU expects to be its own backplane? One CPU daughterboard per GPU, and then the CPU daughterboards get SLIed together into one big CPU using NVLink? :P
wmf 15 hours ago [-]
GPU as motherboard really only makes sense for gaming PCs. Even there SXM might be easier.
db48x 13 hours ago [-]
No, for a gaming computer what we need is the motherboard and gpu to be side by side. That way the heat sinks for the CPU and GPU have similar amounts of space available.
For other use cases like GPU servers it is better to have many GPUs for every CPU, so plugging a CPU card into the GPU doesn’t make much sense there either.
It’s always going to be a back and forth on how you attach stuff.
Maybe the GPU becomes the motherboard and the CPU plugs into it.
avgeek23 4 days ago [-]
And the memory should be a onboard module on the cpu card intel/amd should replicate what apple did with a unified same ringbus sort of memory modules. Lower latency,higher throughput.
Would push performance further. Although companies like intel would bleed the consumer dry with, a certain i5-whatever cpu with onboard memory of 16 gigs could be insanely priced compared to what you'd pay for addon memory.
0x457 16 hours ago [-]
That would pretty much make both intel and amd to start market segmentation by CPU Core + Memory combination. I absolutely do not want that.
iszomer 1 hours ago [-]
I wonder how many additional layers are required in the PCB to achieve this + how this will dramatically affect the TDP; the GPU's aren't the only components with heat tolerance and capacitance.
bgnn 6 hours ago [-]
EE here. There's no reason to not deliver power directly to the GPU by using cables. I'm not sure if it's sooving anything.
But you are right, there's no hierarchy in the systems anymore. Why do we even call something a motherboard? There's a bunch of chips interconnected.
0manrho 12 hours ago [-]
We're already there. That's what a lot of people are using DPU's are for.
PCI-e Networks and CXL are the future of many platforms... like ISA backplanes.
0manrho 10 hours ago [-]
Yep, I have a lot of experience with CXL devices and networked PCIe/NVMe (over Eth/IB) Fabrics and deploying "Headless"/"Micro-Head" compute units which are essentially just a pair of DPU's on a PCIe multiplexer (basically just a bunch of PCIe slots tied to a PCIe Switch or two).
That said my experience in this field is more with storage than GPU compute, but I have done some limited hacking about in the GPGPU space with that tech as well. Really fascinating stuff (and often hard to keep up with and making sure every part in the chain supports the features you want to leverage, not to mention going down the PCIe root topology rabbit hole and dealing with latency/trace-length/SnR issues with retimers vs muxers vs etc etc etc).
It's still a nascent field that's very expensive to play in, but I agree it's the future of at least part of the data infrastructure field.
Really looking forward to finally getting my hands on CXL3.x stuff (outside of a demo environment.)
pshirshov 15 hours ago [-]
Can I just have a backplane? Pretty please?
theandrewbailey 14 hours ago [-]
I've wondered why there hasn't been a desktop with a CPU+RAM card that slots into a PCIe x32 slot (if such a thing could exist), or maybe dual x16 slots, and the motherboard could be a dumb backplane that only connected the other slots and distributed power, and probably be much smaller.
KeplerBoy 2 hours ago [-]
PCIe x 32 actually exists, at least in the specification. I have never seen a picture of a part using it.
namibj 4 hours ago [-]
Those exist; they are used for risers ("vertical mount GPU brackets, for dual GPU" equivalent for servers, where they make the cards flat again).
iszomer 1 hours ago [-]
Retimers.
guerrilla 2 hours ago [-]
Yes, for fucks sake, this is the only way forward. It gives us the ultimate freedom to do whatever we want in the future. Just make everything a card on the bus and quit with all this hierarchy nonsense.
colejohnson66 15 hours ago [-]
Sockets (and especially backplanes) are absolutely atrocious for signal integrity.
pshirshov 14 hours ago [-]
I guess if it's possible to have 30cm PCIe 5 riser cables, it should be possible to have a backplane with traces of similar length.
namibj 4 hours ago [-]
Cables much better sadly, so much so that they started to use cables to jump across the server main board in places.
If I remember correctly the military / aerospace shy away from this spec because the connector with the pins is on the backplane, with the sockets on the cards.
So if you incorrectly insert a card and bend a pin you're in trouble.
VPX has the sockets on the backplane so avoids this issue, if you bend pins you just grab another card from spares.
This may have changed since I last looked at it.
Telecoms industry definitely seem to favour TCA though.
dylan604 15 hours ago [-]
Wouldn't that mean an complete mobo replacement to upgrade the GPU? GPU upgrades seem much more rapid and substantial compared to CPU/RAM. Each upgrade would now mean taking out the CPU/RAM and other cards vs just replacing the GPU
p1esk 15 hours ago [-]
GPUs completely dominate the cost of a server, so a GPU upgrade typically means new servers.
BobbyTables2 15 hours ago [-]
Agree - newer GPU likely will need faster PCIe speeds too.
Kinda like RAM - almost useless in terms of “upgrade” if one waits a few years. (Seems like DDR4 didn’t last long!)
chrismorgan 10 hours ago [-]
> GPU upgrades seem much more rapid and substantial compared to CPU/RAM.
I feel like I’ve been hearing about people selling five-to-ten-year-old GPUs for sometimes as many dollars as they bought them for, for the last five years; and people choosing to stay on 10-series NVIDIA cards (2016) because the similar-cost RTX 30-, 40- or 50-series was actually worse, because they’d been putting the effort and expense into parts of the chips no one actually used. Dunno, I don’t dGPU.
j16sdiz 2 hours ago [-]
It is not a EE problem. It is an ecosystem problem.
You need a whole catalog of compatible hardware for this.
MurkyLabs 18 hours ago [-]
Yes I agree, let's bring back the SECC style CPU's from the Pentium Era, I've still got my Pentium II (with MMX technology)
Dylan16807 17 hours ago [-]
And limit yourself to only one GPU?
Also CPUs are able to make use of more space for memory, both horizontally and vertically.
I don't really see the power delivery advantages, either way you're running a bunch of EPS12V or similar cables around.
14 hours ago [-]
coherentpony 11 hours ago [-]
The concept exists now. You can "reverse offload" work to the CPU.
leoapagano 16 hours ago [-]
One possible advantage of this approach that no one here has mentioned yet is that it would allow us to put RAM on the CPU die (allowing for us to take advantage of the greater memory bandwidth) while also allowing for upgradable RAM.
themafia 8 hours ago [-]
I think you'd want to go the other way.
GPU RAM is high speed and power hungry. So there tends to not be very much of it on the GPU card. This is part of the reason we keep increasing the bandwidth is so the CPU can touch that GPU RAM at the highest speeds.
It makes me wonder though if a NUMA model for the GPU is a better idea. Add more lower power and lower speed RAM onto the GPU card. Then let the CPU preload as much data as is possible onto the card. Then instead of transferring textures through the CPU onto the PCI bus and into the GPU why not just send a DMA request to the GPU and ask it to move it from it's low speed memory to it's high speed memory?
It's a whole new architecture but it seems to get at the actual problems we have in the space.
kokada 6 hours ago [-]
Isn't that what you described Direct Storage?
themafia 5 hours ago [-]
You're still running through the PCIe slot and it's bandwidth limit. I'm suggesting you bypass even that and put more memory directly on the card.
KeplerBoy 2 hours ago [-]
So an additional layer slower and larger than global GPU memory?
I believe that's kind of what bolt graphics is doing with the dimm slots next to the soldered on lpddr5.
https://bolt.graphics/how-it-works/
MBCook 10 hours ago [-]
Couldn’t we do that today if we wanted to?
What’s keeping Intel/AMD from putting memory on package like Apple does other than cost and possibly consumer demand?
iszomer 1 hours ago [-]
Supply + demand, the manufacturing-capacity rabbit hole.
burnt-resistor 17 hours ago [-]
Figure out how much RAM, L1-3|4 cache, integer, vector, graphics, and AI horsepower is needed for a use-case ahead-of-time and cram them all into one huge socket with intensive power rails and cooling. The internal RAM bus doesn't have to be DDRn/X either. An integrated northbridge would deliver PCIe, etc.
Razengan 18 hours ago [-]
Isn't that what has kinda sorta basically happened with Apple Silicon?
trenchpilgrim 15 hours ago [-]
And AMD Strix Halo.
MBCook 16 hours ago [-]
GPU + CPU on the same die, RAM on the same package.
A total computer all-in-one. Just no interface to the world without the motherboard.
LeoPanthera 17 hours ago [-]
Bring back the S100 bus and put literally everything on a card. Your motherboard is just a dumb bus backplane.
MBCook 16 hours ago [-]
We were moving that way, sorta, with Slot 1 and Slot A.
Then that became unnecessary when L2 cache went on-die.
bhouston 18 hours ago [-]
I love the PCIe standard is 3 generations ahead of what is actually released. Gen5 is the live version, but the team behind it is so well organized that they have a roadmap of 3 additional versions now. Love it.
zamadatix 14 hours ago [-]
"3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today, PCIe 7.x was finalized last month, and this is an announcement work on PCIe 8.0 has started for release in 3 years. I.e. it has only been one month of being one generation behind the latest PCIe revision.
It'll be interesting if consumer devices bother trying to stay with the latest at all anymore. It's already extremely difficult to justify the cost of implementing PCIe 5.0 when it makes almost no difference for consumer use cases. The best consumer use case so far is enthusiasts who want really fast NVMe SSDs in x4 lanes, but 5.0 already gives >10 GB/s for a single drive, even with the limited lane count. It makes very little difference for x16 GPUs, even with the 5090. Things always creep up over time, but the rate at which the consumer space creeps is just so vastly different from what the DC space has been seeing that it seems unreasonable to expect the two to be lockstep anymore.
_zoltan_ 14 hours ago [-]
either I'm misinformed in which case is like to see proof, or you are, but GB200 is not Gen6, just Gen5 and this will only be rectified with GB300 which, while exists in the wild, is not in the millions.
so indeed the parent commenter would be correct that everything is Gen5 right now.
that article is from last March. the plan was indeed for the GB200 board to support gen6 but it didn't happen and GB300 will rectify it.
at least that's my best educated guess, looking at supermicro's public spec sheet and that it's shipping with CX7 which is Gen5 and not with CX8.
B200 supports Gen6 there is just nothing that would let it run at Gen6.
zamadatix 13 hours ago [-]
At least the GB200s OpenAI purchased have CX-8s, but it's very possible (perhaps even very likely) most of the millions of GB200s deployed PCIe 5.0 using CX-7s given the SuperMicro spec sheet.
thanks! very interesting that they got a custom SKU.
jsolson 10 hours ago [-]
Datacenter Blackwell is Gen6, which is critical when pairing it with CX8 (2x400G) as otherwise you'd be stranding NIC BW.
_zoltan_ 1 hours ago [-]
please link me a B200 (not a B300) system with CX8 that uses gen6 x16 instead of gen5 x32.
to the best of my knowledge this does not exist, but I'd be happy to stand corrected.
(the official NVIDIA DGX B200 is gen5).
Melatonic 13 hours ago [-]
CPUs themselves only have so many PCI-E lanes though right? Wouldnt it make sense (even for consumers) to have peripherals using less lanes (but more speed per lane) for a multi GPU system or something that uses a lot of drives?
zamadatix 12 hours ago [-]
More lanes = more cost
Faster lanes = more cost
More faster lanes = lots more cost
The chipset also strikes some of the balance for consumers though. It has a narrow high speed connection to the CPU but enables many lower speed devices to share that bandwidth. That way you can have your spare NVMe drive, SATA controller, wired and wireless NICs, sound hardware, most of your USB ports, your capture card, and some other random things connected over a single x4 to x8 sized channel. This leaves the high cost lanes for just the devices that actually use them (GPU and primary, possibly secondary, storage drive). I've got one consumer type Motherboard with 14 NVMe drives connected, for example, just not at full native speed directly to the CPU.
You're just SoL if you want to connect a bunch of really high bandwidth devices simultaneously (100 Gbps+ NICs, multiple GPUs at full connection speed, a dozen NVMe drives at native speed, or similar) because then you'll be paying for a workstation/server class platform which did make the "more faster lanes" tradeoff (plus some market segment gouging, of course).
vladvasiliu 6 hours ago [-]
One issue is that, at least on cheaper mobos, these don't work as a "total bandwidth budget" situation. And, especially with newer generation PCIe, it can be a bit frustrating.
Many mobos will operate the available slots such that the total number of active lanes is split between them. But if you use older-generation cards, you'll only get a fraction of the available bandwidth because you're only using a fraction of their lanes, although the physical lanes are physically present.
What I'm thinking about is something like, say, a pair of Gen3 NVMe drives that are good enough for mass storage (running in RAID-1 for good measure) and some cheap used 10 Gb NIC, which will probably be 8x Gen2, all running on a Gen4+ capable mobo.
And, while for a general-purpose setup I can live with splitting available BW between the NIC and the GPU (I most likely don't care about my download going super fast while I game), the downloads will generally go to the storage, so they must be fast at the same time.
michaelt 12 hours ago [-]
> "3 generations" seems like a bit of a stretch. Millions of Blackwell systems use PCIe 6.x today
True. And yet, if you buy an RTX 5090 today, costing $2400 and released in January this year, it's PCIe 5.0 x16
zamadatix 12 hours ago [-]
Does the second half not already talk to this? Adding on to it further, GN found a few % performance difference on a 5090 going from PCIe 3.0 to 5.0 https://gamersnexus.net/gpus/nvidia-rtx-5090-pcie-50-vs-40-v... meanwhile many DC use cases find PCIe 5.0 unusably slow, driving alternative (non-PCIe) interconnects.
tails4e 17 hours ago [-]
It takes a long time to get form standard to silicon, so I bet there are design teams working on pcie7 right now, which won't see products for 2 or more years
Seattle3503 17 hours ago [-]
Is there an advantage of getting so far ahead of implementations? It seems like it would be more difficult to incorporate lessons.
kvemkon 16 hours ago [-]
When AMD introduces a new Desktop CPU series IIRC they claim the next generation design is (almost) finished (including layout?) and they start with the next-next-gen design. And I'm also asking the same question. But more than a half a year before the CPU becomes available to the public it is already being tested by partners (mainboard manufacturers and ?).
jsolson 10 hours ago [-]
This actually makes sense from a spec perspective if you want to give enough to allow hardware to catch up with the specs and to support true interop.
Contrast this with the wild west that is "Ethernet" where it's extremely common for speeds to track well ahead of specs and where interop is, at best, "exciting."
ThatMedicIsASpy 17 hours ago [-]
Gen6 is in use look at Nvidia ConnectX-8
drewg123 17 hours ago [-]
What hosts support Gen6? AFAIK, Gen5 is the most recent standard that's actually deployed. Eg, what can you plug a CX8 into that will link up at Gen6?
ksec 6 hours ago [-]
Nvidia only for now. Precisely because they got fed up with Intel and AMD ( And Amphere as well ) for extremely slow PCIe adoption.
I believe only next Gen Intel and AMD Zen6 will get PCIe 6.0.
I am hoping Nvidia officially move into Server CPU market not only for their CPU but for wider Web Hosting as well. More competition for Server Hardware.
_zoltan_ 13 hours ago [-]
into a Gen6 Bianca board (baseboard for GB300). that's it today.
how you use those today is twofold:
- Gen5 x32 via two x16 slots (this is how most people use them)
while the B200 chip itself could do PCIe6 as it was planned for GB200, there is no system around it with Gen6. the official DGX B200 is just PCIe5.
GB300 is indeed Gen6.
triknomeister 17 hours ago [-]
Custom Nvidia network cards I guess.
Phelinofist 16 hours ago [-]
So we can skip 6 and 7 and go directly to 8, right?
robotnikman 16 hours ago [-]
I know very little about electronics design, so I always find it amazing that they keep managing to double PCIe throughput over and over. Its also probably the longest lived expansion bus at the moment.
wmf 15 hours ago [-]
It's less surprising if you realize that PCIe is behind Ethernet (per lane).
extraduder_ire 5 hours ago [-]
Which Ethernet standard? Light online searching leads me to believe it tops out at 800Gb/s. (and not in a format I would normally call Ethernet)
My reaction to PCIe gen 8 is essentially "Huh? No, retro data buses are like ISA, PCI, and AGP, right? PCIe Gen 3 and SATA are still pretty new...".
I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.
Dylan16807 17 hours ago [-]
> PCIe Gen 3 and SATA are still pretty new...
That's an interesting thought to look at. PCIe 3 was a while ago, but SATA was nearly a decade before that.
> I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.
Wikipedia says it's planned to be PAM4 just like 6 and 7.
Gen 5 and 6 were 32 gigabaud. If 8 is PAM4 it'll be 128 gigabaud...
eqvinox 17 hours ago [-]
I'd highly advise against using GHz here (without further context, at least), a 32Gbaud / 32Gsym/s NRZ signal toggling at full rate is only a 16GHz square wave.
baud seems out of fashion, sym/s is pretty clear & unambiguous.
(And if you're talking channel bandwidth, that needs clarification)
kvemkon 17 hours ago [-]
> > I think Gen7 used 32GHz, which is ridiculously high.
> 16GHz square wave
Is it for PCIe 5.0? PCIe 6.0 should operate on the same frequency and doubling the bandwidth by using PAM4. If PCIe 7.0 doubled the bandwidth and is still PAM4, what is the underlying frequency?
eqvinox 17 hours ago [-]
PCIe 7 = 128 GT/s = 64 Gbaud × PAM-4 = 32 "GHz" (if you alternate extremes on each symbol)
for gen6, halve all numbers
Dylan16807 17 hours ago [-]
Is it me or are they using the term GigaTransfers wrong? They're counting a single PAM4 pulse as two "transfers".
eqvinox 17 hours ago [-]
They kinda are and kinda aren't, they're just using their own definition…
(I'm accepting it because "Transfers"/"T" as unit is quite rare outside of PCIe)
zamalek 15 hours ago [-]
GT/s is also gaining ground for system RAM in order to clear up the ambiguity that DDR causes for end-consumers.
Dylan16807 14 hours ago [-]
And it's a good way to remove the ambiguity of things like DDR, but ugh "transfers" is not the best word here.
Looking at some documents from Micron I don't see them using GT/s anywhere. And in particular if I go look at their GDDR6X resources because those chips use PAM4, it's all about gigabits per second [per pin]. So for example 6GHz data clock, 12Gbaud, 24Gb/s/pin.
mjevans 7 hours ago [-]
Would you rather go back to the modem days and call a 'Transfer' a 'Baud'?
PAM encoding is already analog, and also correspondingly more expensive (power, silicon size, etc) for the increase in speed.
It really wouldn't surprise me if even on workstation platforms only a subset of core lanes were Gen6+ and the common slots were redriven Gen5 or less off of a router / switch chip.
Dylan16807 6 hours ago [-]
> Would you rather go back to the modem days and call a 'Transfer' a 'Baud'?
We don't have to go back, baud is still in use. I would expect transfers per second to be a synonym for baud though, and for bits per second per pin to use a different word.
wtallis 7 hours ago [-]
Aside from GDDR for GPUs, DRAM is still mostly specified with MT/s rather than GT/s, probably because marketing prefers bigger numbers. It'll probably fall off once 5-digit numbers become commonplace.
guerrilla 17 hours ago [-]
> baud seems out of fashion, sym/s is pretty clear & unambiguous.
Huh? Baud is sym/s.
eqvinox 17 hours ago [-]
Yes, that was the implication, but I've been getting the impression that using baud is kinda unpopular compared to using sym/s.
throwway120385 15 hours ago [-]
A lot of people think that baud rate represents bits per second, which it only does in systems where the symbol set is binary. People got it from RS232.
rbanffy 15 hours ago [-]
IIRC, modems never went much beyond 2400 baud. Everything past that was clever modulation packing more bits onto a single symbol.
guerrilla 4 hours ago [-]
Huh, I never thought of it that way but you're right.
weinzierl 15 hours ago [-]
Don't forget VESA Local Bus.
bpbp-mango 10 hours ago [-]
I wonder if this will help applications like VPP/DPDK. not sure if the CPU or the lanes are the bottleneck there.
iFred 6 hours ago [-]
I can't be the only one let down that there wasn't some new slot design. Something with pizzazz and flare.
pshirshov 15 hours ago [-]
Yeah, such a shame I've just upgraded to a 7.0 motherboard for my socket AM7 CPU.
Being less sarcastic, I would ask if 6.0 mobos are on the horizon.
wmf 15 hours ago [-]
I guess Venice, Diamond Rapids, and Vera will have 6.0.
_zoltan_ 13 hours ago [-]
GB300 has Gen6 so I am pretty sure GV200 would have it at least.
I'm fairly sure they are cooking Gen7 already into CX9.
LeoPanthera 17 hours ago [-]
I thought we were only just up to 5? Did we skip 6 and 7?
pkaye 16 hours ago [-]
Some of the newer ones maybe more for data centers.
ThatMedicIsASpy 18 hours ago [-]
I'll take it if my consumer mb chipset supports giving me 48 PCIe7 lanes if future desktops still would only come with 24 gen 8 lanes
_zoltan_ 13 hours ago [-]
what I don't get: why doesn't AMD just roll Gen6 out in their CPU, bifurcate it to Gen5, and boom, you have 48x2 Gen5s? same argument for gen5 bifurcated to gen4.
this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
zamadatix 13 hours ago [-]
Bifurcation can't create new lanes, only split the existing lane count up into separate logical slots.
What you're saying is possible though, you just need something a little heavier like a PCIe switch do the lane count + mixed-speed conversion magic. That's exactly what the chipset is, but for various reasons it's still only PCIe 4.0, even on the latest generation chips. I wouldn't be surprised if that changed again next generation. The downsides of this approach are switches add cost, latency, and can consume a lot of power. When they first upped the chipset to be a PCIe 4.0 connection in the 500 era, most motherboards capable of using the bandwidth of the chipset actually had to add chipset cooling.
Ideally they'd just add alternative options with more lanes directly from the CPU, but that'd add competition to the bottom of the Threadripper line.
michaelt 12 hours ago [-]
> this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
Is that a problem?
These days, you don't need slots for your sound card and network card, that stuff's all integrated on the motherboard.
Plenty of enthusiast motherboards support 1 GPU, 3-4 nvme drives, 4 SATA drives, and have a PCIe 1x slot or two to spare.
Is anyone struggling, except folks working with LLMs? Seems to me folks looking to put several $2400 RTX 4090s in one machine ain't exactly a big market. And they'll probably want a giant server board, so they have space for all their giant four-slot-wide cards.
mrene 12 hours ago [-]
Devices integrated on the motherboard will either connect to the cpu via PCIe or USB, lanes aren't just for PCIe cards.
For example, motherboards with multiple nvme drives often have 1 drive with dedicated lanes and the remainder multiplexed through a PCIe switch embedded in the chipset.
Melatonic 13 hours ago [-]
I feel like what we really need is a GPU "socket" like we have for CPU's. And then a set of RAM slots dedicated to that GPU socket (or unified RAM shared between CPU and GPU)
Lramseyer 12 hours ago [-]
Unfortunately socketed processors only really work with DDRx type DRAM interfaces. GPUs use GDDR and HBM interfaces, which are not ideal for sockets. In the case of HBM, you have 1024 data traces per DRAM chip, which would make the socket have an insane number of pins. GDDR has fewer pins, but makes up for it with higher data rates (32 pins at 16Gb/s) and that is impractical to use in a socket due to the variance in contact area resulting in impedance matching issues.
Sephr 12 hours ago [-]
HBM would most likely have to be integrated directly on the GPU module given performance demands and signal constraints.
richwater 18 hours ago [-]
Meanwhile paying a premium for a Gen5 motherboard may net you somewhere in the realm of 4% improvements in gaming if you're lucky.
Obviously PCI is not just about gaming but...
checker659 15 hours ago [-]
No matter the leaps in bandwidth, the latency remains the same. Also, with PCIe switches used in AI servers, the latency (and jitter) is even pronounced.
simoncion 17 hours ago [-]
From what I've seen, the faster PCI-E bus is important when you need to shuffle things in and out of VRAM. In a video game, the faster bus reduces the duration of stutters caused by pushing more data into the graphics card.
If you're using a new video card with only 8GB of onboard RAM and are turning on all the heavily-advertised bells and whistles on new games, you're going to be running out of VRAM very, very frequently. The faster bus isn't really important for higher frame rate, it makes the worst-case situations less bad.
I get the impression that many reviewers aren't equipped to do the sort of review that asks questions like "What's the intensity and frequency of the stuttering in the game?" because that's a bit harder than just looking at average, peak, and 90% frame rates. The question "How often do textures load at reduced resolution, or not at all?" probably requires a human in the loop to look at the rendered output to notice those sorts of errors... which is time consuming, attention-demanding work.
Dylan16807 17 hours ago [-]
There's a good amount of reviewers showing 1% lows and 0.1% lows, which should capture stuttering pretty well.
I don't know how many games are even capable of using lower resolutions to avoid stutter. I'd be interested in an analysis.
rbanffy 15 hours ago [-]
I’m sure Windows performance counters can track the volume of data going between CPU memory and VRAM over the PCIe bus.
jeffbee 17 hours ago [-]
By an overwhelming margin, most computers are not in gamers' basements.
Actually the RapsberryPi (appeared 2012) was based on a SoC with a big and powerful GPU and small weak supporting CPU. The board booted the GPU first.
PCIe is a standard/commodity so that multiple vendors can compete and customers can save money. But at 8.0 speeds I'm not sure how many vendors will really be supplying, there's already only a few doing serdes this fast...
The ip companies are the first to support new standards, make their money selling to intel etc. Allowing intel or whomever to take their time to build higher performance ip.
At 50-100W for IO, this only leaves 11W per Core on a 64 Core CPU.
On the consumer side of things where the CPU's are branded Ryzen or Core instead of Epyc or Xeon, a significant chunk of that power consumption is from the boosting behavior they implement to pseudo-artificially[0] inflate their performance numbers. You can save huge (easily 10%, often closer to 30%, but really depends on exact build/generation) on energy by doing a very mild undervolt and limiting boosting behavior on these cpus and keeping the same base clocks. Intel 11th through 14th gen CPU's are especially guilty of this, as are most Threadripper CPU's. you can often trade single digit or even negligible performance losses (depends on what you're using it for and how much you undervolt/underclock/restrict boosting) for double digit reductions in power usage. This phenomenon is also true for GPU's when compared across the enterprise/consumer divide, but not quite to the significant extent in most cases.
Point being, yeah, it's a problem in data centers, but honestly there's a lot of headroom still even if you only have your common American 15A@120VAC outlets available before you need to call your electrician and upgrade your panel and/or install 240VAC outlets or what have you.
0: I say pseudo-artificial because the performance advantages are real, but unless you're doing some intensive/extreme cooling, they aren't sustainable or indicative of nominal performance, just a brief bit of extra headroom before your cooling solution heat-soaks and the CPU/GPU's throttle themselves back down. But it lets them put the "Bigger number means better" on the box for marketing.
Boosting from 4 to 5,5.5 ghz for that brief period shaves a fraction of a second - repeat that for any similar operation and it adds up.
Apparently we still have room, as long as you don't run anything else on the same circuit. :)
If you own the house, sure. Many people don't.
New homes are probably worse than old homes through. The wires a just chucked in the space been the outer and inner walls, there's basically no chance of replacing them of pulling new ones. Old houses at least frequently have piping in which the wires run.
I only have a PhD from YouTube (Electroboom)
If you actually had an electrician do it, I doubt they would've installed a breaker if they thought the wiring wasn't sufficient. Truth is that you can indeed get away with a 20A circuit on 14 AWG wire if the run is short enough, though 12 AWG is recommended. The reason for this is voltage drop; the thinner gauge wire has more resistance, which causes more heat and voltage drop across the wire over the length of it, which can cause a fire if it gets sufficiently hot. I'm not sure how much risk you would put yourself in if you were out-of-spec a bit, but I wouldn't chance it personally.
However! This strategy only works if the outlet was the only one on the circuit, and _that_ isn't particularly common.
(Another outlet type I've seen: I saw a NEMA 7 277V receptacle before. I think you get this from one phase of a 480V three-phase system, which I understand is ran to many businesses.)
Oddly, 14-50 has become the most common receptacle for non-hardwired EV charging, which is rather wasteful since EV charging doesn’t need the neutral at all. 6-50 would make more sense there.
I've definitely seen my share of scary things. I have a lighting circuit that is incomprehensibly wired and seems to kill LED bulbs randomly during a power outage; I have zero clue what is going on with that one. Also, often times opening up wall boxes I will see backstabs that were not properly inserted or wire nuts that are just covering hand-twisted wires and not actually threaded at all (and not even the right size in some cases...) Needless to say, I should really get an electrician in here, but at least with a thermal camera you can look for signs of serious problems.
And even then, even if you do run something 24/7 at max wattage, it's definitely not guaranteed to start a fire even if the wiring is bad. Like, as long as it's not egregiously bad, I'd expect that there's enough margin to cover up less severe issues in most cases. I'm guessing the most danger would come when it's particularly hot outside (especially since then you'll probably have a lot of heat exchangers running.)
Speak for yourself. I’d love to have that much computer at my disposal. Not sure what I’d do with it. Probably open Slack and Teams at the same time.
Too bad it feels like both might as well be single threaded applications somehow
It's often used for things like ACs, Clothes Dryers, Stoves, EV Chargers.
So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
https://appliantology.org/uploads/monthly_2016_06/large.5758...
Two phase power is not the same as split phase (There's basically only weird older installations of 2 phase in use anymore).
"The US electrical system is not 120V" https://youtu.be/jMmUoZh3Hq4
It'd be all new wire run (120 is split at the panel, we aren't running 240v all over the house) and currently electricians are at a premium so it'd likely end up costing a thousand+ to run that if you're using an electrician, more if there's not clear access from an attic/basement/crawlspace.
Though I think it's unlikely we'll see an actual need for it at home, I imaging a 800w cpu is going to be for server class CPUs and rare-ish to see in home environments.
I got a quote for over 2 thousand to run a 24v line literally 9 feet from my electrical panel across my garage to put a EV charger in.
Opening up an actual wall and running it to another room? I can only imagine the insane quotes that'd get.
I’m getting some wiring run about the same distance (to my attic, fished up a wall, with moderately poor access) for non-EV purposes next week and the quote was a few hundred dollars.
running to another room will be done usually (at least in usa) through attic or crawlspace. i got it done a few months ago to have dedicated 20A circuit (for my rack) in my work room. cost was around 300-400 as well
Honestly I wouldn't expect to pay less than $1000 for the job w/o any markups.
I've gotten multiple quotes on running the 240v line, the labor breakdown was always over $400 alone. Just having someone show up to do a job is going to be almost $200 before any work is done.
When I got quotes from unlicensed people, those came in around $1000 even.
In power cost? no
I'm literally any other way? also no
We rarely use 16A but it exists. All buildings are connected to three phases so we can get the real juice when needed (apartments are often single phase).
I'm confident personal computers won't reach 2300W anytime soon though
16A is fine, for most things. 10A used to be kind of ok, with the old IT net and old-style fuses. Nowadays anything under 16A is useless for actual appliances. For the rest it's either 25A and a different plug, or 400V.
On new installations you can choose 10A or 16A so if you're forward thinking you'd go 16 since it gives you another 1300 watts to play with.
The newer circuits in the house are all 16A, but the old ones (very old) are 10A. A real pain, with new TN nets and modern breakers.
Especially a special PDU: https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-202...
And cooling: https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-202...
A computer is becoming a Home Appliance in the it will need 20A wiring and plugs soon, but should move to 220/240v soon anyway (and change the jumper on your standard power supply).
How do you do that, if each GPU expects to be its own backplane? One CPU daughterboard per GPU, and then the CPU daughterboards get SLIed together into one big CPU using NVLink? :P
For other use cases like GPU servers it is better to have many GPUs for every CPU, so plugging a CPU card into the GPU doesn’t make much sense there either.
Maybe the GPU becomes the motherboard and the CPU plugs into it.
Would push performance further. Although companies like intel would bleed the consumer dry with, a certain i5-whatever cpu with onboard memory of 16 gigs could be insanely priced compared to what you'd pay for addon memory.
But you are right, there's no hierarchy in the systems anymore. Why do we even call something a motherboard? There's a bunch of chips interconnected.
An example, This is storage instead of GPU's, but as the SSD's were PCIe NVMe, it's pretty nearly the same concept: https://www.servethehome.com/zfs-without-a-server-using-the-...
PCI-e Networks and CXL are the future of many platforms... like ISA backplanes.
That said my experience in this field is more with storage than GPU compute, but I have done some limited hacking about in the GPGPU space with that tech as well. Really fascinating stuff (and often hard to keep up with and making sure every part in the chain supports the features you want to leverage, not to mention going down the PCIe root topology rabbit hole and dealing with latency/trace-length/SnR issues with retimers vs muxers vs etc etc etc).
It's still a nascent field that's very expensive to play in, but I agree it's the future of at least part of the data infrastructure field.
Really looking forward to finally getting my hands on CXL3.x stuff (outside of a demo environment.)
So if you incorrectly insert a card and bend a pin you're in trouble.
VPX has the sockets on the backplane so avoids this issue, if you bend pins you just grab another card from spares.
This may have changed since I last looked at it.
Telecoms industry definitely seem to favour TCA though.
Kinda like RAM - almost useless in terms of “upgrade” if one waits a few years. (Seems like DDR4 didn’t last long!)
I feel like I’ve been hearing about people selling five-to-ten-year-old GPUs for sometimes as many dollars as they bought them for, for the last five years; and people choosing to stay on 10-series NVIDIA cards (2016) because the similar-cost RTX 30-, 40- or 50-series was actually worse, because they’d been putting the effort and expense into parts of the chips no one actually used. Dunno, I don’t dGPU.
Also CPUs are able to make use of more space for memory, both horizontally and vertically.
I don't really see the power delivery advantages, either way you're running a bunch of EPS12V or similar cables around.
GPU RAM is high speed and power hungry. So there tends to not be very much of it on the GPU card. This is part of the reason we keep increasing the bandwidth is so the CPU can touch that GPU RAM at the highest speeds.
It makes me wonder though if a NUMA model for the GPU is a better idea. Add more lower power and lower speed RAM onto the GPU card. Then let the CPU preload as much data as is possible onto the card. Then instead of transferring textures through the CPU onto the PCI bus and into the GPU why not just send a DMA request to the GPU and ask it to move it from it's low speed memory to it's high speed memory?
It's a whole new architecture but it seems to get at the actual problems we have in the space.
I believe that's kind of what bolt graphics is doing with the dimm slots next to the soldered on lpddr5. https://bolt.graphics/how-it-works/
What’s keeping Intel/AMD from putting memory on package like Apple does other than cost and possibly consumer demand?
A total computer all-in-one. Just no interface to the world without the motherboard.
Then that became unnecessary when L2 cache went on-die.
It'll be interesting if consumer devices bother trying to stay with the latest at all anymore. It's already extremely difficult to justify the cost of implementing PCIe 5.0 when it makes almost no difference for consumer use cases. The best consumer use case so far is enthusiasts who want really fast NVMe SSDs in x4 lanes, but 5.0 already gives >10 GB/s for a single drive, even with the limited lane count. It makes very little difference for x16 GPUs, even with the 5090. Things always creep up over time, but the rate at which the consumer space creeps is just so vastly different from what the DC space has been seeing that it seems unreasonable to expect the two to be lockstep anymore.
so indeed the parent commenter would be correct that everything is Gen5 right now.
at least that's my best educated guess, looking at supermicro's public spec sheet and that it's shipping with CX7 which is Gen5 and not with CX8.
B200 supports Gen6 there is just nothing that would let it run at Gen6.
Edit: found this SemiAnalysis post saying as much https://x.com/SemiAnalysis_/status/1947768988467138645/photo...
to the best of my knowledge this does not exist, but I'd be happy to stand corrected.
(the official NVIDIA DGX B200 is gen5).
Faster lanes = more cost
More faster lanes = lots more cost
The chipset also strikes some of the balance for consumers though. It has a narrow high speed connection to the CPU but enables many lower speed devices to share that bandwidth. That way you can have your spare NVMe drive, SATA controller, wired and wireless NICs, sound hardware, most of your USB ports, your capture card, and some other random things connected over a single x4 to x8 sized channel. This leaves the high cost lanes for just the devices that actually use them (GPU and primary, possibly secondary, storage drive). I've got one consumer type Motherboard with 14 NVMe drives connected, for example, just not at full native speed directly to the CPU.
You're just SoL if you want to connect a bunch of really high bandwidth devices simultaneously (100 Gbps+ NICs, multiple GPUs at full connection speed, a dozen NVMe drives at native speed, or similar) because then you'll be paying for a workstation/server class platform which did make the "more faster lanes" tradeoff (plus some market segment gouging, of course).
Many mobos will operate the available slots such that the total number of active lanes is split between them. But if you use older-generation cards, you'll only get a fraction of the available bandwidth because you're only using a fraction of their lanes, although the physical lanes are physically present.
What I'm thinking about is something like, say, a pair of Gen3 NVMe drives that are good enough for mass storage (running in RAID-1 for good measure) and some cheap used 10 Gb NIC, which will probably be 8x Gen2, all running on a Gen4+ capable mobo.
And, while for a general-purpose setup I can live with splitting available BW between the NIC and the GPU (I most likely don't care about my download going super fast while I game), the downloads will generally go to the storage, so they must be fast at the same time.
True. And yet, if you buy an RTX 5090 today, costing $2400 and released in January this year, it's PCIe 5.0 x16
Contrast this with the wild west that is "Ethernet" where it's extremely common for speeds to track well ahead of specs and where interop is, at best, "exciting."
I believe only next Gen Intel and AMD Zen6 will get PCIe 6.0.
I am hoping Nvidia officially move into Server CPU market not only for their CPU but for wider Web Hosting as well. More competition for Server Hardware.
how you use those today is twofold:
- Gen5 x32 via two x16 slots (this is how most people use them)
- use actually the CX8 as your PCIe switch directly to your GPUs: https://www.servethehome.com/this-is-the-nvidia-mgx-pcie-swi...
GB300 is indeed Gen6.
https://archive.is/oa81K
One would think they get sizeable traffic as-is.
I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.
That's an interesting thought to look at. PCIe 3 was a while ago, but SATA was nearly a decade before that.
> I wonder what modulation order / RF bandwidth they'll be using on the PHY for Gen8. I think Gen7 used 32GHz, which is ridiculously high.
Wikipedia says it's planned to be PAM4 just like 6 and 7.
Gen 5 and 6 were 32 gigabaud. If 8 is PAM4 it'll be 128 gigabaud...
baud seems out of fashion, sym/s is pretty clear & unambiguous.
(And if you're talking channel bandwidth, that needs clarification)
> 16GHz square wave
Is it for PCIe 5.0? PCIe 6.0 should operate on the same frequency and doubling the bandwidth by using PAM4. If PCIe 7.0 doubled the bandwidth and is still PAM4, what is the underlying frequency?
for gen6, halve all numbers
(I'm accepting it because "Transfers"/"T" as unit is quite rare outside of PCIe)
Looking at some documents from Micron I don't see them using GT/s anywhere. And in particular if I go look at their GDDR6X resources because those chips use PAM4, it's all about gigabits per second [per pin]. So for example 6GHz data clock, 12Gbaud, 24Gb/s/pin.
PAM encoding is already analog, and also correspondingly more expensive (power, silicon size, etc) for the increase in speed.
It really wouldn't surprise me if even on workstation platforms only a subset of core lanes were Gen6+ and the common slots were redriven Gen5 or less off of a router / switch chip.
We don't have to go back, baud is still in use. I would expect transfers per second to be a synonym for baud though, and for bits per second per pin to use a different word.
Huh? Baud is sym/s.
Being less sarcastic, I would ask if 6.0 mobos are on the horizon.
I'm fairly sure they are cooking Gen7 already into CX9.
this would solve the biggest issue with non-server motherboards: not enough PCIe lanes.
What you're saying is possible though, you just need something a little heavier like a PCIe switch do the lane count + mixed-speed conversion magic. That's exactly what the chipset is, but for various reasons it's still only PCIe 4.0, even on the latest generation chips. I wouldn't be surprised if that changed again next generation. The downsides of this approach are switches add cost, latency, and can consume a lot of power. When they first upped the chipset to be a PCIe 4.0 connection in the 500 era, most motherboards capable of using the bandwidth of the chipset actually had to add chipset cooling.
Ideally they'd just add alternative options with more lanes directly from the CPU, but that'd add competition to the bottom of the Threadripper line.
Is that a problem?
These days, you don't need slots for your sound card and network card, that stuff's all integrated on the motherboard.
Plenty of enthusiast motherboards support 1 GPU, 3-4 nvme drives, 4 SATA drives, and have a PCIe 1x slot or two to spare.
Is anyone struggling, except folks working with LLMs? Seems to me folks looking to put several $2400 RTX 4090s in one machine ain't exactly a big market. And they'll probably want a giant server board, so they have space for all their giant four-slot-wide cards.
For example, motherboards with multiple nvme drives often have 1 drive with dedicated lanes and the remainder multiplexed through a PCIe switch embedded in the chipset.
Obviously PCI is not just about gaming but...
If you're using a new video card with only 8GB of onboard RAM and are turning on all the heavily-advertised bells and whistles on new games, you're going to be running out of VRAM very, very frequently. The faster bus isn't really important for higher frame rate, it makes the worst-case situations less bad.
I get the impression that many reviewers aren't equipped to do the sort of review that asks questions like "What's the intensity and frequency of the stuttering in the game?" because that's a bit harder than just looking at average, peak, and 90% frame rates. The question "How often do textures load at reduced resolution, or not at all?" probably requires a human in the loop to look at the rendered output to notice those sorts of errors... which is time consuming, attention-demanding work.
I don't know how many games are even capable of using lower resolutions to avoid stutter. I'd be interested in an analysis.