At least two RTX 5090s, or get a NVIDIA A100.

.

To add to the point above: honestly, its kinda disappointing how everyone jumps straight to the most expensive enterprise gear or the newest flagship stuff. Not everyone has ten grand to drop on a home lab, right? Before I give you a full breakdown, i gotta ask... what quantization level are you actually aiming for? Running DeepSeek 70B at 4-bit vs 8-bit makes a massive difference in the VRAM floor you need to hit. If youre thinking about long-term ownership and want to be smart with your cash, stay away from the NVIDIA GeForce RTX 4090 24GB because the prices are just not as good as expected lately. Id suggest looking for a pair of used NVIDIA GeForce RTX 3090 24GB cards instead. You get 48GB total VRAM which is basically the sweet spot for these 70B models, and itll cost you way less than a single A100 or 5090 setup. Just make sure your power supply is beefy enough because those 3090s can be real power hogs when theyre working hard tho.

Honestly its just ridiculous how these companies keep pushing us into a corner. I have been building rigs for years and it really drives me crazy how the cost of entry for decent inference is skyrocketing. They know we need the VRAM to make these models actually useful and they just price it like we are all printing money. Its such a scam. Everything feels like it is going downhill lately. You spend a fortune on the latest hardware just for it to feel dated in a few months because the software bloat is out of control. Nobody seems to care about efficiency anymore, they just want to sell you the next expensive chip. My current setup works well enough for now, but I am honestly fed up with the market state. It is just draining to keep up with this treadmill when the actual quality of support is plummeting.

two or three 4090s.

You'd better to get the RTX 5080 or RTX 5090.

NVIDIA A100.

Has anyone tried AMD GPUs?