Fine-tuning and serving 13B-class models on one 24GB card.
- Processor
- AMD 16 Core Processor
- Graphics
- ZOTAC 24 GB Vram graphics card
- Memory
- 64GB High speed memory
- Form
- Tower, desk-side
AI & Machine Learning
Training, fine-tuning and local inference
Video memory decides which models fit.
Start with your software
Select your software
PyTorch training and fine-tuningDeep learning trainingLocal LLMsRunning language models on your own hardware
Also common in this work: NVIDIA DALI, CUDA.
Most important first
What this work needs
1
Video memory capacity, ahead of everything
A model larger than the card will not start, however fast the card is.
2
Memory bandwidth on the card
Once the model fits, memory bandwidth sets how fast it answers.
3
A host that can keep the loader ahead of the card
Enough cores and fast drives that the card never waits for data.
Where money gets wasted
Huge core counts. The same money on a bigger card changes what you can run.
Our picks
Machines for AI & machine learning
30B-class work, with ECC memory and a dedicated dataset drive.
- Processor
- AMD 24 Core Processor
- Graphics
- Palit 32 GB Vram graphics card
- Memory
- 128GB High speed memory
- Form
- Tower, desk-side
Two 48GB cards in a rack, shared by a team.
- Processor
- AMD 64 Core Processor
- Graphics
- HPE 96 GB Vram graphics card
- Memory
- 384GB High speed memory
- Form
- 4U rack
192GB of card memory in one node, for the largest models.
- Processor
- AMD 96 Core Processor
- Graphics
- HPE 192 GB Vram graphics card
- Memory
- 768GB High speed memory
- Form
- 4U rack
Worth knowing: The entry machine has no ECC memory, so it is not for runs that last two weeks.
Not sure which AI & machine learning machine?
Tell us your software and your largest job. An engineer replies within one working day.