NelsaHost

Hardware AI Models Compatibility Compare

search Login

smart_toy

Llama Large Language Models

Llama 3 70B (70.00B)

Parameters

70.00B

VRAM (FP16)

140.0GB

VRAM (INT4)

35.0GB

Context

8192

check_circle View Compatible GPUs open_in_new View on HuggingFace

tune Quantization Options

Quantization	VRAM Required	Min GPU
FP16 (Half Precision)	140.0GB	A100 / H100
INT8 (8-bit Integer)	70.0GB	A100 / H100
Q4_K_M (GGUF 4-bit)	35.0GB	A6000 / 2x 4090
q3_k_m	28.0GB	A6000 / 2x 4090

memory Compatible GPUs

A100 40GB

40.0GB VRAM

A100 40GB

40.0GB VRAM

H100 SXM

80.0GB VRAM

H100 SXM

80.0GB VRAM

H100 SXM

80.0GB VRAM

H100 PCIe

80.0GB VRAM

View All Compatible GPUs

Model Details

Family Llama

Category Large Language Models

Parameters 70.00B

Context Length 8192

Similar Models

Llama 3 8B 8.00B Llama 3.1 405B 405.00B Llama 3.1 70B 70.00B Llama 3.1 8B 8.00B