usually performance takes a hit with quantization. are you getting quality respo... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		JJneid on June 14, 2024 \| parent \| context \| favorite \| on: Deploying Llama3 70B on AWS – GPU Requirement, Cos... usually performance takes a hit with quantization. are you getting quality responses?

rini17 on June 14, 2024 [–]

Since llama3, yes, quite satisfying.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact