text-gen

HuggingFaceM4/tiny-random-LlamaForCausalLM

569,006 downloads

20 likes

Hardware Recommendations

Three tiers based on VRAM requirements: minimum for basic inference, comfortable for longer contexts, headroom for future model sizes.

Minimum $299

8GB VRAM 115W TDP

Runs 7B at Q4_K_M quantization with 4K context.

Comfortable $599

12GB VRAM 200W TDP

Runs 7B at Q5_K_M quantization with 8K+ context.

Headroom $799

24GB unified 25W TDP

Runs 7B at FP16 with 32K+ context or comfortably handles the next size up.