Uttam Emmaniuel Dammu

This project presents quantitative benchmarking of Large Language Models (LLMs) to analyze how energy consumption, latency, and accuracy vary across open-source and closed-source systems. It highlights efficiency trade-offs among quantized open models such as Mamba-Codestral-7B and Qwen2-7B (4-bit and 8-bit precision) versus closed APIs like ChatGPT and Gemini. Open models were benchmarked locally using GPU telemetry and CodeCarbon, while closed APIs were tested through response-time monitoring. The study investigates how electricity usage changes with prompt size, how response time changes as token counts grow, and how efficient open models are per 1,000 tokens. Open models were measured locally with energy tracking, while closed API models are shown for latency context as energy is not observable. Energy consumption increases linearly with the number of input tokens. Mamba-7B (4-bit) demonstrates the lowest energy curve, while Qwen2-7B (8-bit) maintains a slightly higher curve, balancing precision and efficiency. Additional analysis includes energy per 1,000 tokens, latency distribution, and carbon emissions, demonstrating that reducing model precision and optimizing token throughput can significantly lower carbon emissions in real deployments.

-