notesum.ai

Published at December 6

A Survey of Sustainability in Large Language Models: Applications, Economics, and Challenges

cs.AI

cs.CE

Released Date: December 6, 2024

Authors: Aditi Singh¹, Nirmal Prakashbhai Patel¹, Abul Ehtesham², Saket Kumar³, Tala Talaei Khoei⁴

Aff.: ¹Cleveland State University; ²The Davey Tree Expert Company; ³The Mathworks; ⁴Khoury College of Computer Science, Roux Institute at Northeastern University

Arxiv: http://arxiv.org/pdf/2412.04782v1

Cost Type	Cost Components	Description	Estimated Cost (USD)
Training LLM	Energy	Power for GPU clusters, consumption depends on model size and training duration (e.g., GPT-3 training required 1 GWh/day).	$500,000 - $5 million per training run [29]
Training LLM	Computing	Use of specialized hardware like NVIDIA A100/H100 GPUs or TPUs. For PaLM, training took 8,404,992 TPUv4-core hours.	$9M - $23M per training cycle [30], [31]
Model Development	Data Collection and Cleaning	Acquisition, filtering, and preparation of large, high-quality datasets.	$50,000 - $100,000 [32], [33]
Model Development	Model Fine-Tuning	Additional compute and expert involvement required to tailor models for specific use cases.	$50,000 - $500,000 [34], [32]
Maintain Server	Storage	Storing model weights, checkpoints, and datasets (cloud storage costs vary).	$10,000 - $100,000/month [33], [34]
	Electric	Continuous power for servers and cooling systems (data centers consume 1 GWh/day for models like ChatGPT).	$5,000 - $50,000/month [35], [29]
	Energy Resource	Usage of renewable energy to reduce environmental impact (scales with model size).	Initial setup $50,000+; $2,000 - $20,000/month [30], [32]
	Cooling Systems	Data centers with high-performance GPUs require extensive cooling infrastructure.	$10,000 - $100,000/month [32], [29]
Software Licensing	Proprietary Software/Frameworks	Licensing fees for tools like TensorFlow, PyTorch, etc.	$5,000 - $50,000/year [30], [29]
Compliance & Security	Data Security and Compliance	Adherence to GDPR, HIPAA, and other regulations during training and deployment.	$20,000 - $100,000/year [32], [33]
Compliance & Security	AI Ethics and Bias Audits	Ensuring fairness and ethical compliance through regular audits.	$50,000 - $150,000 per audit [33], [32]
Personnel	Skilled Workforce	Salaries for AI researchers, engineers, and cloud DevOps experts.	$500,000 - $1 million/year for a small team [33], [35]
Monitoring and Updates	Model Performance Monitoring	Ongoing evaluation to ensure model performance doesn’t degrade.	$50,000 - $200,000/year [33], [32]
Monitoring and Updates	Regular Model Retraining	Required for maintaining relevance in dynamic fields like finance or health.	$500,000 - $2 million/year [29], [33]
Deployment Costs	API Management	Hosting, scaling, and managing APIs for model access.	$1,000 - $10,000/month [33], [35]
Deployment Costs	Scaling Infrastructure	Load balancing and redundancy for handling real-time traffic and requests.	$10,000 - $100,000/month [33], [32]
Token-Related Cost	Usage Case-Specific (Min - Max)	Token charges vary by use case (OpenAI GPT-3 costs $0.0001 - $0.06 per token; Amazon Nova Pro $0.80 - $3.20 per token).	$0.0001 - $0.06/token or $0.80 - $3.20/token [33], [34, 36]
User Feedback Loop	Continuous Improvement	Collecting user feedback to enhance model performance and personalization.	$50,000 - $200,000/year [33], [35]
Edge or On-Prem Deployment	Specialized Hardware	On-prem hardware setups for privacy-focused or latency-sensitive tasks (e.g., NVIDIA A100).	$100,000+ for setup; $5,000 - $50,000/month [32], [29]