notesum.ai

Published at November 12

Evaluating ChatGPT-3.5 Efficiency in Solving Coding Problems of Different Complexity Levels: An Empirical Analysis

cs.SE

cs.AI

Released Date: November 12, 2024

Authors: Minda Li¹, Bhaskar Krishnamachari¹

Aff.: ¹University of Southern California

Arxiv: http://arxiv.org/abs/2411.07529v1

Refer to caption

	Easy	Medium	Hard
Prompt Engineering	29%	19%	14%
Providing Failed Test Cases	38%	60%	45%
GPT-4	33%	58%	52%
Claude 3 Sonnet	36%	48%	26%
Gemini 1.0 Pro	34%	17%	6%