๐Ÿค– LLM Long Code Generation Output Experiment

A Comparative Analysis of Code Generation Capabilities

Data Analysis

Model accessui Character Count Code Characters Code Percentage Code Blocks Output Number
0 Qwen2.5 Coder 32B Hugging Face Chat 13950 12350 88.53 5 1
1 o1 Preview LibreChat 13622 11221 82.37 10 2
2 Qwen2.5 Coder 32B OpenWebUI 13159 10822 82.24 7 3
3 Qwen 72B HuggingFaceChat 12921 10840 83.89 7 4
4 DeepSeek 2.5 OnlinePlayground 10105 8820 87.28 8 5
5 Claude 3.5 Sonnet LibreChat 10007 8705 86.99 2 6
6 Gemini 1.5 Pro Google AI Studio 7926 6583 83.06 3 7
7 LeChat Mistral 7719 6595 85.44 4 8
8 Phind 70B Site 7678 5474 71.29 6 9
9 Llama 3.1 3B LocalLM 7321 6279 85.77 2 10
10 Llama 3.3 70B Hugging Face Chat 7195 4718 65.57 3 11
11 Cohere Command R Plus Hugging Face Chat 7145 6069 84.94 1 12
12 DeepSeek 2.5 Chatbox (Desktop UI) 7017 4297 61.24 1 13
13 Nova Pro 1.0 Open Router 6954 5911 85.00 1 14
14 Qwen 2.5 7B LM Studio 6773 5253 77.56 5 15
15 Nova Pro 1.0 Open Router WebUI 6568 5536 84.29 5 16
16 DeepSeek Coder 7B LM Studio 6372 5246 82.33 2 17
17 GPT 3.5 (16K) LibreChat 3972 3129 78.78 1 18
18 Codetral Mamba Python GUI 2091 0 0.00 0 19