notesum.ai
Published at November 27Thai Financial Domain Adaptation of THaLLE -- Technical Report
cs.CL
cs.AI
Released Date: November 27, 2024
Authors: KBTG Labs, Atthakorn Petchsod, Pornchanan Balee, Danupat Khamnuansin, Anuruth Lertpiya, Chanatip Saetia, Tawunrat Chalothorn, Thadpong Pongthawornkamol, Monchai Lertsutthiwong
| Model | Public IC Practice Exams | ||
|---|---|---|---|
| P1 | P2 | P3 | |
| Commercial APIs | |||
| gemini-1.5-flash-002 [3] | 68% | 56% | 72% |
| gemini-1.5-pro-002 [3] | 74% | 76% | 72% |
| gpt-4-turbo-2024-04-09 | 72% | 68% | 84% |
| gpt-4o-2024-11-20 [4] | 74% | 76% | 84% |
| Open Instruction-Tuned Foundational LLMs | |||
| Llama3.1-8B-Instruct [1] | 54% | 64% | 48% |
| Qwen2-7B-Instruct [11] | 64% | 48% | 56% |
| SeaLLMs-v3-7B-Chat [12] | 50% | 28% | 48% |
| THaLLE-0.1-7B-fa [23] | 64% | 52% | 60% |
| THaLLE-0.1-7B-ic | 72% | 72% | 84% |