GPT-4o: Complete Benchmark Performance Guide

openai 4o 2024-05-13 Proprietary

In-depth analysis of GPT-4o performance on MMLU, HumanEval, GSM8K, MATH, BBH benchmarks, including pricing and use cases.

Key Specifications

Vendoropenai
Version4o
Release Date2024-05-13
Context Window128000 tokens
Input Modalitiestext, image, audio
Output Modalitiestext, audio
LicenseProprietary
Documentationhttps://platform.openai.com/docs/models/gpt-4o

Benchmark Performance

BenchmarkScoreUnitEvaluated AtNotesSource
MMLU88.7%2024-05-135-shotview
HUMANEVAL90.2pass@12024-05-13—view
GSM8K95.8%2024-05-130-shot CoTview
MATH76.6%2024-05-130-shot CoTview
BBH83.1%2024-05-133-shot CoTview

Pricing

TierPriceCurrency
Input$2.5 / MtokUSD
Output$10 / MtokUSD
Cache Read$1.25 / MtokUSD
Cache Write$2.5 / MtokUSD

Source: https://openai.com/api/pricing/ · as of 2024-08-01

Compliance

  • Data Residency: US
  • SOC2: ✓
  • HIPAA: ✓
  • GDPR: ✓
  • ISO 27001: ✓

GPT-4o

Aperçu du modèle

OpenAI 旗舰多模态模型，支持文本、图像、音频输入与文本、音频输出，128K 上下文窗口，在 MMLU、HumanEval、GSM8K 等基准上表现领先。

Spécifications principales

FournisseurVersionDate de sortieFenêtre de contexteModalités d’entréeModalités de sortieLicence
Openai4o2024-05-13128Ktext, image, audiotext, audioProprietary

Performance aux benchmarks

BenchmarkScoreUnitéNotes
MMLU (Massive Multitask Language Understanding)88.7%5-shot
HumanEval90.2pass@1—
GSM8K (Grade School Math 8K)95.8%0-shot CoT
MATH76.6%0-shot CoT
BBH (BIG-Bench Hard)83.1%3-shot CoT

Tarification

EntréeSortieLecture cacheÉcriture cache
————

par million de jetons

Forces

  • MMLU score 88.7, strong knowledge reasoning.
  • HumanEval 90.2, excellent code generation.
  • GSM8K 95.8, robust math reasoning.
  • 支持文本、图像、音频多模态输入。

Faiblesses

  • 闭源专有模型，不支持自托管。

Cas d’usage

  • 代码生成与调试
  • 视觉与图像理解
  • Agent 工作流与工具调用

Références