Criteria for inclusion

Listed models are usually one of:

• New pretrained model with over 1 billion parameters.
  • E.g. GPT-Neo-Japanese-1.3B (pretrained from scratch)
• New model over 1 billion parameters that improves over state of the art performance.
  • E.g. Minerva (finetuned model)

Known models currently not included:

• RETRO, Atlas

Organization color coding

Grey - Industry research groups

Purple - Nonprofit collaborations

Blue - Academic labs

Release color coding

Generally:

• Code | Green check: Code available. Red x-mark: not available
• Model | Green check: Checkpoints available. Yellow diamond: Available, but restrictions / license requirements / manual approval block access to weights. Red x-mark: Not available.
• Train Data | Green check: Exact training dataset available. Yellow diamond: Parent dataset available, but filtering or processing code not provided. Red x-mark: Not available.
• Eval artifacts: Green check: Evaluation predictions available, all prompts available, code used to run evaluations. Yellow diamond: Some artifacts available. Red x-mark: Not available.