I remember to have read somewhere that GPT4 is not a single model but several mo... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

_nalply on July 11, 2023 | parent | context | favorite | on: GPT-4 details leaked?

I remember to have read somewhere that GPT4 is not a single model but several models whose parameter counts are reported as a single sum. Perhaps quite doable but at lower speeds?

kirill5pol on July 11, 2023 [–]

The article linked here talks about GPT4 being a mixture of experts, which is exactly what you’re describing

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact