That is true, and so I suspect the only truly FLOSS models we will ever see will be highly specialised to a particular task; which is, in the long run, fine by me. A coding llm that comes with a huge corpus of open source training code, maybe even just in a specific language, a speech to text model with a corpus of transcribed creative commons audio, probably a single language or pair of languages if its for translation, etc. That way the datasets, while still huge, could actually be curated by a single community.