source includes the training data. If i gave someone the full source code of a wasm interpreter and a wasm blob that contains all of the logic of the actual application, with no way of building that blob yourself, and call that open source, I’d be laughed out of the room.
That is true, and so I suspect the only truly FLOSS models we will ever see will be highly specialised to a particular task; which is, in the long run, fine by me. A coding llm that comes with a huge corpus of open source training code, maybe even just in a specific language, a speech to text model with a corpus of transcribed creative commons audio, probably a single language or pair of languages if its for translation, etc. That way the datasets, while still huge, could actually be curated by a single community.
source includes the training data. If i gave someone the full source code of a wasm interpreter and a wasm blob that contains all of the logic of the actual application, with no way of building that blob yourself, and call that open source, I’d be laughed out of the room.
“open source” models usually do exactly this
That is true, and so I suspect the only truly FLOSS models we will ever see will be highly specialised to a particular task; which is, in the long run, fine by me. A coding llm that comes with a huge corpus of open source training code, maybe even just in a specific language, a speech to text model with a corpus of transcribed creative commons audio, probably a single language or pair of languages if its for translation, etc. That way the datasets, while still huge, could actually be curated by a single community.
I use olmo from AI2 which also includes the training data.