GPT-3 model discussion

Hi all! You might have heard of the GPT-3 NLP model, a pretty wild incremental step in artificial intelligence research. If not, check out the paper: https://arxiv.org/abs/2005.14165 and these tweets/links of people using it to write poetry, management essays, code, creative writing and a very profound sentence:
https://twitter.com/sharifshameem/status/1282676454690451457?s=21
https://twitter.com/edleonklinger/status/1284251420544372737?s=21
https://twitter.com/zebulgar/status/1283927560435326976?s=21
https://twitter.com/edleonklinger/status/1284138261963735041?s=21
https://www.gwern.net/GPT-3
This all seems very relevant to what we’re learning during this program so I thought it could be fun to start a discussion with whoever’s interested. I’d love to dive into the algorithm with people here. Also if anyone has been playing with this already and would be willing to teach/share, please step forward that’d be amazing!

5 Likes

I don’t think it’s publicly available yet, is it? I was just reading this interesting observation from an “early access” user on twitter. Looks like you need to “prime” the model well in order to get good outputs.

I haven’t been keeping up with GPT, but would love to be involved in a discussion. I’ll try to read the paper in the next day or so.

Saw that thread about priming the model, she got it to work! The few-shot extrapolation sounds insane.
You’re right the API isn’t publicly available, but the paper is - that would be the matter of the discussion. Also asking if anyone has played with it, i.e. has access to the beta.

I’ve been messing around a bit with AI Dungeon’s ‘dragon model’ which implements some form of GPT-3. I think there are some modifications made to make it more ‘story-like’, but I have been able to get some interesting results out of it. There’s more discussion of using AI Dungeon to mess around with GTP-3 here. My favorite result I’ve gotten thus far is this completion of Dr. Seuss (prompt in bold).

Do you like green eggs and ham?

I don’t like them, Sam-i-am. I do not like green eggs and ham!

Would you like them here or there?

I would not like them here or there.

I would not like them anywhere.

I am a man of the world, Sam. You are a man of the world too; but you have chosen to be different from me. And that is why we quarrel.

Do you know what it means to live in a city where everything is new and strange every day? It makes one long for home, doesn’t it? But then one has no home except death…

Death is always with us, Sam.