GAN diagram (source)
It's a very intuitive and clever concept, and one that I personally feel mirrors the internal dialog that I constantly have in my own brain's decision making system. The critic in my head feels almost like a discriminator algorithm throwing shade on my internal generator algorithm. Anyway, if you're interested in how they work in detail, you can read more here.
Unfortunately, I wasn't quite smart enough to figure out how to modify the pre-existing GPT-2 model to turn it into a GAN. I think it's possible, but tensorflow is a confusing beast, and I'm not yet at the point where I care enough to untangle that mess. Instead, I did something a little simpler that was just effective enough to make the results passable.
I instead opted for a multi-stage modeling framework, utilizing three separate deep-learning models stitched together one after the next. This diagram illustrates the many parts that needed to be trained, and how they were strung together to produce replies from comments.
In this setup, I first pick a comment on reddit to serve as seed text for the generator. I generate a whole bunch of replies for this comment using my GPT-2 model. Then I pass all the candidates to the discriminator model to filter out the messed up comments and only select the best ones.
To build the discriminators, I fine-tuned another deep-learning language model called BERT. I made two models, one for how realistic the reply was and another for how many upvotes the reply would get.
You can read this post for a detailed walkthrough of how the whole system was constructed, trained, tested, and deployed.
Just by looking at the initial results, it seemed likely that the bot was going to be able to communicate pretty convincingly. But the only real test was to put it into use and see how people respond.
Once I had the models built and hooked together, the last step was to plug the bot into reddit. I made an account for the bot called tupperware-party, which I figured sounded innocuous enough. I used praw to submit the replies automatically, and then I went through and examined all of them to make sure none were too offensive or annoying. As I was reading through the results, there were so many gems that it's hard to pick just a few examples to share.
This first one seems like a perfect imitation of someone with a strong opinion on the internet.
I actually can't explain how this next one could possibly work. It seems as if the bot is responding to the specific numerical bullet points in the original comment.
Notice that in the original comment, point 2 is about sleep and the bot says "2" right before talking about sleep. Then it says "3" before switching subjects to talking about something that induces anxiety. It doesn't make perfect sense but it somehow knows to respond to bullet points separately, which seems like a huge leap given that it was never trained to do that specifically.
This one is great on a number of levels. First, it's kind of meta because the bot was posting into the r/artifical subreddit, which is a forum dedicated to artificial intelligence. Second, not only is the comment pretty darn coherent, it is so much so that the original author writes a well thought out reply further expanding on his point in light of the bot's comment.
I honestly don't even know what to say to that. Is it possible that every conversation on the internet right now has at least one slightly ill-informed bot in the mix. We are seriously screwed. But don't worry, this bot is at least a little woke too already.
There are so many surprisingly realistic replies that I enourage you to go through tupperware-party's whole comment list. Overall, the bot wrote 80 replies and 24 of them received at least one upvote. I'm impressed with that and hoping that maybe it will eventually be able to help me become more popular on reddit.
On the other hand, the single most popular comment (with 8 votes) was this one, which is just innocuous flattery.
Since this could easily be copy and pasted to every other comment and still be totally in context, I guess maybe I didn't need to try so hard.
If you find this interesting, I've written a tutorial post with details and code describing how I built everything and showing what you'd need to do to recreate one of your own. I realize there are definite ethicality concerns with building and using something like this so I encourage you to be an excellent human and only use this tool sparingly and for that which you deem to be good.
I know there are definitely some ethical considerations when creating something like this. The reason I'm presenting it is because I actually think it is better for more people to know about and be able to grapple with this kind of technology. If just a few people know about the capacity of these machines, then it is more likely that those small groups of people can abuse their advantage.
I also think that this technology is going to change the way we think about what's important about being human. After all, if a computer can effectively automate the paper-pushing jobs we've constructed and all the bullshit we create on the internet to distract us, then maybe it'll be time for us to move on to something more meaningful.
If you think what I've done is a problem feel free to, or publically shame me on Twitter.