Identify AI-generated comments on Hacker News
A UserScript to detect GPT generated comments on Hackernews.
This is a browser UserScript that tries to detect comments on Hacker News that are GPT/OpenAI generated ChatGPT. It is based on GPT output detector.
It puts every comment into the GPT output detector and colors and writes a short comment on the HN comment based on its assessment, like you see in the screenshot. This is based on a threshold set by me. >0.7 is probably AI, >0.9 is definitely AI. Lower than that is most likely human. Most comments on HN still appear to be human.
It only becomes reliable after about 50 tokens (one token is around 4 characters) so I mark the comments that are too short with gray and make no assessment on those.
Remember that the point is to detect comments obviously written completely by GPT, not detecting that a human a wrote it. You have to discern between the two, as the scale is not really utilized here. What I've found over the last few days of playing with this is that most AI generated text will almost always be identified with 99.97% probability , while human text ranges somewhere between 70-98. So I could move the threshold for my script quite a lot. I just wanted to see how it works with these thresholds for a while.
I know pretty much nothing about Javascript. This is shitty code largely written by ChatGPT itself, untested beyond Chrome on MacOS with Violenmonkey. I have no plans for maintenance or extending it. If you want features or support, you have to ask ChatGPT, not me.
It's just sending a HTTP request to https://huggingface.co/openai-detector/ with the contents of a comment. This have some privacy, performance and data usage concerns. Additionally I am guessing that there is some rate limiting to using huggingface's service for free.
Install Violentmonkey
Alternative: Tampermonkey （Chrome / Firefox / Edge）
| Greasyfork | GitHub |
|---|---|
| Not published yet | Github |
