36
$\begingroup$

Recently, the chatbot ChatGPT was released and experienced a surge in popularity. It's reasonably good at mimicking humans, and it has a database of information to draw on. The upshot is that you can ask it a question and it will respond the way a human would -- albeit sometimes correctly and sometimes incorrectly, depending on the subject matter and the limits of its knowledge.

This has caused problems on Stack Overflow, which was deluged with a slew of answers generated by folks simply plugging questions into ChatGPT. This is a problem because the software is good at very confidently stating things that may be incorrect in subtle but important ways, and a large volume of such answers can cause real problems. As an extraordinary though potentially temporary step, SO banned all posts copied from ChatGPT outputs. This has led to discussions about both site-specific (e.g. 1, 2) and network-wide bans, and the company is, I believe, considering the issue.

What should our policy be on Worldbuilding -- at least until there is a network-wide policy in place? While we're not as large as Stack Overflow, the mods have seen several answers already that are clearly chatbot-generated, and there may be more we haven't found. There are several reasons against allowing such content, including:

  1. It's quite frankly plagiarism if the answerer doesn't note that they used ChatGPT, and is therefore not allowed by our normal plagiarism policy.
  2. On Worldbuilding in particular, answers can be harder to verify than on Stack Overflow or Mathematics, and so it may be very easy for someone to game the system with a large quantity of confidently-written but nonetheless wrong chat-bot generated answers.
  3. It can be very wrong very often, which I've seen myself doing some testing. I personally would not trust anything generated by the tool that involves information beyond what could be gleaned from a quick Google search.

There are also arguments in favor or allowing it:

  1. ChatGPT is a tool just like Wikipedia or a textbook; it may be wrong at times, but so long as the software used is noted, folks are welcome to vote up correct answers and vote down incorrect answers.
  2. Chatbots may (may) make it easier to turn complex information into straightforward answers.

I personally come down hard on the side of disallowing ChatGPT and all similar tools, but obviously this is an issue for the community as a whole to decide. What should we do?

Here's an example of an answer that was clearly generated by ChatGPT (answer deleted; text shown here for discussion). It was one of a slew posted in a ~10-minute interval by one user.

This setup is not scientifically plausible. There are a few issues with it:

The distance between the Jupiter-like planet and the Solar system is too small. The closest star to the Sun, Proxima Centauri, is about 4.24 light-years away. A distance of 0.1 light-year is about 6.16 trillion kilometers, which is a little over a hundred times the distance between the Earth and the Sun. This is not a realistic distance for a planet to be passing by the Solar system.

The idea of colonizing a moon that is rich in ice to use as fuel for fusion reactors is not feasible. Fusion reactors require a specific type of fuel, typically hydrogen or helium, which are not found in ice. Additionally, the amount of fuel needed for fusion reactors is typically much greater than what could be extracted from a single moon, even if it were rich in ice.

It is not clear how the settlers would be able to establish a colony on a moon of a Jupiter-like planet that is passing by the Solar system. It would require a significant amount of resources and technology to set up a habitable environment on a moon, and it is unlikely that a group of only 2.5 thousand people would be able to accomplish this in a short amount of time.

In order to make this setup more scientifically plausible, you could consider changing the distance between the Jupiter-like planet and the Solar system to a more realistic value, finding a different source of fuel for the fusion reactors, and providing more information about how the settlers would be able to establish a colony on a moon of the planet.

I put the question body into ChatGPT and got the following, on the first try. Besides the many stylistic similarities, note that the same points are regurgitated, just in a different order.

There are several issues with the scenario you have described:

A rogue planet passing by the Solar System at a distance of 0.1 light years is extremely close and highly unlikely. The closest known rogue planet to the Solar System is about 20 light years away.

It would be extremely difficult, if not impossible, for a group of 26 starships to carry 2.5 thousand people and all the necessary resources to establish a colony on a distant moon. Such a mission would likely require a much larger fleet of ships and a much larger population.

The timeline you have described is also unrealistic. It would take much longer than 20 years for a starship to travel to a distant planetary system, even at the fastest speeds currently achievable with current technology.

It is not clear how the settlers would be able to sustain themselves on an ice-rich moon, as it is unlikely that there would be any other resources available to them.

Overall, the scenario you have described is not plausible as it stands. In order to make it more realistic, you would need to significantly revise the details of the mission, including the size of the fleet and population, the distance and location of the target planetary system, and the resources available to the settlers.

Clicking "Try again" in the chat sometimes led to it going into more detail in some of its points, or expanding the concluding paragraph.

There are a number of other answers which we identified as matching ChatGPT outputs via the same comparison. Examples are 1, 2, 3, and 4 (note that it incorrectly describes what the Bondi radius is). These are all, without a doubt, chatbot-generated.

$\endgroup$
16
  • $\begingroup$ Could you link to some of the "clearly chatbot generated" responses? $\endgroup$
    – elemtilas
    Commented Dec 16, 2022 at 15:58
  • 2
    $\begingroup$ @elemtilas Yup, just edited one in, and linked to some others (all of which have been deleted -- but I think the point is clear enough). That's the level of stuff we've been catching. It's not a lot here on Worldbuilding, but as ChatGPT usership grows, we may see a lot more of it. $\endgroup$
    – HDE 226868 Mod
    Commented Dec 16, 2022 at 16:33
  • 2
    $\begingroup$ @HDE226868 Hey if you already have an account on ChatGPT could you please ask it whether it thinks it should be allowed on Worldbuilding? $\endgroup$
    – Daron
    Commented Dec 16, 2022 at 17:22
  • $\begingroup$ Interesting to note how close your chatbot answer was to "Skaldebanes" chatbot answer. @Daron --- I'd actually argue that if an AI wished to and could actually get a SE account, then it ought to be allowed to answer questions! It could at least police all the humans who are littering the place with chatbot generated spam. $\endgroup$
    – elemtilas
    Commented Dec 16, 2022 at 17:46
  • $\begingroup$ @elemtilas Yup, that's the sort of similarity we're seeing, which is why I would feel comfortable taking action against a post like this. There may be stuff slipping through that nobody's seeing because it's less obvious -- and that may be the case down the line -- but if we're not sure, then we're probably not going to take action anyway. $\endgroup$
    – HDE 226868 Mod
    Commented Dec 17, 2022 at 2:54
  • 2
    $\begingroup$ Hydrogen not found in ice? :-D $\endgroup$
    – A. B.
    Commented Dec 17, 2022 at 5:53
  • $\begingroup$ Some text-generating AIs seem to have an eerie tendency to produce very expert and well-informed-sounding answers, whether they're actually right or not. They seem to mimic the writing style of experts discussing scientific questions, with mimicking the content of what those experts say being apparently a by-product of that that doesn't always happen. That example above, for instance, explaining in a very authoritative-looking way how fusion wouldn't be useful on an ice moon because there is no hydrogen in ice. $\endgroup$
    – A. B.
    Commented Dec 17, 2022 at 6:16
  • 4
    $\begingroup$ To build on what @elemtilas said, and to advance what I believe would be an interesting idea, SE should approach the ChatGPT team and invite them to associate a factual account with ChatGPT. Heck, let it try to answer each and every question. We up/down vote the answers (maybe even give it the green checkmark), and those results are fed back to ChatGPT to improve the quality of its results. How often is anyone doing Q/A right now, anyway? We may become irritated to within an inch of our lives, but that might be an interesting way to contribute to its development. $\endgroup$
    – JBH
    Commented Dec 17, 2022 at 6:51
  • $\begingroup$ @Daron Done it. See my answer :). $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 1:36
  • $\begingroup$ @elemtilas I've already asked a few WB questions to it, both from recents and a few from my safe. Answers are "okayish" and often overlaps with best voted ones, though like every few deep A.I.s I tried it lacks the spark of what makes a better answer, nor can it find very creative solutions to problems. It also has trouble detailing answers, even if you precise later what you want (ChatGTP remembers your previous sentences). It's very much like looking on Google or Bing, excepted that a human-looking bot answers you... And only shows the first page everytime. $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 1:45
  • $\begingroup$ @Elemtilas Here, you have a sample of what it does with this old question of mine. I can provide more with different technics of asking if you wish. $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 2:13
  • $\begingroup$ Are there some guidelines for moderators on how to deal with a user who consistently post misleading (factually wrong) answers? $\endgroup$
    – Igor G
    Commented Jan 6, 2023 at 12:24
  • $\begingroup$ @IgorG This is outside this post's main topic, you might want to ask your own meta-question. Here's a short answer though : Downvotes and comments :). $\endgroup$
    – Tortliena - inactive
    Commented Jan 6, 2023 at 14:04
  • $\begingroup$ @Tortliena, I'm sorry, I thought the duscussion here was about ChatGPT and acceptability of its answers? I was endeavouring to point out that the wrong-to-correct fact ratio in texts produces by ChatGPT is alarmingly high. With implied hint that ChatGPT should be treated like a consistently wrong user who does not learn from downvotes, never reads comments and just keeps posting nonsense no matter what. I expect, if some human user behaved like that, the mods'd soon get annoyed and ban them? $\endgroup$
    – Igor G
    Commented Jan 6, 2023 at 14:13
  • $\begingroup$ @IgorG Ah, under this perspective ^^'... Well, there's an automatic system to ban users with low-voted posts. You don't even need mods (though they act from time to time). $\endgroup$
    – Tortliena - inactive
    Commented Jan 6, 2023 at 15:20

13 Answers 13

Reset to default
34
$\begingroup$

We should follow SO's example and ban answers generated by ChatGPT for now.

We already have an incredible creative community and can regularly see questions getting 10+ answers. We also have some very prolific users who have individually answered over 5,000 questions. I'd rather recognize and elevate the hard work of our community over whatever can be cranked out by a bot. If someone wants to get a chatbot answer they can always ask bot directly on their own.

While it's technically possible to derive a prohibition of ChatGPT from existing policy, it's better to make that prohibition an explicit expectation, than to rely on everyone knowing and interpreting site policy in the same way.

I strongly suggest reading Tim C's answer for a more detailed exploration of why ChatGPT and it's ilk are a unique problem, and why creating a specific policy about it is needed.

$\endgroup$
16
  • 1
    $\begingroup$ I suppose it's a question of what's more valued: an answer to a question, or the source of the answer. Asking chatbots isn't necessarily straight-forward either. Different ways of asking can yield very different results. I think there's even a budding career path for people who are simply given the task of asking questions in the right way to achieve useful answers. A nail gun is a pretty straightforward tool, but there are still a lot of wrong ways to use it. $\endgroup$
    – JamieB
    Commented Dec 15, 2022 at 19:53
  • 2
    $\begingroup$ We want good answers. Spammy low quality answers that look like they could answer the question, but upon careful inspection fall apart are particularly problematic. If you want that you could do what a lot of successful writers do. Skip this site entirely. Write a cool story with whatever cool ideas you can think up and don't sweat the details. $\endgroup$
    – sphennings
    Commented Dec 15, 2022 at 20:00
  • $\begingroup$ Then it sounds like you agree: it doesn't matter if the answer came from ChatGPT or not. They just should not be blindly pasted. A good answer is a good answer. The source is irrelevant. Similarly, a bad answer from ChatGPT is no worse than, and should be downvoted exactly like, a bad answer from a human. $\endgroup$
    – JamieB
    Commented Dec 15, 2022 at 21:11
  • 4
    $\begingroup$ Except that ChatGPT enables the mass production of low quality answers at a rate far greater than even our most prolific answerers could produce (to their credit, they put real effort into their answers and it shows the quality). Spamming low quality answers would create additional burden on the moderation of this site. Our mods are already overworked. In the future we can and should revisit banning ChatGPT down the line, but at the moment all I see is a lot of downsides for little if any actual benefit. $\endgroup$
    – sphennings
    Commented Dec 15, 2022 at 21:32
  • $\begingroup$ The real problem we're trying to avoid is "spamming low quality answers". We don't want humans doing that either. Hopefully there is already a system in place to tap down any user who spams low quality answers, regardless of how they're doing it. I'm saying there's no apparent value in making rules about a source (especially one that can't be verified) when we already have rules about quality. $\endgroup$
    – JamieB
    Commented Dec 15, 2022 at 21:58
  • 1
    $\begingroup$ Question spam is only one of the problems with ChatGPT. Being explicit about not permitting ChatGPT is clearer, than deriving action from existing policy. Many of our rules are exactly for this reason. You can derive our no questions about story rule, from other rules, but it's clearer to just say "don't ask about stories", than to have to explain how story questions violate a bunch of other policies every time someone posts a story question. $\endgroup$
    – sphennings
    Commented Dec 15, 2022 at 22:08
  • 3
    $\begingroup$ Human answer spam is limited by typing speed. Chatbots don't have that problem. $\endgroup$
    – Robert Rapplean
    Commented Dec 15, 2022 at 23:03
  • $\begingroup$ @Sphennings "Except that ChatGPT enables the mass production of low quality answers at a rate far greater than even our most prolific answerers could produce[...]. Spamming low quality answers would create additional burden on the moderation of this site. Our mods are already overworked." This is irrelevant to the point of banning explicitly, indirectly or allowing deep A.Is.. The mods will in any case need to moderate this, regardless of which rule is in place! $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 1:29
  • $\begingroup$ @Tortliena Having a clear explicit rule reduces the moderation burden from everyone who would use ChatGPT. To only those who are ignorant of the policy or willingly chose to ignore it. Much easier to say "we have a rule against X. Please follow it" for those who are ignorant than to expect them to behave appropriately when there isn't an explicit guideline in place. $\endgroup$
    – sphennings
    Commented Dec 19, 2022 at 4:14
  • $\begingroup$ @Sphennings Really? You know that even old users skip reading the rules (see the last sandbox post), do you think that the people who will actually make big harm to the site's answers with this bot will read them or care what others say? They're not the same audience as the few who read in-depth the rules. In comparison, new rules slow down the process and reduce the likelyhood anyone read the rules. It's an understandable desire to write new laws against new things, but let's not scribble things with absolutely no data to ease the fear, our fear. ChatGPT is only 1 month old, after all :). $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 8:19
  • 1
    $\begingroup$ @Tortliena Right now we don't want ChatGPT on this site. Should the situation change we can always revisit that in the future. Can you explain how "Don't use chatbots to produce answers on this site." is a complex or in depth rule? Having a simple clear rule like this makes it easier to deal with people willingly violating policy, as well as the far more prominent group of people who are actually engaging with this site in good faith. $\endgroup$
    – sphennings
    Commented Dec 19, 2022 at 17:56
  • $\begingroup$ @Sphennings Among other things, you'd have to define what a chatbot is, if it includes copy-paste and how much we can't use it. It's already worth one full paragraph. Given that the targets skip them anyway, the ratio benefits/cost isn't that great. Moreover revoking a rule is always harder than setting it. It's like politicians jumping to recent dramas and creating new laws to "please the population" for a lack of better word : It looks good in the short term, but it's not always beneficial on the long term. I think that we are falling in this case, one month is too short to know. $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 19:46
  • $\begingroup$ We already have rules which prevent the site's main goal : If the bot is bad (and it isn't top-notch right now), answers are bad with the consequences you know. Also, it will soon no longer become free like other OpenAI's product, because this technology is very expensive. That caused a good loss for AI Dungeon's Gamemaster AI for instance. That's why taking action now is too soon to know. $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 19:52
  • 1
    $\begingroup$ @Tortliena We can deal with edge cases as they come up. No need to prevent any action in the meantime. So far there isn't a compelling case to permit ChatGPT. Should someone have a compelling case they can use that to campaign for a change in policy. Whether it's free or not should not matter in our assessment. $\endgroup$
    – sphennings
    Commented Dec 19, 2022 at 20:38
  • 1
    $\begingroup$ Boldface this: “Especially since if someone wants to get a chatbot answer they can always ask bot directly on their own.” WB is the source for chat bots, not the sink! $\endgroup$
    – SRM
    Commented Jan 19, 2023 at 3:55
31
$\begingroup$

+1 on banning it, for the simple reason:

If the asker wanted to ask ChatGPT the question, they could have done so themselves. If they're posting here, it's because they want a human to answer.

$\endgroup$
2
  • 1
    $\begingroup$ Light disagree if only because there are different ways to ask the same question, yielding different results. You can even specify things like "in simple terms" and get a very different answer. One can argue that "AI wrangler is some other website's job" but my argument is that "human only answers" is actually some other site's job. This site's job is "answers, with voting to determine the best one", and we're doing a disservice to the community by making hard rules about AI contribution. $\endgroup$
    – JamieB
    Commented Dec 16, 2022 at 15:16
  • 10
    $\begingroup$ All of the SE sites already have chronic issues with "plausible sounding answers that are wrong". Allowing GPT answers would put that problem into overdrive. $\endgroup$
    – FuzzyChef
    Commented Dec 16, 2022 at 16:48
23
$\begingroup$

I would agree with banning chatbot answers for numerous reasons.

  1. From a plagiarism perspective, it would be functionally identical to aggregating everyone else's answers into a single post. It just scrapes the answers from multiple sources. I'm pretty sure that there are already rules against that.

  2. ChatGPT, in particular, is good at providing an overview of a topic, and can even provide some basic advice, but it presents clearly generalized information, without consideration for the specifics of the question at hand. Cut and paste from a chatbot is the antithesis of well thought-out analysis.

  3. The points system's purpose is to incentivize great minds to provide thoughtful responses. Even if the chatbot answers are 100% correct, it undermines the intent of the points system and discourages participation.

  4. Our world is already suffering from sites that repost other people's answers, making finding a different, accurate answer difficult. This is especially a problem when reality slides out from underneath recorded answers. Reposting chatbot answers would make that worse.

$\endgroup$
3
  • 1
    $\begingroup$ Although it's worth considering that we are on the cusp of AI being coherent. The real question in my mind is should the site have a blanket ban against AIs. "Bad answers" are already downvoted or, in extreme cases, deleted, and we definitely don't need teams of AI using Stack Exchange for training, but I'd worry about excluding actual good answers from future AIs because, well, we decided here and now to discriminate based on the source. $\endgroup$
    – JamieB
    Commented Dec 15, 2022 at 21:18
  • 2
    $\begingroup$ I have a hard enough time finding decent answers to questions because too many sites just repost other's original work. Even the research papers are overloaded with remixes of other research papers. We're already in a dangerous cognitive situation because Google keeps everyone in their own bubble. I fear that flooding people-answer sites with chatbot answers would make that worse. $\endgroup$
    – Robert Rapplean
    Commented Dec 15, 2022 at 23:00
  • 1
    $\begingroup$ +1 for points 1 and 2. Especially 2. ChatGPT is great at answering questions that have already been answered in it's training set; its actual creativity is fairly low (possessing any creativity for an AI is insane!). This is an argument specifically against ChatGPT; its not likely to provide good quality answers on worldbuilding.SE. On other sites, like SO, this would not apply (it's actually quite good at basic coding). $\endgroup$
    – Rafael
    Commented Jan 17, 2023 at 5:31
14
$\begingroup$

An AI that can answer creative questions will perhaps exist in our lifetimes. But that's a research tool in competition with Stack Exchange, not a research tool complementing Stack Exchange

TL;DR: We should ban the use of ChatGPT, and not just ChatGPT, but globally ban the use of any AI that supplants human creativity, talent, and/or education.

I remember the early days of texting. No keyboards, just number pads, like those you'd find on old AT&T keypad phones. Competitions existed to see who could "text" a message quickly. You don't see those anymore, but you do see the latent consequences of that cul de sac technology. Abbreviations like "lol" and people who insist on posting messages without capitalization or punctuation. There was a time when forum posts and emails were hard to read — because people had grown accustomed to an imposed culture due to the limitations of technology. I remember posting in one such forum, and receiving an answer that basically said, "I can't answer your question, but I wanted to thank you for writing your post using good English."

It takes time to restore balance when a new tool is introduced

Whatever ChatGPT may be, what it is, is automation. Automation has value! But it can also lead to laziness. To sound overly religious and remarkably cynical, automation leverages human ingenuity to raise the productivity of everyone at the cost of the human soul.

Which is an artistic way of saying, automating the process of building cars has removed innovation and ingenuity from the process of building cars. The people still involved in the process of building cars (not the process of building or maintaining the automation, but actually building cars) are as a group no longer creative. They have been reduced to the lowest common denominator. Because they need not be and are expected to not be anything more than an adjunct to the more efficient and profitable use of automation.

If you think that sounds ridiculous, I can remember the High School teacher moderating my AP Mathematics test providing that explanation as a justification for why we couldn't use calculators to take the test. He may have been right, he may have been wrong, but having watched generations of U.S. High School students graduate using calculators to take tests, I can tell you that while they are more empowered for possessing the automation, they are also something less because they depend on it.

There may come a time when we find the balance that allows ChatGPT to be used as the useful tool it may yet become — but that time is not today

Right now we're in the "you can't use calculators to take your math test" phase because (a) ChatGPT is not wholly capable of replacing the human mind, (b) students Worlbuilding.SE users have not yet learned how to not be crippled by its use, and (c) how we write tests Stack Exchange as a whole and Worldbuilding in particular have not yet learned how to incorporate that new tool into our day-to-day of doing things.

In other words, today... it's cheating... because people are not using the tool to improve their value, they're using it to replace their value in a way that makes them look valuable. Such people may be gaining some social value from feeling artificially useful and/or popular, but what they're really doing is simply farming reputation. It's feeding a social addiction that isn't healthy for the user and isn't valuable for the Stack.

JBH, you're completely full of yourself...

Maybe. OK, probably, but that doesn't change the facts. It's just like texting back before the technology to text had become mature, or using a calculator to take a math test before mathematics teachers had learned how to incorporate calculators into the testing process. It's nice to suggest that Stack Exchange's only purpose is to help people find quality answers — but the flip side of that is the goal to bring intelligent, creative, experienced, talented, educated, and skilled people together so that people are improved.

Because the machine simply won't care if it's improved or not. In fact, it won't care if the querent is improved or not, too.

Once we've learned how to incorporate this wonderful new tool, this amazing piece of automation into the Worldbuilding Stack Exchange process such that the respondent is benefited as much as the querent, then we can allow ChatGPT onto the Stack.

Until then, it's the equivalent of putting up with number-pad-text quality posts or allowing cheating.

And I wouldn't be surprised if, by that time, Stack Exchange has been replaced by the descendant of ChatGPT. Or, perhaps more prophetically, that Stack Exchange replaces respondents with that descendant. We users are a pestiferous group of people, and chatbots need no moderation.

Computers make excellent and efficient servants, but I have no wish to serve under them.
(Mr. Spock, Star Trek TOS, "The Ultimate Computer")

If you think I've been unreasonable, please explain to me the difference between using ChatGPT (or any other AI) and buying a black-market midterm paper.

Addendum: A recent question on Main asked for completely trivial help. It periodically happens. The querent didn't realize or didn't now how to ask Google for basic data and then use a calculator to get the sought-for answer. The question was rightfully closed, but I left a comment explaining the Google searches and calculator actions.

And then it hit me... The question represents exactly the problem I think we want to avoid. Respondents who never learn how to think for themselves because they're having too much "mindless social fun" (see the Help Center) farming rep. I believe it's just as wrong to encourage people to become addicted to ChatGPT as it is for people to become addicted to Stack Exchange.

$\endgroup$
19
  • 1
    $\begingroup$ For me it just boils down to "what is the point of this site". If the point is "answering questions" then AI is not in competition, but rather, is another tool that the site should probably just consider building in: you will always get 1 answer, automatically, from ChatGPT, clearly labeled and perhaps held separately, but automatically generated as a service. If you see the point as "humans answering questions" then that's fair, but, I think, not the actual point. People just want good answers and if an AI can provide it, I don't see why they would care about the source. $\endgroup$
    – JamieB
    Commented Dec 16, 2022 at 15:09
  • 1
    $\begingroup$ @JamieB Were the point of that site that simple, most of its behavior is meaningless and painful. But, for the reasons I've mentioned, I don't think ChatGPT is ready for Worldbuilding and I don't thing Worldbuilding is ready for ChatGPT. It's like buying a first-model-year car... let someone else work out the bugs. $\endgroup$
    – JBH
    Commented Dec 16, 2022 at 15:27
  • 1
    $\begingroup$ To answer your last question in your OP: if all you need to understand is how a lack of oxygen could impact plant development in your world, a black-market midterm paper would be fine. You need a plausible world-building explanation, not a thesis for your degree. It's not cheating to ask an AI. The real job of an answerer, if they use an AI, should be to scrutinize the answer, see if it makes sense, see if it's original, and clean it up a bit if necessary. But "Ah ha! This answer is great and on-topic but has a sentence from an AI! Deleted!" is not a thing that should ever happen. $\endgroup$
    – JamieB
    Commented Dec 16, 2022 at 15:42
  • 2
    $\begingroup$ @JamieB --- AI is entirely beside the point. The reason WB.SE exists is so that we can help people build worlds through the process of answering questions about their specific and focused problems. AI is not "competition" for this activity, because AI can't get a SE account. I also doubt that AI (in particular ChatGPT) can really give a good answer to a good worldbuilding query. Using AI as a tool in a two edged sword. You get an answer, but if you just copy and paste and do no research, you don't know if the answer is good or bad. Since you have to check and correct (cont) $\endgroup$
    – elemtilas
    Commented Dec 16, 2022 at 20:31
  • 1
    $\begingroup$ (cont) --- the AI, you might just as well spend the time writing your own answer. If research and writing aren't your forte, you shouldn't be on this kind of forum at all. I think you might be misleading yourself in calling ChatGPT a "tool". It is not. At best it seems to be an aggregator of information and a spitter out of information in a way that looks reasonable. Looking at the two examples HDE quotes, I'm not convinced the AI is all that useful at giving the kinds of answers that we, as a community, can be proud of. A tool is something that a person uses to create something (cont) $\endgroup$
    – elemtilas
    Commented Dec 16, 2022 at 20:37
  • 2
    $\begingroup$ (cont) --- else. In our case, we use tools like WP, like PubMed, like standard references, like our own professional, artistic, and personal experiences to create content. Copying what an AI spits out is not an example of using a tool to create content. It's an example of cheating. There is a reason why we downvote / close / delete / excoriate answers that are just copy and paste from Wikipedia. Same thing here! $\endgroup$
    – elemtilas
    Commented Dec 16, 2022 at 20:39
  • 1
    $\begingroup$ @elemtilas I actually haven't tried to see what ChatGPT would say to these questions (I know what I'm doing tonight!) but if you're right, then again my point stands: we don't need rules about how to handle bad answers. If ChatGPT can't answer these kinds of questions coherently then the problem has solved itself, and existing rules should be sufficient. Remember the real bottom line question here is "do we need a new rule". If existing rules already cover the real problem, then there's no issue. [Do we already have a rule covering accounts that spam bad answers? I assume we must.] $\endgroup$
    – JamieB
    Commented Dec 16, 2022 at 20:40
  • 3
    $\begingroup$ Of course we've already got a process for bad answers. I don't think that's the issue here. The issue seems to be the MISUSE of a text generator to make answers, rather than making the answer yourself. Copy and pasting someone else's work is plagiarism, whether that "someone" is a WP writer, an AI or that AI's programming team. So, yes, I do think we need a new rule. Or at the very least, a specification of an existing rule. $\endgroup$
    – elemtilas
    Commented Dec 16, 2022 at 20:44
  • 1
    $\begingroup$ @JamieB You're acting like answering the question is the only useful consequence of Stack Exchange. Nothing could be further from the truth for Stack Exchange or Worldbuilding. While some users are only interested in posting questions, and other are only interested in posting answers, some of us are interested in helping people become better users of Stack Exchange and better worldbuilders. It's impossible to teach ChatGPT to improve at either. And your beliefs appear to be that those last two goals are irrelevant. If that's so, we don't need SE, just ask your question over at ChatGPT. $\endgroup$
    – JBH
    Commented Dec 17, 2022 at 6:38
  • 1
    $\begingroup$ You seem stuck on the idea that there can be only 1 answer, and if we let chatbots give it, then there's no point in the rest of us. Possibly that's true in a math or programming stack, but in worldbuilding (and many others), answers are subjective. Even if the chat AI gave good answers, there's always other angles to take. The human perspective would still be useful. As would the AI's input. In many cases, several answers together provide the real solution. We don't delete answers just because they didn't get the checkmark. Why limit yourself to just humans? Or just AI? why-not-both.jpg $\endgroup$
    – JamieB
    Commented Dec 18, 2022 at 4:10
  • $\begingroup$ @JamieB You seem stuck on the idea that Stack Exchange's rules and model of operation don't matter. Were you suggesting ChatGPT had its own account such that people where voting on ChatGPT's answers to ChatGPT's benefit or detriment, I'd be supporting you 100%. But you're not. If the OP wants ChatGPT's opinion, nothing is stopping the OP from getting it. $\endgroup$
    – JBH
    Commented Dec 18, 2022 at 20:39
  • $\begingroup$ We're here discussing the "rules and model of operation", specifically whether they need modification in light of potential future AI answers. So not only do I think they matter, but I think they need to be forward looking and not knee-jerk (and not based on what's happening in Stack Overflow, which has a very different type of Q&A.) Answers should be policed (which we already do) case by case (like now), and if the AI gets better then the "rules and model of operation" will already have it covered. Unless you think we should revisit this every time there's an update. $\endgroup$
    – JamieB
    Commented Dec 19, 2022 at 14:26
  • $\begingroup$ @JamieB Congratulations. We disagree. Welcome to the essence of democracy. I'm in favor of a global ban for a minimum of two years to give Stack Exchange time to work out how they want to deal with either (a) giving a machine an account, which I'm OK with or (b) whether to let humans use the voice of a machine as a substitute for education and experience, which I'm not OK with. If you don't like it, down vote my answer and move on. $\endgroup$
    – JBH
    Commented Dec 19, 2022 at 15:55
  • $\begingroup$ I agree with some points in the post, but if we're arguing "it's cheating now, but will eventually be like a calculator", we need to establish what would make it not-cheating (that is, at what point should the rule be revisited?); and what can we do to move the needle towards that better future? (are there non-cheating uses we'd support?) $\endgroup$
    – Dewi Morgan
    Commented Dec 20, 2022 at 0:13
  • $\begingroup$ @JBH Laziness in it's essence is catalyst for the invention, we wouldn't be here In terms of technology if some people weren't been really lazy. That's aside banning tools like this saying that "They are supplants human creativity" is meaningless since true creativity will not going to be stopped by any third parties. If we are expecting a future where we are able to incorporate tools like ChatGPT, banning them now going to halt it's natural process of integration. We must BAN incorrect answers but not the bot itself. $\endgroup$
    – vivus
    Commented Dec 28, 2022 at 8:03
9
$\begingroup$

While I'm not as active as I was once upon a time, I did see that block of chatGPT answers come through, and for the most part they were as I expected and matched the descriptions of chatGPT output I've heard about. It was best described as Dunning-Kruger As-A-Service, fluent, confident, apparently well written, utter twaddle.

In summary, ban it to save us arguing over every meaningless response it churns out with no benefit to anyone.

$\endgroup$
1
  • $\begingroup$ I'm not sure referencing an article which itself states an opinion referencing Stack-Exchange brings much value. Self-looping reference, you know :p. But yes, right now the A.I. doesn't tell everytime it could be wrong, which can be very misleading when it is in fact wrong. You really shouldn't rely solely on it, like any other source, in fact. $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 16:15
7
$\begingroup$

Let's ask directly ChatGPT!

-Tortliena : Hi, on Stack-exchange a question about you has been raised. Do you think we should ban posts using ChatGPT or similar software?

-ChatGPT : As a language model, I am not able to provide a personal opinion on whether posts using ChatGPT or similar software should be banned on Stack Exchange or any other platform. It is up to the individual platform or community to decide their own policies and guidelines for acceptable content.
In general, it is important for platforms to have clear policies in place to ensure that users understand what is expected of them and to create a positive and respectful community. However, it is also important to consider the potential benefits and drawbacks of any policy decision, and to be transparent and fair in the decision-making process.

-Tortliena : Are you good enough to answer specific problems about worldbuilding?

-ChatGTP : As a language model, I have been trained on a wide range of texts and topics and may be able to provide helpful information and insights on worldbuilding. However, it's important to keep in mind that I am a machine learning model and do not have personal experience or expertise in any particular field.
I can certainly try my best to help with specific problems related to worldbuilding, but it may be helpful to do some research and gather additional information on your own as well. The process of worldbuilding involves creating a fictional world or setting, and can involve many different elements such as geography, history, culture, and more. It can be a complex and creative process, and it may be helpful to consider various factors and make informed decisions based on your specific goals and vision for your world.

This is not made up, here are the tusks of proof: 1st picture and 2nd one. The lines preceded by ChatGTP are generated by an A.I. and not an actual person. Output's rights assigned to input's author according to OpenAI's terms of use.

Quite wise. I've highlighted what I think is the most important : This A.I. (or any A.I. we know today) can't claim to be an expert, especially on complex topics like worldbuilding. Stack-Exchange's goal is to have the best answers to every question. Not being an expert in the given field makes a pretty bad start to write "best" answers 🦋. Then, as advised by the bot itself, people should gather additional pieces of data themselves. Not doing so, is well... Likely to prevent you from making informed decisions. Also, if you rely solely on an A.I. to give an answer, it also probably mean you're not willing or cannot provide the skills people need. Not nice!

My point of view is therefore simple : It's like 2-minutes long searches on the web. If it's to take a 1st broad glance, it's alright. But you shall always cross-examine the output as well as you need to look for more, much more with your own skills, experience and writing style. Don't copy-paste the first result as your answer, since this contradicts a major part of what good answers are. Moreover, answering without actual effort show lil' to no respect nor patience to the querent, who are spending a lot of time building their world and are asking for your help.

N.B. : This answer has fallen in a funny paradox now : If this answer is upvoted because ChatGTP should not be used, then it has to be downvoted since it heavily uses ChatGTP. But if it is downvoted, the statements inside are no longer valid, meaning using ChatGTP is okay and therefore it should be upvoted if using ChatGTP is not good 🔄. Hawk, my head x)...

$\endgroup$
4
  • $\begingroup$ Anecdotally, I spent a little time over the weekend running questions though ChatGPT to see how it does. Sometimes it's very good. There was an obscure C syntax question on Stack Overflow that it answered as well or better than the best answer. Other times, it gave confidently worded answers that were straight up wrong ("did Dr. Who have facial hair"). So yeah, it can be useful, and sometimes you can even copy-paste and have a great answer, but more often you'd end spending time fact-checking its answer and having to throw it away or just use it as inspiration for your real answer. $\endgroup$
    – JamieB
    Commented Dec 19, 2022 at 16:02
  • 1
    $\begingroup$ ... in summary, it's a tool, but like any tool, one cannot swing it around willy-nilly. I don't think it's as useless as a lot of people here seem to believe, though. $\endgroup$
    – JamieB
    Commented Dec 19, 2022 at 16:03
  • 2
    $\begingroup$ @JamieB Yes neural network AIs are far from being perfect; From my experimentations with it, chatGPT still need work to be reliably used alone, but I've seen it cover most actual user answers in a single swipe, sometimes. It could reveal paths to check if used wisely and correctly cross-referenced. $\endgroup$
    – Tortliena - inactive
    Commented Dec 19, 2022 at 16:17
  • 2
    $\begingroup$ What I like most about ChatGPT's responses is that you can replace the word "worldbuilding" with any other noun in any human language and the answers would be just as relevant to said noun. 'O course, what ChatGPT is really saying is, "don't trust me." That's a good enough reason to ban for me. $\endgroup$
    – JBH
    Commented Dec 26, 2022 at 22:00
7
$\begingroup$

Yes, a site-policy against ChatGPT is safe and useful

I'm late to this discussion, so I'll avoid retreading what's already been said, and focus only on a counterargument I see going on in a lot of the comment threads:

  • "We don't need this rule because we already know how to handle low quality answers" and its variant, "We allow low quality answers, why is ChatGPT any different?"
  • "We can't enforce this rule because it's hard to spot ChatGPT answers."
  • "This will lead to false positives."

I argue that ChatGPT answers are both qualtiatively and quantitatively different than run-of-the-mill low effort answers, and that a rule against them will simplify moderation.

ChatGPT acts like a ban-evading hydra

If we ban someone for being disruptive, and they come back under a new account (in ways that we can prove is the same user), a sensible community doesn't treat the new account as a new person and start over, giving the "new" member of their community a chance - they ban the sockpuppet.

If we find the current instances of ChatGPT disruptive enough to ban, we should treat the others as sockpuppets of the same. It's an extraordinarily well-resourced hydra that gets humans all over the world to sockpuppet for it, but it's a hydra, nonetheless.

ChatGPT users have no possibility for reform other than to stop using ChatGPT

If a human user posts an answer that is under-researched (or doesn't respect the [hard-science] tag), they will receive downvotes and comments explaining what's wrong. They might fix the answer, delete it, or just leave it downvoted, but whatever else they do, they will also incorporate that feedback into their internal model (or "Learn," as we humans call it). If their goal is to get upvotes, they will research their answers better. If their goal is to avoid criticism, they will leave the stack. Either of these outcomes is acceptable.

ChatGPT users are different. Any feedback we give to ChatGPT users gets blackholed. ChatGPT is not going to update its model in response to our feedback, in part because it never receives our feedback. The user might look at the feedback (and may even learn something from it), but since we're never seeing their writing anyway, the quality of their posts does not improve.

I think most ChatGPT answers are made in bad faith

I suspect most ChatGPT users know full well what they're doing. The technology has been discussed enough now that its users can be expected to know that ChatGPT doesn't know anything they don't. People who continue to post uncited ChatGPT answers are likely doing so knowing full well that they cannot defend the answer's quality.

I can only speculate about their motivations for doing so. Some might be earnestly trying to see how well ChatGPT does with creative and esoteric prompts and others might be trying to test our community to see how well we can distinguish robots from humans, and still others might be trying to have some "harmless" fun - but I suspect most are trying to farm accounts with history and rep (and the privileges that accompany it), to sell them to spammers.

Regardless of their motivations, all motivations I can imagine have two things in common:

  • A ChatGPT poster is not making a good-faith effort to provide a high-quality answer to the question asked.
  • A user whose first few posts are all unedited and unsourced ChatGPT responses is unlikely to ever contribute anything else to the site, and we should act accordingly.

So why a policy?

A policy against ChatGPT serves two purposes: it will reduce the incentive to post such answers, and it will make moderation actions against ChatGPT trolls more efficient and consistent.

Reducing incentive to post

People who post bad-faith ChatGPT answers are doing so with a purpose: they want an account with rep and history. If we make it clear, in policy and action, that repeatedly posting ChatGPT answers will not get them an account with rep and history, we take away that incentive. If we fail to do so, others will join them.

Making it easier to moderate

We do not, as far as I know, ban people for making low-quality answers in good faith. I certainly don't think we should ban users just because their posts are low quality (as long as they're responsive to feedback and don't repeat mistakes excessively). The problem is that usage of ChatGPT implies that someone will not be responsive to feedback and that they will repeat mistakes excessively. A policy against it allows moderators to act on that evidence, taking the usage of ChatGPT as evidence of bad faith, which will allow quicker, more consistent, and more objective moderation.

What about false positives?

I don't think anyone's suggesting a zero-tolerance rule for anything that looks like ChatGPT (like some of what's happened on r/art). ChatGPT answers might be difficult to distinguish from human answers, but ChatGPT users are not hard to distinguish from human users.

  • ChatGPT users can post long answers very quickly.
  • ChatGPT users don't meaningfully engage in comment threads, even if they have the rep to comment.
  • ChatGPT users don't fix their answers when problems are reported. (They often can't because they don't understand the corrections).
  • ChatGPT users never have unique insight. (No frame challenges, no lateral-thinking to solve an engineering problem with sociology, etc. - someone who does these things is probably not posting ChatGPT output.)
$\endgroup$
4
$\begingroup$

A simple allow-or-ban choice makes stuff simpler for mods, I suppose, but I dislike such binary thinking: it's almost always a false dichotomy.

At the moment, it's a useful tool for researchers to get ideas on direction, but isn't good enough for generating verbatim, unchecked answers. Its answers have potential value, but are in no way definitive.

WorldBuilding.se seems a good target audience for non-definitive, directional answers, but use of it needs to be clearly marked, and clearly distinguished from regular usage.

It feels shortsighted to entirely prevent use of a tool; like some early iteration of math.se preventing any use of calculation software in answers, or like StackOverflow requiring all answers be handwritten instead of typed.

Technology moves on.

However, early forms of technology are often deeply flawed, in implementation or usage. This AI makes claims like "atmospheric reentry slows craft down by moving kinetic energy to gravitational potential energy", which... no.

So I would support a timed ban (with, say, a promise to revisit the question "in 2005"). The ban needs an explicit sunset date.

But I'd also advocate a trial of a bot that automatically answers each question this way, to save users the trouble (and temptation!) of doing so, with a clear disclaimer at the top of each answer that it's a bot-and-AI-generated answer and may contain blatant falsehoods. Especially if there's a new checkbox "hide bot answers" added to people's settings.

$\endgroup$
3
  • 1
    $\begingroup$ Do you have an idea of how would the revision date should be set, if any should be set strictly? $\endgroup$
    – Tortliena - inactive
    Commented Dec 20, 2022 at 14:33
  • 1
    $\begingroup$ We should definitely revisit our decisions, especially as things evolve. I'd be very interested in seeing how the technology evolves and it's applications for building fictional worlds in the coming years. $\endgroup$
    – sphennings
    Commented Dec 20, 2022 at 15:19
  • 1
    $\begingroup$ @Tortliena I'd be open to suggestions, but I'd advocate for re-voting at durations no longer than every couple of years, because it's quite a fast-moving target. We could also go for much vaguer, user-driven criteria, like "anyone can call for a re-vote if circumstances change (if SE changes its rules on the topic, or significant advancements are made, etc), but there can be no more than one re-vote per year". I'd be fine with either; I just wouldn't want to see it considered a permanent, irrevocable decision. $\endgroup$
    – Dewi Morgan
    Commented Dec 20, 2022 at 21:35
3
$\begingroup$

I'm in favor of a blanket ban on AI's answers in the foreseeable future. On top of all the reasons already stated by everybody else... Even if we had a functional AI posting here and giving good answers, all it takes is one bug or a change in the AI's company management to turn it into a big headache for us.

Imagine an hypothetical scenario where some crazy dude buys the company that runs the AI and, because he has a beef with some site like Wikipedia, he orders the AI (or whomever manages its account here) to retroactively redact all links to Wikipedia from the AI generated answers throughout the year. Worse, if that account has edit rights, they could also do this site-wide, on everybody's questions and answers.

I think that even after they become fully sentient, we should keep any AI that is not fully independent out of the site.

$\endgroup$
4
  • $\begingroup$ You know the likelyhood of clipping directly the AI into Stack-Exchange with full power is extremely unlikely? In our current case all end-content isn't coming directly from the A.I., just from users asking the A.I. and posting the results, often without much thought/effort put into it. $\endgroup$
    – Tortliena - inactive
    Commented Dec 21, 2022 at 23:35
  • $\begingroup$ @Tortliena creating a script that reads a question and posts an answer is as easy as homework assignment for many programmers. Plugging it to an AI would be a next step. It likely has not been tried yet, but it's a matter of time. I actually feel tempted to make a proof of concept just to see the feasibility. $\endgroup$
    – The Square-Cube Law
    Commented Dec 22, 2022 at 4:07
  • 1
    $\begingroup$ The problem resides in the A.I. itself : It manages extremely poorly when putting all the post, and is far from being accurate if you only give the title. Its suspension by downvotes would happen quickly. It's just not meant to be used in an auto fashion ^^. $\endgroup$
    – Tortliena - inactive
    Commented Dec 22, 2022 at 10:58
  • 1
    $\begingroup$ Stack exchange temp-bans IP addresses that make to many requests (don't ask me how I know). Many modern websites today have ways of detecting and blocking bots from directly accessing them. Blocking a script that uploads ChatGPT answers is nothing new. $\endgroup$
    – Rafael
    Commented Jan 17, 2023 at 5:20
3
$\begingroup$

I will go with the grain and agree that AI generated answer should not be allowed, at least on THIS SE.

this is an artistic SE and as such we are searching for answer to improve our work, so the concept of

the issue on SO was different to me, because there you need to give out the right answer and explain it , chat gpt answer were giving answer that were consistently wrong, but giving advice that did look just right enough that beginner could misstake them as functional. where as in here, while you can agree more with one answer, most if not all of them are correct in their own way, but simply another way of answering your prompt. (except maybe for answer asking for hard science, but those are a particular case)

using ready made answer is totally contrary to what's been done here, which is to offer elegant and artistic solution to an artistic problem.

$\endgroup$
-2
$\begingroup$

I don't think a ban on ChatGPT is meaningful, because it would be unenforceable. In practice, there is effectively nothing we can do to detect someone occasionally posting answers from ChatGPT (with just a little bit of babysitting and touch-ups). A ban that's unenforceable is not a useful ban. Imagine banning bad/wrong answers. We may not want them on our site, but there's no reasonable way to stop people from posting them here.

The fact that ChatGPT can't be effectively banned leads to the next question; why should it? The reason why it's undetectable is because with a little bit of babysitting, a user can provide generally useful and correct answers. Why is that something that needs to be banned?

It seems to me from this that we need to differentiate between 'malicious' use of ChatGPT, where multiple answers are posted very rapidly, with little/no babysitting, and answers that ChatGPT generates, but real users are ready to stand behind. A blanket ban would be useless, as the latter category is virtually undetectable anyways (think a user who every now and then gets ChatGPT to write an answer and then touches it up themselves). Banning specific users who abuse ChatGPT makes sense, but banning the software entirely is neither logical, nor possible.

$\endgroup$
7
  • $\begingroup$ I think you misunderstand what's going on. The proposed effort is to punish people who use ChatGPT, not to stop the use of ChatGPT. You're completely correct, it's impossible ban the tool, but that's an irrational statement because the tool isn't trying to make itself known on SE. This effort is entirely about users who use the tool. And for the record, if you think ChatGPT's responses only need a bit of babysitting to be useful, you might need to try using it to answer questions here a bit more. The goal is to keep SE from becoming congested with garbage by people who don't know any better. $\endgroup$
    – JBH
    Commented Jan 17, 2023 at 5:43
  • $\begingroup$ @JBH I don't understand your point. My remarks refer specifically to banning the use of the tool on this website, not in general. And that even so, it's impossible to ban many of its users. As to whether or not ChatGPT requires a 'bit' of babysitting or 'a lot', I don't think those are details that are central to this debate. $\endgroup$
    – Rafael
    Commented Jan 17, 2023 at 6:07
  • $\begingroup$ Actually, those details are the point. When asked esoteric questions (our bread and butter), ChatGPT churns out beautiful garbage. When users begin leaning on it as a crutch (especially younger users who haven't enough education or experience to know that what they're reading is garbage), it fills SE with useless answers. That's specifically the point. As for banning the tool, my point was that this discussion is about people using the tool. Not the tool. The "rule" would be "ChatGPT is not allowed." The consequence is that users get banned. $\endgroup$
    – JBH
    Commented Jan 17, 2023 at 6:28
  • $\begingroup$ We don't need perfect compliance with a policy for it to have value. We can't enforce compliance with our be nice policy either. People will break both policies and all we can do is react to it. However we still benefit from from having the policy as most people choose to abide by it. Simply saying this is the site we want has value. $\endgroup$
    – sphennings
    Commented Jan 17, 2023 at 10:52
  • $\begingroup$ @sphennings, that is part of my argument. We can ban users who write volumes of poor answers anyways, and we can't catch those who use it for good ones, so why ban it in the first place (at least as a blanket ban)? $\endgroup$
    – Rafael
    Commented Jan 17, 2023 at 13:55
  • $\begingroup$ It's much easier to take action if there's a clear and unambiguous rule against it. A clear rule will also reduce it's usage among the vast majority of users who choose to abide by site policy. As for why we don't want AI generated answers, look at any of the well written and well received answers to this post for multiple explanations. $\endgroup$
    – sphennings
    Commented Jan 17, 2023 at 15:09
  • $\begingroup$ @sphennings, it's quite the opposite. A clear and unambiguous ban will lead to mods struggling to figure out which answers are AI generated and which are just poorly constructed (or even the good ones!). A ban on users who post large volumes of suspect answers is far easier to follow. $\endgroup$
    – Rafael
    Commented Jan 17, 2023 at 19:39
-2
$\begingroup$

A weird option: WB could automatically plug any question into ChatGPT and post that answer, flagging it as a bot and pending its validity until people ver it. The answer would automatically be a community answer.

That process preempts any user trying to score karma points by submitting bot answers. It gets people answers fast if they’re available. And it lets the community try to improve on the bot.

$\endgroup$
1
  • $\begingroup$ I wouldn't want it spamming the main site, but I am curious how well it can answer worldbuilding questions. Perhaps a chat channel or something offsite. $\endgroup$
    – sphennings
    Commented Jan 19, 2023 at 4:24
-12
$\begingroup$

I'm tentatively in favor of keeping ChatGPT answers provided there was some value-add in how you asked the question.

e.g. You ask a site question. I copy-paste it into ChatGPT and copy-paste the result. This adds zero value. I would downvote these answers on principle.

However, I think your question would be interesting if answered by Conan the Barbarian. I paste your question into ChatGPT and append that ChatGPT should answer as if it was Conan the Barbarian (this actually works and I highly recommend asking it to write poems about things "in the voice of Conan the Barbarian". Entertainment ensues).

Basically, I realized there was a better way to ask your question, modified your question accordingly, and posted the result [ideally because it was really good, and not, you know, just kind of funny]. I have added value that you, the asker, might not have thought of.

On the other hand, it's not really a high requirement to ask people to rephrase things on their own. Use ChatGPT for ideas, but make your own answer rather than simply copy-paste it.

On the third hand(!?), "how would you know?" If I don't tell you that my answer came from ChatGPT, you might never realize it, especially on a site like this. It's more damning on Stack Overflow, where the AI making small, quirky errors stands out like a sore thumb, but on a site like this, I'm honestly not sure you wouldn't end up with a rule you have no way to enforce. All you'd really do is cause people to stop saying they got their answer from ChatGPT.

Unenforceable rules are silly rules.

Edit: Anecdotally, I don't know why anyone here is worried. I set out tonight to start asking ChatGPT questions from worldbuilding. There are three immediate problems:

  1. A lot of people ask questions that are frankly hard to parse as a human. You have to get to the "meat" of their question as they feel the need to overly elaborate and this just confuses the chatbot's answer.

  2. It's not at all creative, typically yielding a long and boring answer that seems to be more like a tangent than an answer. (Its answer to the recently asked poison dart question actually was not bad but highlights how it focused on the "organic" keyword and didn't creatively think of alternatives like all the humans did.)

  3. It is objectively wrong at times, especially if there's math involved. Like it knows the formula for calculating the distance to the horizon from an object in space, but it still gives you the wrong answer to your question.

Consequently, I don't think anyone would copy-paste a question into ChatGPT, copy-paste the answer, and expect more than downvotes in most cases.

Still, my conclusion is the same, if not moreso: the rule would be redundant with existing policies.

$\endgroup$
15
  • 1
    $\begingroup$ There's still value in choosing to create a rule even if the rule is difficult to enforce. You would reduce the number of people using ChatGPT to the subset of people who are willing to violate site policy to use it on this site. Even if that isn't perfect that's will still be a meaningful reduction. It also makes it much easier to take explicit action should it's use become obvious. $\endgroup$
    – sphennings
    Commented Dec 15, 2022 at 19:49
  • 3
    $\begingroup$ The point about enforceability is an important one to discuss. There are a number of stylistic fingerprints from ChatGPT answers that make them stand out; mods and folks aross the network have put in a lot of work lately to figure out and compile those. These work well across sites, regardless of subject matter -- and there are other characteristics of accounts which do this which makes them easy for us to identify. I'm sure that may change in the future, but for now, we can catch a lot of them. $\endgroup$
    – HDE 226868 Mod
    Commented Dec 15, 2022 at 20:03
  • $\begingroup$ @HDE226868 That sounds even more worrisome. Are we going to start banning people or deleting answers because they hit a certain number of flags that are stylistically similar to ChatGPT answers? I smell a lot of "false positive" witch burnings in our future. We'll do more harm trying to stop ChatGPT than we ever would have gotten from ChatGPT. $\endgroup$
    – JamieB
    Commented Dec 15, 2022 at 21:13
  • 1
    $\begingroup$ @JamieB Those are usually used to identify a candidate chatbot-generated answer; a way to confirm is to plug the question title or body into ChatGPT itself and see how similar it is. It'll give a slightly different result each time you ask it the question, but it'll keep some key phrases or sentences; the answers we've seen so far had significant overlap (word-for-word for some sections) with what I get when I ask ChatGPT the question. There are other clues sometimes -- if a new user suddenly posts five four-paragraph answers in a ten-minute span, a human certainly did not write those. (1/2) $\endgroup$
    – HDE 226868 Mod
    Commented Dec 15, 2022 at 21:32
  • 1
    $\begingroup$ I can't speak for what mods are other sites are doing, but I don't plan on taking action unless we're sure -- so that means the answer matches most general ChatGPT patterns and the answer structures it yields, and closely matches the various responses the software gives when I ask it the question. Another clincher, of course, would be a user who has used ChatGPT on other sites and is clearly continuing the pattern -- one of the other ones I deleted was from a user who had admitted it outright. Sorry 'bout that -- my original comment should have been clearer. (2/2) $\endgroup$
    – HDE 226868 Mod
    Commented Dec 15, 2022 at 21:35
  • $\begingroup$ @HDE226868 I'd still frown on deleting an answer, if it was useful, even if it turned out to be a copy-paste from some AI. Extra true in future years, as these AIs doubtlessly improve. The line in the sand should be "answer quality", not "answer source". Although I see a market for "ask-a-human.com" wherein all answers are from verified humans but for a Q&A site, I'm not sure it matters, so long as there is quality and accuracy. We can ding ChatGPT for inaccuracies, but then, we do that for humans too, so no special rules are needed. $\endgroup$
    – JamieB
    Commented Dec 15, 2022 at 22:02
  • 4
    $\begingroup$ Enforcement is a non-issue. You can't cite a post if there's no rule against it. The rule doesn't have to be 100% effective as long as it's useful in cases of abuse. $\endgroup$
    – Robert Rapplean
    Commented Dec 15, 2022 at 23:07
  • $\begingroup$ @JamieB Online there are always false-positives, like if you inspected for cheaters at coins throws. The two things which help these nefarious cases is that ban is (or at least shouldn't) be an absolute permanent -people can appeal through the contact form-, and most importantly how and where thresholds are set. From a pure utilitarian point-of-view, it should be set where cheaters are equally as nefarious as losing on user retention. $\endgroup$
    – Tortliena - inactive
    Commented Dec 16, 2022 at 1:38
  • 1
    $\begingroup$ I downvoted when I came to " I realized there was a better way to ask your question, modified your question accordingly." The Help Center states, "To clarify the meaning of the post (without changing that meaning)." I have my doubts that using ChatGPT to find "a better way to ask your question" can consistently (if ever) meet that expectation. Indeed, I don't see it as a plus when, on this Stack, it's far better to teach the OP how to improve their question vs. simply changing it so something you think is better because of ChatGPT. $\endgroup$
    – JBH
    Commented Dec 16, 2022 at 5:29
  • 1
    $\begingroup$ I think that banning chatGPT is pretty much neutral in effect. Like this post says, it’s pretty much unenforceable, and in any cases where it would be enforceable are already covered by existing rules. There is practically no point to this rule. $\endgroup$
    – Topcode
    Commented Dec 16, 2022 at 13:53
  • $\begingroup$ @JBH The trick I found is to ask ChatGPT to answer as different people. Its own voice is pretty neutral (and a real killjoy) but you can ask it to answer a question as if it was someone else and get a completely different result, in a different voice. This also, I believe, adds to the "impossible to enforce" problem. You can't predict the answer because you don't know precisely how the inputs were modified. $\endgroup$
    – JamieB
    Commented Dec 16, 2022 at 15:05
  • 2
    $\begingroup$ @JamieB As has been said elsewhere, just saying "Don't use ChatGPT" will have a positive effect. Unless you assume that everyone that would ever use it are willing to deliberately go against site policy. Saying that we shouldn't have rules because some people will ignore them is a silly proposition. $\endgroup$
    – sphennings
    Commented Dec 16, 2022 at 15:19
  • $\begingroup$ @sphennings I see it more like "only people with blond hair can post answers". There is no actual point to a rule that you not only can't enforce, but could result in the deletion of some answers, not because they aren't good answers, but because we made a pretty arbitrary rule about the source. $\endgroup$
    – JamieB
    Commented Dec 16, 2022 at 15:26
  • 2
    $\begingroup$ (a) This isn't that unenforceable. Yes, there may be instances where it's impossible to discern the source, but I've played with ChatGPT myself and it won't be that hard to detect a huge chunk of its use. (b) No law can stop a determined violator. That's a really bad reason to say there should be no law. (c) In our experience, 20% of the users using ChatGPT will represent 80% of the problem. It won't be that hard to catch them. (d) If you're worry about deleting answers is valid, then there's no longer a point to Stack Exchange, people should just ask ChatGPT. $\endgroup$
    – JBH
    Commented Dec 16, 2022 at 15:37
  • 1
    $\begingroup$ We have plenty of rules that are already redundant. Our policy around no 3rd party worlds is just an explicit clarification of our existing site policies. However being able to just say "Don't ask about 3rd party worlds" is far easier to communicate than expect everyone to interpret and apply site policy in the same way. Better to be explicit that we don't want ChatGPT than rely on everyone knowing and applying existing rules in the exact same way. $\endgroup$
    – sphennings
    Commented Dec 18, 2022 at 21:06

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .