# Phrasing questions such that answers can be objectively voted upon

A Bit of Background

While reviewing Q&As at GraphicDesign.se for some insight I read this post about logo reviews. That post (which is trying to deal with the balance between subjective and objective and seems pertinent to us), states in part:

Your question must be phrased in such a way that answers can be objectively voted on by the community.

We do not have a statement like this on our site, and it caused me to wonder whether or not we should. Then @Molot and I had a brief discussion about whether or not this question should be closed as Primarily Opinion-Based.

I'm not asking for input on that discussion, I'm only using it as an example of the problem with asking questions that can or cannot be answered objectively here.

As I started to ponder asking this question, I found this historical meta question here at Worldbuilding that touches on this very issue. It was looked at a lot, but not voted on very much.

Question

Should we incorporate the above statement, or a similar statement, in our Help Center?

A "yes" answer might be cause to review our definitions for VTC reasons to help clarify them. This would especially be useful for POB.

Disclaimers

• This is definitely worth discussing. We do have "Questions must be specific as well as answerable." in What topics can I ask about here?, where "answerable" links to What types of questions should I avoid asking?, which says to avoid asking questions where "every answer is equally valid". However, it is sort of buried in the noise down there.
– user
Dec 6 '18 at 18:21
• If this language is added, we need to be much more specific about what constitutes "opinion-based". I was yelled at on my last question (regarding magic) because it couldn't be answered with real-world "facts", only "author choice" (aka opinion, i.e. any answer would be equally valid because it was magic). However, questions about magic systems can't be answered with real-world facts, only inferences - which can be better or worse, and voted on accordingly. Guidance would have been helpful to clarify whether any question about magic is automatically opinion-based and therefore invalid. Dec 8 '18 at 5:16
• @thatgirldm, while the adoption of this language would require a change to the POB definition, the biggest problem is that we cannot change the text SE uses for all its sites. Obviously, no question about magic can be asked under that restriction. We have our own definition which is much more practical, but it's a never-ending war to get new users to read about it. Alas, until SE allows the POB definition's text to be changed, it's the best we can do. Dec 8 '18 at 19:05
• @aCVn Where "every answer is equally valid", looking at that shows it isn't exactly helpful. The example given is pure individual subjectivity. In fact, I wonder what questions would answers that are equally valid; apart from answers about taste, likes & dislikes. There are disciplines where there are sets of answers depending on their various schools of thought, say, economics or political science, but not all answers will be equally valid. Often mutually contradictory, certainly, but some answers will be invalid. Thank you for making me think about it. Dec 16 '18 at 8:30
• I think it's worth considering that it's possible for a question to meet this criteria for the OP, but not meet it for the rest of the community. I mean, even open ended stuff can technically meet the objectivity criteria if the OP has better knowledge of the type of answer they want, but can't clearly articulate that information. This criteria could also stifle approaches where the OP wishes to see what other people think is the best option, but isn't too picky about a specific answer (though that too borders on an invalid question). Dec 16 '18 at 12:06
• @a4android Regarding your economics and political sciences examples, one can argue that some answers are valid according to some schools of thought and the same answers are invalid according to some other schools of thought, but it's still possible to cite sources claiming that a given answer is correct. I don't think it's a problem if someone answers "according to sources A1 and B2, X is the case, which with your assumptions leads to Z". Primarily opinion-based is really more intended for the case where all any answer can say is "I think X, therefore Z" without backing it up in any way.
– user
Dec 16 '18 at 14:03
• @AustinHemmelgarn stifle approaches where the OP wishes to see what other people think ... but isn't too picky... Regrettably, that's part of the problem. SE forces us to deal with this issue because we can't remove POB as an option to close. SE excludes that specific consideration as a valid perspective because SE is not a discussion forum. Dec 16 '18 at 16:03
• @aCVn, it's frustrating that a discussion of what's POB is intrinsically POB. Let me run a completely hypothetical question past you. Let's assume that We the People descided that POB simply didn't mean anything at WB.SE. So, rather than imposing our own definition of POB, we simpy exclude it (and the only policing is to remind people it's not a valid VTC reason). How would that affect criteria for low-quality posts? One-liners are easy, but what separates "that's just a comment" from "it's just my opinion?" Dec 16 '18 at 16:07
• @JBH I'm struggling to see what you are referring to. A comment shouldn't attempt to provide an answer; if it does, it's no longer a comment, but an answer. Questions usually need to be longer than a oneliner to provide sufficient information and context for a good answer. (One possible exception to that would be questions specifically seeking a definition of a term, but we don't get much of that on Worldbuilding.) A comment posted as a question is meaningless because there is no context for it. An opinion posted as a question is meaningless because there's likely no answerable question in it.
– user
Dec 16 '18 at 20:18
• To try to answer what I think you're asking, there's a difference between hypothetical and primarily opinion-based. A hypothetical question can be answered, possibly by making certain assumptions. Those assumptions can be stated, and we can discuss whether they are reasonable to make. A hypothetical question could be "assuming FTL communication, can paradoxes be avoided, and how?". A POB question is a question where any answer can only be based in opinion, rather than fact, references or expertise; "why would intelligent cat-like aliens of unspecified biology do or not drink tea?".
– user
Dec 16 '18 at 20:31
• @aCVn, that makes sense to me (the distinction, though not worded that way, is even in my POB proposal), but it appears there are too many people who don't want to be bound by any defintion of POB. If that's the case, what is the effect on other aspects of SE's constraints, like flagging as low-quality. Dec 16 '18 at 23:17
• @aCVn Validity & ivalidity according to different schools of thought was exactly what I was proposing. If POB is "Primarily opinion-based is really more intended for the case where all any answer can say is "I think X, therefore Z" without backing it up in any way." This is a reasonable proposition, unfortunately,POB VTCs often ignore that, & where valid & invalid schools of thought apply. Questions about politics or economics fall chief among them. Better if more WBers followed your standard for POB. Dec 17 '18 at 5:53
• @aCVn I agree with your distinction between hypothetical and primarily opinion-based. Good example involving FTL communication. However, "why would intelligent cat-like aliens of unspecified biology do or not drink tea?" can be asked with references (possibly) & expertise. The expertise, in question, is evolutionary biology. Namely, the probability of cat-like aliens and tea-like plants evolving on the same planet is so remote that they won't be drinking tea. To make questions like this example depart from POB, the area of expertise or knowledge needs to be cited. As seen here, Dec 17 '18 at 6:00
• @JBH A better approach to resolving problems with SE's definition of POB is not to redefine it, but to establish criteria more suitable for WB SE's questions & answers. I had previously adding that answers should be based on reason, logic, commonsense, and by analogy with history, culture, mythology, the social sciences, & other disciplines. I like aCVn's distinction between the hypothetical & POB. This too could be part of further refinement criteria. Better to refine & not redefine. I'm sure SE Supreme Command won't like various SE's having their individual definitions. Dec 17 '18 at 6:08
• @a4android, from a practical standpoint, there's no difference between refine and redefine because we can't change the POB text in the VTC popup and must still direct people to some Meta page explaining the refinement/redefinition. Semantics make for good arguments, but poor solutions. If we can't agree where the line is drawn for VTC:POB, then the easiest solution is to vacate its use (easy to understand, simple to implement) - but I'm worried about how that will impact other behaviors. Dec 17 '18 at 8:26

While I understand the motivations behind this proposal and actually support them, my concern is that the word 'objective' can itself (in this particular context) be subjective. I'd argue that it's possible (albeit unlikely) that this proposal, if implemented, could muddy the already murky POB waters. At best, it's likely to reduce magical traffic and I'll deal with that assertion first.

The proposal above actually does fit into our existing zeitgeist of POB definitions but my problem is that using the words objective and magical crosses the streams, so to speak. Either your system of magic is so well defined that the question involves no speculation whatsoever and you could extrapolate out the answer for yourself (but it's objective) or the question seeks a technical solution to a seemingly magical effect in keeping with the kinds of questions we like (but contains some measure of opinion on the 'best' approach).

When you get right down to it, the reason that POB gives us so much trouble is that it's a continuum - We know what blatant POB looks like, we know what blatant objective questions look like, but we're not really sure where one becomes the other; it's a boundary condition problem. Add to that the fact that we all have a slightly different perspective on where the boundary actually is, and we discover that the question of where POB starts should probably be closed as POB.

@JBH, I think that your constant efforts to help us define this are exemplary and worthy of the best contributors to this august site. It's clear that your passion to resolve this exceeds the patience most of us would have with dealing with this problem. That said, I'm wondering if in fact we (as a community) are approaching this problem from the wrong end.

Instead of trying to tie down as precisely as possible where the dial for POB should be set, perhaps it would be better for our serial VTC'ers on POB to show a touch more restraint.

I do not say this as a criticism; I know many of you are as passionate about this issue as is JBH. I say it more to point out that if you have the option to VTC and you have any doubt about whether or not to do so, perhaps you - well, shouldn't. In so doing, you leave the question for those who believe it's legitimate and want to provide an answer. If you're certain the question is POB, then close by all means, but this would allow people to set their own dials a little, and make their own judgement calls.

Alternatively, if you feel that aforementioned doubt, hold off on the VTC for 24hrs. We tell OPs not to accept an answer before 24hrs so that the entire globe gets a bite of the cherry, so to speak. Why not do the same (on a voluntary basis of course) if you're not sure about a question's POBness?

(I'm not recommending a 24hr waiting period on VTCs because I know that would cause more trouble than it's worth. All I'm saying is that a bit of latitude on doubtful calls might at least allow a VTCer to judge based on the relative merits of the answers as much as the relative merits of the question.)

This approach would at least afford those who fall on the side of it being a legitimate question posting an answer and potentially getting it accepted. It might also give VTCers some context (in terms of possible answers) before making a final decision. It would also (in theory) allow site contributors to make up their own mind on where the dial should be set and answer questions before they get closed if they feel they have an answer that is objective, even if the question isn't.

To quote a famous fictional union rep, what we seem to be asking for is clearly defined boundaries of uncertainty and doubt. Perhaps (with respect,) we need less effort in policing this issue, not more effort in defining it.

• My concern is that, lacking boundaries or influence, people are very unpredictable. Ambiguously asking them to be more tolerant rather than pointing them to a clearly defined rule won't (with sincere regret) work. It has never worked. Everything from fist fights in a bar to wars between nations all too frequently exist because people do not know or refuse to care about clear rules. I'd love to ignore the issue and trust everyone to work for the greater good - but the people who need to hear this aren't even reading this post. Law + order vs. liberty + chaos. it's an old battle. (*continued*) Dec 14 '18 at 16:01
• And the worst part about it is that a line has already been drawn by SE. A line that doesn't work for us and yet is constantly being used to judge questions. Call it OCD (it probably is...), but whether I VTC with a comment as to why the question is POB or comment to VTCers that they're being to loosey-goosey with the POB flag (I've done both), my intent is to help the OP ask better questions - not help the greater (and uncontrollable) community vote with greater wisdom or compassion. I gave up that fight long ago. Dec 14 '18 at 16:06

The problem with the statement "Your question must be phrased in such a way that answers can be objectively voted on by the community." is that it provides zero guidance on what constitutes "objective". No-one needs to be a philosopher to know what constitutes "objective' is essentially "subjective". And especially here where the majority of topics that are concerned with worldbuilding are speculative, hypothetical, conjectural, fictional entities or partially all the way through to purely imaginary.

"Objective" may sound good. But what is objective? Ask any philosopher and you soon discover there is nothing obvious about what constitutes objectivity. Is it the real? But what is real? For example, I know neutrinos are real. But I've seen one, nor do I expect to, I know their existence emerged from theoretical physics and there are instruments that can detect them.

Magnetic monopoles emerge from various theoretical physical models, but we don't have instruments that have detected them. There are instruments that should be able to detect them (or so we think) if they exist. So can someone answer a question about magnetic monopoles based on the certain models of physics? Yes they can, how objective are questions and answers? They have to be based on the assumption that monopoles exist. But does that make them objective? Guess what! The answer is both yes and no. Depending on what assumptions you prepared to make. To the monopole tolerant, the questions and their answers are objective. The research papers can be read and considered. To monopole deniers, this is pure speculation and not objective.

The other problem with "objective" is that essentially collides with the issue, previously discussed on meta, about whether questions about non-realistic topics like magic, time travel or other hypothetical entities should be part of WB SE, and that only scientifically or technologically oriented questions are allowed here. It was decided they should be allowed.

While, in principle, it sounds fine to have an improved way of guiding people to make better decisions it is necessary to ensure that the people who apply that rule can make better decisions. That requires them to have sufficient understanding of the rules, the context(s) within which they are applied, a body of precedents about the ways such rules have been applied previously, and the knowledge and experience about making those decisions.

The answer to those things is obvious. The existing criteria for vote to close decisions are observed in the breach than in being followed. Adding a rule about objectivity will only another set of criteria to be ignored. Any expectation that it would lead to improvement is a chimera. To make objectivity work requires wisdom and compassion, and is there enough of that to do so?

One consequence of SE model is that it naturally excludes for consideration here many questions that should be dealt with as worldbuilding. There are times when I have considered there should be a "too good to close" button for questions that might be closed because they meet vote to close criteria, but are so worthwhile in terms of worldbuilding that they should be allowed to be asked and definitely should be answered, and so would be left open. After all, it is sort of thing that would improve WB SE and help build it up as a database of good questions and good answers. This, of course, will never happen.

Worldbuilding SE is very subjective by nature. I have noticed that what makes a good (that is, not closed, highly upvoted) answer often has to do with how exactly the answerer interprets the rules of the proposed world, and whether the questioneer/voters/mods agree with that interpretation of the rules of the proposed world (which could be the real world or not).

I think with the question you linked to, at the end of the day it was closed because it didn't feel like a specific question. One of the things put forth in the comments on that question was that there's only one data point--but I would submit to you that the number of data points hasn't been an impedence to many other questions getting happily answered and highly upvoted.

Any other criteria we could try to come up with would probably have the same problem: it would probably shut down a lot of good questions that we see on this site.

I think that the beauty of the worldbuilding SE is that it does what none of the other SE sites do: it lets you ask theoretical questions that can't be objectively answered, when you just want to discuss what sounds good from a story-writing perspective and you want to be as realistic as you can be, to sound feasible. We get questions that are in fields that are all over the map, in all of the different fields that other SE sites exist for (biology, programming, etc)--but the reason that worldbuilding exists is that those sites are based on only strict, provably reality. They're intended for professionals, not for fun.

I'm making broad generalizations here but I think the point is made. Worldbuilding SE's weakness is really its strength.