Phrasing questions such that answers can be objectively voted upon

Question

A Bit of Background

While reviewing Q&As at GraphicDesign.se for some insight I read this post about logo reviews. That post (which is trying to deal with the balance between subjective and objective and seems pertinent to us), states in part:

Your question must be phrased in such a way that answers can be objectively voted on by the community.

We do not have a statement like this on our site, and it caused me to wonder whether or not we should. Then @Molot and I had a brief discussion about whether or not this question should be closed as Primarily Opinion-Based.

I'm not asking for input on that discussion, I'm only using it as an example of the problem with asking questions that can or cannot be answered objectively here.

As I started to ponder asking this question, I found this historical meta question here at Worldbuilding that touches on this very issue. It was looked at a lot, but not voted on very much.

Question

Should we incorporate the above statement, or a similar statement, in our Help Center?

A "yes" answer might be cause to review our definitions for VTC reasons to help clarify them. This would especially be useful for POB.

Disclaimers

• I am a proponent of a specific definition for "Primarily Opinion-Based" that varies from SE's standard definition because no question about magic can be legitimately asked using SE's definition. It is a pain that we can't change the descriptive text for POB.

• I am a proponent of SE's opinion that SE is not a discussion forum, and that open-ended idea-fishing questions are intrinsically subjective and discussion-oriented.

• I am as guilty as the next guy for answering questions that would have been affected by the outcome of this question.

Answer 1

While I understand the motivations behind this proposal and actually support them, my concern is that the word 'objective' can itself (in this particular context) be subjective. I'd argue that it's possible (albeit unlikely) that this proposal, if implemented, could muddy the already murky POB waters. At best, it's likely to reduce magical traffic and I'll deal with that assertion first.

The proposal above actually does fit into our existing zeitgeist of POB definitions but my problem is that using the words objective and magical crosses the streams, so to speak. Either your system of magic is so well defined that the question involves no speculation whatsoever and you could extrapolate out the answer for yourself (but it's objective) or the question seeks a technical solution to a seemingly magical effect in keeping with the kinds of questions we like (but contains some measure of opinion on the 'best' approach).

When you get right down to it, the reason that POB gives us so much trouble is that it's a continuum - We know what blatant POB looks like, we know what blatant objective questions look like, but we're not really sure where one becomes the other; it's a boundary condition problem. Add to that the fact that we all have a slightly different perspective on where the boundary actually is, and we discover that the question of where POB starts should probably be closed as POB.

@JBH, I think that your constant efforts to help us define this are exemplary and worthy of the best contributors to this august site. It's clear that your passion to resolve this exceeds the patience most of us would have with dealing with this problem. That said, I'm wondering if in fact we (as a community) are approaching this problem from the wrong end.

Instead of trying to tie down as precisely as possible where the dial for POB should be set, perhaps it would be better for our serial VTC'ers on POB to show a touch more restraint.

I do not say this as a criticism; I know many of you are as passionate about this issue as is JBH. I say it more to point out that if you have the option to VTC and you have any doubt about whether or not to do so, perhaps you - well, shouldn't. In so doing, you leave the question for those who believe it's legitimate and want to provide an answer. If you're certain the question is POB, then close by all means, but this would allow people to set their own dials a little, and make their own judgement calls.

Alternatively, if you feel that aforementioned doubt, hold off on the VTC for 24hrs. We tell OPs not to accept an answer before 24hrs so that the entire globe gets a bite of the cherry, so to speak. Why not do the same (on a voluntary basis of course) if you're not sure about a question's POBness?

(I'm not recommending a 24hr waiting period on VTCs because I know that would cause more trouble than it's worth. All I'm saying is that a bit of latitude on doubtful calls might at least allow a VTCer to judge based on the relative merits of the answers as much as the relative merits of the question.)

This approach would at least afford those who fall on the side of it being a legitimate question posting an answer and potentially getting it accepted. It might also give VTCers some context (in terms of possible answers) before making a final decision. It would also (in theory) allow site contributors to make up their own mind on where the dial should be set and answer questions before they get closed if they feel they have an answer that is objective, even if the question isn't.

To quote a famous fictional union rep, what we seem to be asking for is clearly defined boundaries of uncertainty and doubt. Perhaps (with respect,) we need less effort in policing this issue, not more effort in defining it.

Answer 2

The problem with the statement "Your question must be phrased in such a way that answers can be objectively voted on by the community." is that it provides zero guidance on what constitutes "objective". No-one needs to be a philosopher to know what constitutes "objective' is essentially "subjective". And especially here where the majority of topics that are concerned with worldbuilding are speculative, hypothetical, conjectural, fictional entities or partially all the way through to purely imaginary.

"Objective" may sound good. But what is objective? Ask any philosopher and you soon discover there is nothing obvious about what constitutes objectivity. Is it the real? But what is real? For example, I know neutrinos are real. But I've seen one, nor do I expect to, I know their existence emerged from theoretical physics and there are instruments that can detect them.

Magnetic monopoles emerge from various theoretical physical models, but we don't have instruments that have detected them. There are instruments that should be able to detect them (or so we think) if they exist. So can someone answer a question about magnetic monopoles based on the certain models of physics? Yes they can, how objective are questions and answers? They have to be based on the assumption that monopoles exist. But does that make them objective? Guess what! The answer is both yes and no. Depending on what assumptions you prepared to make. To the monopole tolerant, the questions and their answers are objective. The research papers can be read and considered. To monopole deniers, this is pure speculation and not objective.

The other problem with "objective" is that essentially collides with the issue, previously discussed on meta, about whether questions about non-realistic topics like magic, time travel or other hypothetical entities should be part of WB SE, and that only scientifically or technologically oriented questions are allowed here. It was decided they should be allowed.

While, in principle, it sounds fine to have an improved way of guiding people to make better decisions it is necessary to ensure that the people who apply that rule can make better decisions. That requires them to have sufficient understanding of the rules, the context(s) within which they are applied, a body of precedents about the ways such rules have been applied previously, and the knowledge and experience about making those decisions.

The answer to those things is obvious. The existing criteria for vote to close decisions are observed in the breach than in being followed. Adding a rule about objectivity will only another set of criteria to be ignored. Any expectation that it would lead to improvement is a chimera. To make objectivity work requires wisdom and compassion, and is there enough of that to do so?

One consequence of SE model is that it naturally excludes for consideration here many questions that should be dealt with as worldbuilding. There are times when I have considered there should be a "too good to close" button for questions that might be closed because they meet vote to close criteria, but are so worthwhile in terms of worldbuilding that they should be allowed to be asked and definitely should be answered, and so would be left open. After all, it is sort of thing that would improve WB SE and help build it up as a database of good questions and good answers. This, of course, will never happen.

Answer 3

Worldbuilding SE is very subjective by nature. I have noticed that what makes a good (that is, not closed, highly upvoted) answer often has to do with how exactly the answerer interprets the rules of the proposed world, and whether the questioneer/voters/mods agree with that interpretation of the rules of the proposed world (which could be the real world or not).

I think with the question you linked to, at the end of the day it was closed because it didn't feel like a specific question. One of the things put forth in the comments on that question was that there's only one data point--but I would submit to you that the number of data points hasn't been an impedence to many other questions getting happily answered and highly upvoted.

Any other criteria we could try to come up with would probably have the same problem: it would probably shut down a lot of good questions that we see on this site.

I think that the beauty of the worldbuilding SE is that it does what none of the other SE sites do: it lets you ask theoretical questions that can't be objectively answered, when you just want to discuss what sounds good from a story-writing perspective and you want to be as realistic as you can be, to sound feasible. We get questions that are in fields that are all over the map, in all of the different fields that other SE sites exist for (biology, programming, etc)--but the reason that worldbuilding exists is that those sites are based on only strict, provably reality. They're intended for professionals, not for fun.

I'm making broad generalizations here but I think the point is made. Worldbuilding SE's weakness is really its strength.