Should close/hold-votes be given out arbitrarily on Worldbuilding?

Question

This is prompted by How much explosive power does a missile need to destroy a 100km³ meteorite heading for earth? [on hold]

The question is short, yes, but it is very to-the-point and asks for a straightforward answer. It was put on hold, and the comments are begging for more information. "Please tell us exactly what the meteor is made of," for example.

I am not going to claim that question is high-quality, as it is not. It might even be low quality (Debatable, and my opinion is that it's more "simple" than it is "low-quality"). However, if the question were much more specific, it would be so narrow that it would beg for low-quality one-liner answers that just show the calculation and answer. The way it is, it is specific enough that it is answerable, but it is open enough to invite good quality answers discussing the aspects involved (material, solid vs loose chunks, time until impact, etc., all of which affect the answer).

My question is not "Why was that question put on hold?" Rather, I am making a comparison between that question and other common questions on Worldbuilding. The sample question above was put on hold because of its lack of details, yet there are very, very, very many questions asked all the time which are much more lacking in details than the sample question; in fact, this might be true of the majority of Worldbuilding questions. This is often true even for long questions which provide lots of details (often extraneous details, but they set the tone for the question so that's ok); they are still often wide open.

The "how much explosives to take out a meteor" question, after its edit (which came before the hold), is fully answerable. Details such as "What material is it made of?" are important enough that they can change the answer significantly, but only quantitatively; the answer is still a calculation, and a good answer would provide a range based on best-case/worst-case anyway and so are not needed in the question.

Right now, on the "front page" (or whatever you call it), there is: A government of the near future is trying to send a self-governing crew to populate an earth-like planet 100 years away and How big could an alien-made object on the far side of the moon become so that discovery still can only happen by accident? and If Earth's gravity suddenly became stronger, how would we cope? and... well, I was originally going down the page to include all the current questions which are left open to more interpretation and opinion, need more extra information than the initial sample question, and are generally more open-ended than the initial sample question; but at this rate it looks like it really is most of Worldbuilding questions, at least by the sampling pool of the current questions.

This is actually quite common around here, that questions are targeted for action where the rationale actually explains a large portion of Worldbuilding, and the targeted questions are not infrequently less broad than the others around it.

My point is just that the hold/close reasons such as "too broad" seem to be used somewhat arbitrarily around here, and if the same criteria that holds/closes any question around here were applied equally to all other questions, then 90%+ of Worldbuilding would be closed.

I realize deciding when to flag and going through the queues for Worldbuilding must be difficult because of its very nature, so I assume these holds/closes are in good faith, but they seem arbitrary nonetheless.

My opinion is that there are two ways to deal with this: relax the close trigger-fingers, or do just the opposite and be more strict but along with a more specific set of guidelines too which are applied globally.

Does anyone else notice this issue?

I think it was especially abrasive in this specific circumstance since it was done to a new 1-rep user, after the user edited the question once (shows that they are willing to work with us), and the question got a negative vote. We will get more quality members using honey than vinegar.

Answer 1

Should close/hold votes be given out arbitrarily? No. Definitely and emphatically no.

Beyond that, however, things start to get a little murky. We don't have a close reason specifically for "insufficient information." A question can be perfectly answerable given the information, but a good answer may not appear or be distinguishable with what has already been provided. Should this type of question be put on hold?

To me, it's a case-by-case basis. If I can Google three words from the question and find half a dozen different answers, then the question is too broad: there's simply too many ways the question could be well-answered. (Side note: if Googling three words provides the perfect answer, then the question is downvote-worthy for lack of research.)

Remember, though, that review activities are conducted by people. When I first gained the close-vote privilege, I was thrilled. I wanted to use it. I'm sure everyone with the privilege had a brief period where they wanted to use their newfound powers. Now I carefully review everything that makes it into the queues, adding comments where warranted. It's a major responsibility and I take it seriously. I wish everyone did, but I doubt it's the case.

We've had a number of discussions about closing questions during the site's lifetime. For example, we've laid out what it means for a question to be too broad, whether we're using the close vote too much, and how to improve too-broad questions.

The problem is that many users don't track or even know the Meta site exists, so all these references go to waste. They only see the commentary from people voting to close and are left to figure out when to apply close votes on their own. I'm not saying it's only the newly-privileged users doing this, but comments are the only way to know why a person is voting to close, and they don't always do that. (Side note: I've linked that enough times to get a badge, so thanks community, I guess?)

Is there anything we can do about it? We've tried in the past and I'm sure we will again in the future. I don't see the problem going away anytime soon.

Answer 2

The question you linked was actually one-liner. It did not define what kind of asteroid it is supposed to be, it didn't define what does it mean to "destroy" - in context of asteroids it is far, far from obvious! - and it did not provide any kind of context. This means that there could be many, many equally valid answers, or answer covering everything possible wouldn't fit into answer box, the very definition of "too broad".

From the linked "examples" two are already on hold, one is from 2014, so it was reviewed by totally different set of users, and one (How big could an alien-made object on the far side of the moon become so that discovery still can only happen by accident?) very well defines what it's about. There is no doubt what does it meant to discover something. "by rockets circling the moon or similar" is also pretty precise definition of what OP thinks as regular survey instead of accident. So it provided pretty solid framework.

That said, I believe votes here to be reasonably consistent, at least when looking at the examples you provided. Not perfect, sure, but people who voted did it in their own time, for free. And earlier spent substantial unpaid time providing huge amounts of well-received content. They are on this site longer than you, or are more active here, so please assume good faith and appreciate the amount of work they did for free.

Answer 3

This answer is inspired by @Molot's comment "But I admit, there is a bias against people that appear lazy, write one-liners, don't show any research efforts, ignore requests for clarification... Or is it bias, really?" I would have told him to make it an answer, but he already has an answer going in a different direction.

I had not considered this aspect before, but it is likely important and should be taken into account.

These kind of questions which appear lazy attract the extra scrutiny that increases their likelihood to be closed. That is likely the problem at least in the specifically cited example.

In fact, now that I think about it, I can see that as a trend also. When I have posted meta questions in the past asking why such-and-such an issue is being blown out of proportion, I think it is commonly (I'll guess half the time) accompanied by statements such as "It is possibly arguable that this question (or answer) might have this other problem, and if you made that argument I could understand it more, but the given reason seems overstated."

Given this view on the situation, I am willing to accept that half of the issues I have of this nature ("I don't agree with action X. Why was it applied here?") are because the other aspects of the post cause it to receive greater scrutiny. Whether that is good or bad, justified or not, is another story (probably is good sometimes, not others). But this does make the action understandable.

To apply this view to the issue at hand, other questions are likely to stand because it is not readily apparent that there may be a problem (whether a real problem or just relative to some other closed/held post). It is easy to read over a question which is longer, more fun to read, and which makes you excited to answer without thinking of the flaws; but a short, simple question, which could appear lazier to some people, which is not as fun to read may be much easier to trigger the spidey sense and make you analyze it for critique in that way. In this case, the close or hold may not be arbitrary so much as it was an easier target.

I like analogies, so I'm going to give one: A burglar who blends in well with the crowd might not be likely caught, but if I spray-paint my screen-name onto my own house (or better yet, start smashing my own car with a club) the cops are likely going to at least stop and question me since it looks like I'm engaging in illegal activity. The latter case is not as bad, and depending on where/how it might not even be illegal at all, but it is more likely to invite a run-in with law enforcement.

Please do not assume that, because of my analogy, I'm implying the example case (blow up a meteor) is great and the others are bad and need attention. That is not the case, and my analogy cases were chosen merely to make the argument of "scrutiny because of visibility" more clearly.