Actually you are in the right place because while this happens in all of SE, Worldbuilding is only one where it is an actual issue.
The assumption behind "move it to the chat" is that the relevant information to the topic is in the question and the answer and comments either lack direct relevance to the topic or result in a change to the question or answer after discussion.
This means that keeping the comments generally has low value and if there is a need for longer discussion there is no loss in hiding it in the chat. In fact there is a benefit as less comments increases the legibility of the site.
The problem is that there is a hidden assumption in this model that is true for most sites but almost always not true for Worldbuilding.
That assumption is that the relevance of the comment to what is commented can be evaluated non-ambiguously. That a comment either is directly and imminently relevant or has no permanent general relevance at all.
Unfortunately this in turn relies on the hidden assumption that the context in which questions, answers and comments are evaluated is defined in some manner that all agree on.
On most sites this is no issue. The questions are defined on the context of the real world we live in or the fictional world they are related to or the works or beliefs they relate to. It varies but there is some clearly defined real world entity that provides the context you interpret everything in. Step outside it and people will instantly snap at you to get back in line.
Here in Worldbuilding this is simply not true despite the best attempts of various people to apply the general SE model as if it might be. Apart from questions with few specific tags, all questions and answers, and comments, are evaluated in the context of a world that is still Work In Progress and not fully defined by definition.
This means that for most questions in Worldbuilding there is simply no way to evaluate relevance of comment non-ambiguously. Comment not being relevant to how I see my answer and me choosing not to edit my answer, tells absolutely nothing about how relevant that comment is to some other person reading my answer and associated comments. They might and almost certainly will understand the question and the answer in some other context than I do.
So in summary, the reason for the comments policy is because for most sites it does work. Killing or hiding the discussion loses nothing because reading the question and answer is all that is needed. It just happens to be nonsense for the particular site we are on because the hidden assumptions it is based on are not true here.
Enforcement of various rules about allowed questions does hide the issue somewhat but it is fundamental mismatch between the SE model and what we do here and cannot be really solved other than just conceding that we really should have different set of rules and then somehow agreeing on those rules and getting SE to accept them.
Not holding my breath and neither should you.
That said, comments in Worldbuilding really should not be considered ephemeral since they often contain lots of information that is relevant to the topic without being relevant to the question or the answer. Something that on other SE sites is pretty much impossible and thus not supported by the rules and guidelines. Or the comment system. We really should have longer comments on answers than other sites.
