26

Once, in a college class of mine, my teacher defied each of us to elaborate a good question for an IQ test. I proposed the following one.

Is the option C a correct answer for this question? Please select one of the following options.

A - YES
B - YES
C - NO
D - NO

The correct answer is quite simple (I think), but it triggered a huge discussion on the classroom about its validity. Some of my colleagues were saying the question was not a good sentence because it has references to itself. Meanwhile, others (and I) argued that it is, in fact, a good "meta" question.

I really would like to know some opinions about it. And, of course, If you want, you may answer it.

Rewan Demontay
  • 8,263
  • 3
  • 15
  • 56
Pspl
  • 2,263
  • 5
  • 20
  • 1
    I’m not sure [tag:meta-knowledge] is the right tag: that’s more about multiple perfect logicians having to come to a conclusion based on each others’ perfect behaviour. Maybe you want [tag:puzzle-creation] - an IQ test isn’t necessarily a puzzle but this kind of question could be considered one? – boboquack Jul 08 '20 at 13:06
  • 5
    IMHO, this is a good logic puzzle. But if I'm asked whether this is a good IQ test, I'm not sure. Most IQ tests deal with some abstractions, visuals, or patterns. So perhaps logic questions are a bit out of topic for IQ test. But who knows. – athin Jul 08 '20 at 22:58
  • 5
    I find the question whether this is a good IQ question opinion-based and in general off-topic for puzzleSE. – infinitezero Jul 09 '20 at 02:52
  • 11
    I have no comment on whether it is good for IQ because IQ has criteria beyond the scope of this board. Is it a good riddle? Sure, I got the correct answer but i have a beef with the question. It should be “is C the correct answer...” – Damila Jul 09 '20 at 04:15
  • I would argue that the concept of the question is fine, but it relies on interpretation that is easily-contested, and thus the specific wording is troublesome. Rewording it might assist with that. You can test the exact same thing without explicit self-reference. – Glen O Jul 09 '20 at 05:53
  • 3
    If any company will let me do this in a test, I will stand up, go home and refuse any job offer. – Thomas Weller Jul 09 '20 at 07:33
  • What is the question? And why are A and B the sane and C and D the same. – Srinivasa Pulugurtha Jul 09 '20 at 13:17
  • 2
    @SrinivasaPulugurtha The question is "Is the option C a correct answer for this question?" – Kevin Jul 09 '20 at 14:32
  • 9
    How is this even remotely on-topic? One of the fundamental rules across all of Stackexchange is that this is a Q&A site, not a discussion forum. – BlueRaja - Danny Pflughoeft Jul 09 '20 at 18:41
  • 5
    The phrasing of the question could be improved. Answer (D) seems to be the intended correct answer - but the problem is that (D) reads "No", which is the same answer as part (C). Which means that by picking (D) you are LITERALLY picking the same "answer" as had you picked (C). I would change it to say "Is the option C the correct OPTION for this question?" - the difference between option and answer is essential. – Dast Jul 10 '20 at 09:40
  • 2
    Agreed with @Dast. As it is worded, there is no 'correct' (logically consistent) answer. – J... Jul 10 '20 at 11:05
  • 2
    @Dast sure, but C's "no" is incorrect – user253751 Jul 10 '20 at 11:32
  • 1
    @user253751 The question is malformed. Why can't the answer be as well? "Yes, the correct answer to the question is C > NO" – J... Jul 10 '20 at 12:12
  • 2
    So I thought, the opinion part was the optional part (and the puzzle itself is the real question.) I was wrong. After seeing how this turns out, and additionally looking at the "checkmarked" answer, this is very opinion-based and should be closed as out-of-topic. If we are looking at the discussion too, seems like OP is more keen to accept the answer which "shares the same belief as themselves." The most upvoted ones are actually in the contraries. (But of course this is my personal opinion too, so let's see how the votes turns out.) – athin Jul 10 '20 at 12:23
  • 1
    @J... If the correct answer was C then the correct answer wouldn't be C. Obviously. – user253751 Jul 10 '20 at 12:33
  • @user253751 Yes, that's what malformed means. If the question can be a legitimate question despite being malformed then why can not the answer be as well? – J... Jul 10 '20 at 12:36
  • 2
    @J... You unambiguously cannot choose C as the correct answer. The question is not malformed, it's just a puzzle. – user253751 Jul 10 '20 at 14:17
  • 2
    The question can't be answered. If "D" were the answer, it would mean that C is not a correct answer to the question, which would mean that C is right (because it says "no"), meaning it is a correct answer and making D the wrong answer. It's a paradox, no answer can be marked and is a bad question for any test, IQ or not. It's not even a puzzle, because puzzles have answers. It belongs in the realm of philosophy. This post should be closed. – msb Jul 10 '20 at 19:08
  • The word "answer" is defined as a response. https://www.dictionary.com/browse/answer?s=t The OP question, as stated moves the letter identifier into the domain of the responses, and can be correctly answered as-is. – anregen Jul 11 '20 at 01:32
  • Another thing to consider is that "IQ" can and should cover a hugely broad area of thinking and understanding. What I want this question to target, may not be what the OP intends to target! For example, the specific wording "a correct answer" can draw many of us to consider how many correct answers there may be. 'C' is not a correct answer, so we must not chose A or B or C. D is the correct answer, and requires us to view the question from different angles to come to that conclusion. – anregen Jul 11 '20 at 01:39
  • More technically, I would say this question creates 2-dimensional response space (by specifically including the identifier 'C' in the yes/no question). We must consider both the yes/no value of the response AND the identifier of the response. This means our 4 options are unique: A:yes, B:yes, C:no, D:no. When considered as a 2d answer space, C:no is not the same response as D:no. So we can give the answer "D:no" and satisfy both aspects of the questions simultaneously, without creating a paradox. – anregen Jul 11 '20 at 03:15
  • @user253751 You unambiguously cannot choose anything as the correct answer...because the question is malformed. – J... Jul 11 '20 at 15:38
  • @msb Note that a statement can be true and still not be the correct answer. What colour is the sky, typically, during the day? (A) blue (B) green (C) red (D) bush didn't do 9/11 – user253751 Jul 11 '20 at 18:12
  • @user253751 I'd argue that's fallacious: the answers you gave are not written in full, and that is the only reason they are correct. The full answer for D would be "The typical colour of the sky, during the day, is Bush didn't do 9/11", which is obvious garbled nonsense and not true. – Helen Jul 11 '20 at 21:00
  • @Helen but the question wasn't: "The typical colour of the sky, during the day, is __________." – user253751 Jul 12 '20 at 14:40
  • @user253751 That's very true, I did paraphrase the question to an extent, but even the original phrasing of "The sky is Bush didn't do 9/11, typically, during the day", would be false. – Helen Jul 12 '20 at 15:31
  • @Helen The question also wasn't: "Complete the sentence: The sky is ___________, typically, during the day." – user253751 Jul 12 '20 at 19:46
  • @user253751 Regardless of the full version of the answers (which we seem to disagree on), there's an obvious grammatical difference between the choices you gave: "Bush didn't do 9/11" is a full sentence involving a subject, a verb and a direct object. We can definitively say that "Bush didn't do 9/11" is either true or false. "Blue" as a standalone answer lacks any of those features, and (no surprises) this means that we can't say definitively whether "blue" is true or false, because "blue" isn't a full answer, but rather information that completes the question to form an answer. – Helen Jul 12 '20 at 21:05
  • @Helen Then: "(A) the typical colour of the sky during the day is blue (B) the typical colour of the sky during the day is red (C) the typical colour of the sky during the day is purple (D) bush didn't do 9/11" – user253751 Jul 13 '20 at 10:09
  • @user253751 Well at that point the argument of whether or not that's a well-formed question (which I don't think it is) is irrelevant to the argument of whether or not the original question is malformed. For the question the OP posed, all answers work with the question (unlike your "Bush didn't do 9/11" example). The reason that the original question doesn't work is different to the reason your question doesn't work. – Helen Jul 13 '20 at 11:49
  • @Helen then why did you complain that in my first example the answers were not written in full? – user253751 Jul 13 '20 at 12:02
  • Let us continue this discussion in chat. – Helen Jul 13 '20 at 12:05

15 Answers15

22

I don't think this is a good question for an IQ test. The problem is that there are some strategies to solve this kind of meta-question and who know one of these strategies can answer in seconds. Also, there is some linguistical knowledge involved (for example a non-native speaker could score less than a native one).

Thus you will be testing the knowledge of the participants rather than their IQ.

The same reasoning holds for chess puzzles, grid-deduction, logical-deduction...

melfnt
  • 5,132
  • 2
  • 13
  • 58
  • 25
    IQ is completely arbitrary anyway – Quintec Jul 08 '20 at 17:11
  • 2
    A lot of the questions from my old 'Know Your Own IQ' book are language- or culture- based. For example: "Underline the odd-man-out: Alexander, Napoleon, Wellington, Nelson, Hannibal". Or: "Underline which of these towns is not in the USA. GICOHAC, SHENAT, TONSOB, GOTHNINSAW". And even the ones that aren't require you to parse sentences like, "Underline which of the five numbered figures fits into the empty space." The amount of language understanding required for this one seems fairly low in comparison. So it's a 'fair' question by historic standards, if not by modern standards. – user3153372 Jul 09 '20 at 06:01
  • @Quintec you're right. "standard" questions are usually meant to measure logical skills (which I agree is not IQ). – melfnt Jul 09 '20 at 07:05
  • 3
    @user3153372 that sounds like an illegitimate IQ test – Christopher King Jul 09 '20 at 23:25
  • 3
    @user3153372 ... the joke being that all of the names are actually (once you unscramble them) towns in the USA. Athens Ohio, Athens Georgia, Athens Alabama, Athens Texas ... – Prime Mover Jul 10 '20 at 21:26
  • 3
    @user3153372 There are a lot of things historically that billed themselves as 'IQ' tests but did not actually test anything other than some basic pattern recognition combined with knowledge that was quite often highly prevalent in a specific group of people but almost nonexistent outside of that group. Those aren't 'IQ' tests though, they're elitist propaganda designed to make that specific group of people feel smarter. – Austin Hemmelgarn Jul 11 '20 at 02:09
18

Bad question.

I would imagine a person who is familiar with the idea of paradoxes and self-referentiality would immediately go "aha, I get what this is" and answer D. Another person may have never seen something like this before and may think "what in the world are they even asking? Is C the right answer to the question? But what question? I see no question here?". Such a person may rather assume that the test-maker made a mistake and may decide, especially if the test is timed and with multiple questions, to just skip it.

Another way to see that this is a bad question is that if you know how to solve it, then the solution is obvious (D). You state so yourself. So what exactly are you measuring here? The people that already know, get the right answer immediately. The people that don't know, might or might not figure it out, but either way, it's clear that you are testing whether people know what sort of question it is, rather than their ability to figure something out.

I think a better intelligence question would be to ask why in the world college professors and their students are wasting their time on this sort of useless stuff. It's going to take a real genius to figure that one out.

Jaood
  • 289
  • 1
  • 2
  • 3
    I agree with you on some level. However, doesn't your first point of view is valid for any other kind of question? Just imagine a person who is familiar with the idea of sequences and patterns would immediately go "aha, I get what this is"... – Pspl Jul 09 '20 at 08:20
  • 2
    OP asks if this is a good for an IQ test. IQ is not what you wish it to be. It is not what you think intelligence is or should be. It is a number that can be measured - the factor of variance behind performance in different mental-ability related questions. A better question is one that better correlates with that factor, that's it. The question proposed in your final paragraph might be an "intelligence question", but it wouldn't be measurable and usable in an IQ test and would not be a useful "IQ question". – Džuris Jul 09 '20 at 09:45
  • 3
    Indeed, everything is obvious once you know the answer. I don't see how it's specific to this question, though. – Eric Duminil Jul 09 '20 at 20:33
17

I disagree with most of the other answers - I think it is a better question than most actual IQ test questions.

The point is that if you haven't seen something like this before, then whether you're capable of figuring it out is a really good determiner of the ability to think logically. And it's perfectly plausible to do so; you don't need any outside knowledge, and it's pure logic. By contrast, a lot of IQ-style questions of the form "which grid fits the pattern", etc, really do rely on having seen lots of those questions before to have an idea of what type of "patterns" are considered valid, and while they certainly do require intelligence once you know the unstated rules for designing them, they also rely on your knowledge of those rules (somewhat like cryptic crosswords).

Even if a candidate has seen something like your question before, I think actually going through the process and getting the right answer takes intelligence rather than just knowledge. It's really not just a question of turn the handle and the answer comes out - you have to think about the options carefully in a way specific to this question.

However, it's a bad question from an IQ-test writing point of view for the following reason: you can't write lots more questions in the same style. If someone can do this one, they will be able to do all the rest. (The only benefit to having more than one question like this is to detect guessing.)

But actually that drawback is what I like about it - it measures what it measures very well. A lot of questions on IQ tests are about trying to spot what the question writer was thinking, and consequently which candidates will succeed varies a lot from question to question. So these questions do not even measure what they are trying to measure particularly well, which is why you need a lot of them. So if you were getting paid for writing IQ test questions, you might want to stick to the traditional style!

Especially Lime
  • 2,943
  • 11
  • 17
  • 1
    Thanks! I share the same feeling about this. – Pspl Jul 09 '20 at 08:23
  • 9
    The fact this answer was accepted, especially with that comment, strongly implies that the asker wasn't actually asking a question, but looking for someone who agreed with him. Now, that's okay, but not really what a site like this is for... – Jasper Jul 10 '20 at 12:48
  • 4
    You selected this as "correct" answer when you are not capable nor qualified to make that judgment. – IvanP Jul 10 '20 at 20:08
  • 2
    @IvanP You should see this site called StackOverflow. – Kaz Jul 10 '20 at 21:32
15

Is this a good question?

No it isn't. The reason is that due to the broken way self-reference is worked into it, it has no answer.

Here is why.

Firstly, the answer to the question is a value from the domain { "Yes", "No" }, not one from the domain { A, B, C, D }. Those letters are just labels for the answer. That's how multiple choice testing works. The label is not the answer, the labeled answer is the answer.

Because both C and D are labels for "No", those two choices are equivalent: they constitute the same answer. It cannot be the case that C is a correct answer, but D isn't.

Because C is a label for "No", the question "is option C a correct answer?", means exactly the same thing as "is 'No' a correct answer?"

Thus, let us entertain the possibility that "No" is the correct answer, i.e. C is not correct. If "No" is the correct answer, that is equally represented either by C or D; it means that C is a valid choice, just like D. But that cannot be because "No" means "C is not a valid choice".

Thus, "Yes" must be the correct answer. But then, only the choices A and B are tied to that answer. That "Yes" answer, however, means that C is correct (and also D), and not A or B.

So the situation is not well-formed; though it may not be the intent, the question reduces to the Liar Paradox.

It may have been the intent that D must be the answer; but that can only be the case if we consider C and D to be distinct from each other, even though they both map to the same "No" answer, which is not a valid concept.

It cannot be that the "D flavored 'No'" is correct but the "C flavored 'No'" isn't; 'No' is just 'No'. The specific alphabetic labels do not contribute anything to the semantics of the "Yes" or "No".

The multiple choice test is just a format for easier grading: the correct answer is given away along with several distractors, and for the sake of simplicity, these items are all given symbolic labels, which otherwise don't mean anything.

A well-formed multiple choice question has a correct answer even if we take away the choices and ask for the answer to be stated.

In this regard, our question runs aground already: it contains a reference to one of the labels, which makes it impossible to remove the choices such that the question still makes sense.

Kaz
  • 436
  • 2
  • 7
  • 2
    Exactly right. There is no correct answer. And D is not the correct answer. – gnasher729 Jul 10 '20 at 09:04
  • 1
    Could the question be improved if it asked "Is C the correct answer?" and the choices were "A-YES B-Yes C-NO D-No, where the correct answer was "No", in mixed-case italics. – supercat Jul 10 '20 at 16:13
  • 1
    @supercat, If the question is "Is C the correct answer?", then it's still a yes-or-no-question. Since "D-No" is neither "yes", nor "no", it's not a plausible answer. – Kaz Jul 10 '20 at 16:17
  • Great answer, upvoted. As to the "IQ question" in the OP, I'd have answered "E". – bobflux Jul 10 '20 at 23:57
  • This argument requires a draconian distinction between "answer" and "response". In fact, the word "answer" is defined as a response. https://www.dictionary.com/browse/answer?s=t So, using natural language conventions, as well as the precise definition, the domain of available answers (responses) is {A: yes, B: yes, C: no, D: no}. The question statement moves the identifiers into the answer domain. By keeping the identifiers attached, each response is unique, and only D is acceptable. Even using the rules you propose, D is still the best answer, which is a phase commonly used in tests – anregen Jul 11 '20 at 01:27
  • 2
    @anregen My answer doesn't revolve around any distinction between "answer" or "response" at all; I didn't even mention the latter word. Let them be synonyms; it's clear that the question is a yes-or-no question: if it has a correct response/answer, it is either "yes" or "no". D is just the name of the response/answer, which is also known by the name C. "Is C a correct answer" means the same thing as "Is C the name of an answer which is correct?" not literally that the letter "C" is the answer. Just like "I will have combo C" doesn't mean you will be eating the third letter of the alphabet. – Kaz Jul 11 '20 at 01:41
  • @kaz The question refers to the response C specifically. In many questions, you can re-assign identifiers to different values without changing the nature of the question. 2+2 = __. {A: 1, B: 4} (same as {A: 4, B: 1}). But this questions requires consideration of the entire response, and specifically moves the identifier into the domain of the responses. {A: Yes, B: Yes, C: No, D: No}. You can see this is true because we can no longer re-arrange the values associated with the identifiers without changing the meaning of the option. consider the trivial case {A: Yes, B: No, C: Yes, D: No} – anregen Jul 11 '20 at 01:50
  • @anregen The question could be fixable, perhaps, if posed like this: "Is 'No' the correct to answer this question by way of option C?". If you give a "No" answer, you're admitting that a No answer by option C is not correct. So then if you proceed to give that answer using option C, you're effectively declaring your own answer/response incorrect. Whether that makes it incorrect, or makes D correct, is another question. That is to say, whether or not the test subject declares their answer correct isn't necessarily the determiner of that. – Kaz Jul 11 '20 at 01:51
  • @anregen We can easily rename the identifiers in the question, if we do it everywhere: in the question and the answers. A B C D can become I J K L, or D C B A or whatever we want. If B and D are now "No", then we must change the question to refer to either B or D instead of C. Substituting a symbol doesn't change the "nature of the question". – Kaz Jul 11 '20 at 01:57
  • That's the very reason I find this question interesting: you must simultaneously consider that the identifier in the response could cause a paradox, as well as the value of the response addressing the yes/no question that was posed. The fixing you suggest changes the nature of the question and I believe is exploring a part of the mind a bit different than the OP question. – anregen Jul 11 '20 at 01:58
  • Note that when you answer this question, you answer it with a letter A, B, C or D. You don't answer "yes" or "no". Note that "this question" encompasses the entire question and list of options, not just the part ending with a question mark. – user253751 Jul 11 '20 at 13:54
14

I assume the answer is meant to be

D-NO

Reasoning:

It's the only option that's completely consistent with the semantic conditions. A and B are out because they imply YES, which means choosing either would contradict C being the correct choice and choosing C itself would mean C wasn't the correct answer, another contradiction. That leaves only D.

It's not a bad question, but it's best to avoid self-referential questions in my view.

Deepak
  • 439
  • 2
  • 8
  • So you are on the group of the OTHERS... LOL! Thank you for your feedback. And yes, your answer is the correct one! – Pspl Jul 08 '20 at 12:17
  • 3
    I disagree with your logic for A and B. The question asks if C is "a correct answer" not if it is "the correct answer". It would not be contradictory for example if A, B, and C were somehow all correct answers (e.g. if they all contained "yes"). Your conclusion about A and B are correct, but the reasoning is not. The reason they are wrong is because those answers are themselves incorrect. – JBentley Jul 09 '20 at 08:43
  • 1
    @JBentley: But then : "Please select one of the following options". To me, it meant that since there's no reason to prefer A over B, the answer had to be either C or D. It cannot be C, so... – Eric Duminil Jul 09 '20 at 20:35
  • 5
    @EricDuminil That you can only select one doesn’t mean only one is correct. – 11684 Jul 09 '20 at 23:23
  • 1
    @JBentley Well, C can't be a correct answer for obvious reasons. Note that it does not ask "is NO a correct answer to this question?" – user253751 Jul 10 '20 at 11:33
  • @user253751 Yes, I understand that. What is your point? Mine was that if A or B were correct, this doesn't mean that it logically has to "contradict C being the correct choice" as this answer claims. A and B are wrong because the answers they contain - "yes", are incorrect, because in this case C is wrong. – JBentley Jul 11 '20 at 12:45
  • 1
    @JBentley A can't be the correct answer (obviously). B can't be the correct answer (obviously). C can't be the correct answer. D can be the correct answer. Therefore the correct answer is D. – user253751 Jul 11 '20 at 13:52
  • @user253751 I think we are at tangents to one another. I don't disagree with anything you have just said. What I am saying is that the reasoning for ruling out A and B given in this answer is the wrong reasoning, whilst the conclusion that A and B should be ruled out is the right conclusion. I suggest you re-read my original comment. – JBentley Jul 11 '20 at 16:37
  • @user253751 Basically, this answer's line of reasoning is as follows: "A is saying 'YES, C is a correct answer'. Therefore A cannot be correct, because it is a contradiction for A to be correct and for C to be correct at the same time." But that incorporates a logical error, which is that A and C can't both be correct. Instead the correct reasoning is: "A is saying 'YES, C a correct answer'. Therefore A cannot be correct, because C is not a correct answer". – JBentley Jul 11 '20 at 16:44
  • @JBentley Questions in this format have only one correct answer. – user253751 Jul 11 '20 at 18:13
  • @user253751 Logic imposes no such constraint, nor does it provide any definition for "questions in this format". Answers merely need to be logically consistent, and "is C a correct answer?" (emphasis added) means that multiple correct answers are logically consistent with the question. If the question intended such a constraint then it should have asked "is C the correct answer?". But I already covered all that in my first comment. – JBentley Jul 11 '20 at 18:38
  • @JBentley in any case, C can't be a correct answer because that would lead to a logical contradiction. – user253751 Jul 12 '20 at 14:42
  • @user253751 Yes, once again, and at the risk of sounding highly repetitive, I don't disagree with that. Please re-read what I wrote and and try to understand the point I'm making. My point is not in any way related to any of your responses so far, with the possible exception of your claim that there can only be one correct answer (which as you see above I have disputed). If you wish to debate my point that's fine but I kindly ask you to give a counter-argument to mine rather than just random but valid unrelated points. I suggest also that we take it to chat. – JBentley Jul 12 '20 at 15:59
7

The aim of an IQ test is to measure (some form of) intelligence. Therefore a good question is any question that requires this intelligence. If you consider that being comfortable with this kind of self-referential questions counts as "intelligence", then it is a good question. I personally think id does.

On the other hand, there might be an expectation of "fairness" in the sense that the questions should be clear, only solving it should be difficult. There is another example in a closed question on this site where the correct answer can only be found by understanding there ia a typo in the question. That requires intelligence and "out of tne box" thinking to solve. But that would not be a "fair" question if made intentionally, in my opinion.

Florian F
  • 29,623
  • 4
  • 63
  • 138
5

Yes - it is a good question, but is is not as good as this (very old) question http://faculty.uml.edu/jpropp/srat-Q.txt

Ian Davidson
  • 67
  • 1
  • 6
    This is a neat link, but generally the community likes to have answers that are more than just a link. Can you expand your answer to include some relevant details from the link? No need to reproduce the whole thing. – Jeremy Dover Jul 09 '20 at 21:03
  • 1
    Very interesting link. Thanks for sharing. – Pspl Jul 09 '20 at 22:46
  • Came here to post this. The SRAT is my favorite puzzle ever. – par Jul 10 '20 at 04:20
4

I also think it's a bad question.

The purpose of any IQ question is to distinguish intelligent people from the other ones. So obviously: the more intelligent the person is, the clearer they should see that one of the answers is correct and the others are not. In particular: every sufficiently intelligent person must be convinced that the question is well posed.

The puzzle at hand does not meet this criterium. It is well known by logicians that self-reference (or generally: non-wellfoundedness of reference) leads to nonsense. Even if such a question appears to be valid in the sense that there is exactly one answer from which it is impossible to derive a logical contradiction, it is still insufficient for the question to make sense, as illustrated by the following example:

Which of the following is true?

(a) Both sentences are false.
(b) The Earth has the shape of a banana.

Therefore if every person in the world was to answer the said question, the results would include:

- a dummy: "I don't know" - FAIL
- a smarty: "D" - PASS
- a professional logician: "The question is not well-posed" - FAIL

Ergo: the dividing line that question draws between people is not "the intelligent" vs "the unintelligent", but rather "the intelligent but logically uneducated" and "the rest", which is not what an IQ question should do.

Adayah
  • 149
  • 3
  • 2
    @user253751 Self-reference doesn't have to lead to a contradiction, but even if it doesn't, it's still nonsense. I think that's what Adayah is getting at, though the example is flawed. "This sentence is true" is "self-referentially-stable" in the sense that if we regard it as true, there is no contradiction, and if we regard it as false, there is also no contradiction. But ... that's a bit of a problem, isn't it. – Kaz Jul 11 '20 at 00:19
  • 1
    @Kaz How is the example flawed? – Adayah Jul 11 '20 at 11:46
  • @user253751 To clarify: by a self-referential statement I mean one for which determining the truthfullness eventually requires knowing it in advance. I claim any such statement is nonsensical. If you disagree (after the clarification), I would like to see a counterexample. – Adayah Jul 11 '20 at 11:58
  • 2
    @Kaz "This sentence is the correct answer" can be a correct answer for a similar question. "This sentence is not the correct answer." can never be the correct answer, because that leads to a logical contradiction. The fact that it's not the correct answer, however, doesn't imply that it's false. It can be true, and also not be the correct answer. (Example: "What colour is the sky?" "A: It's red" "B: It's green" "C: It's blue" "D: Hydrogen has an atomic number of 1") – user253751 Jul 11 '20 at 12:32
3

I think this is a bad multiple choice question. Not just a bad (MC) question for IQ tests, but in general a bad (MC) question.

Let's start by defining three categories of multiple choice questions:

  1. Good questions: exactly one answer* is right, and getting the answer right conveys a high likelihood the answerer understands the matter at hand well enough
  2. Passable questions: exactly one answer* is right
  3. Bad questions: a good case can be made for multiple answers being right

This is definitely not a good question:

If I consider the fact that A and B are in no way distinguishable and you can only pick one answer, neither can be the right answer. That leaves C and D. C is clearly a contradiction, so the answer must be D. Using the fact that there is only one correct question allowed me to get to the right answer without fully understanding why it is the right answer, so this is not a good question.

This is not a passable question either:

There is definitely a case to be made that there is no good answer here. Other answers have gone into more details about this, but the brief version is that C and D are both "no", so if one is right, the other is too. That means that D is not the right answer, and neither are A, B or C. At this point, answering the question is more about trying to figure out what the asker meant to be the answer than what is actually the right answer, and that's a pretty big sign of a bad question. Surely, trying to figure out what the asker meant is always a part of (multiple choice) questions, but it really shouldn't be more important than answering the question itself.

(The "please select one answer" is a strong hint towards what the asker what answer the asker wanted to hear here, and even the implicit version of it where you expect one answer per multiple choice question unless stated otherwise is enough of a hint that many people would be able to guess it. But how do you select one answer if none of them are correct? On the other hand, if you are trying to test people's ability to make the correct assumptions when there is ambiguity then it's not a bad question.)

Moreover, once you acknowledge that filling out none of the answer is as - if not more - correct as D, you can no longer distinguish between those who skipped the question and those who answered it correctly. And if you consider "nothing" to be a good answer here, you are valuing those who skipped the question (or didn't get to it) over those who got it wrong and that feels rather off to me. So, you cannot even fix the question by allowing "both" answers. So, the best thing would be to disregard the question all together, which makes it worse than just a "bad question" in my opinion.

*: in the case you are doing "select any number of answers", consider the one answer to be a single correct set of letters.

Jasper
  • 133
  • 6
  • 1
    You could fix the indistinguishability thing by making the answers YES/NO/NO. Bonus: Then, if you just assume it can't be B or C because it can't be both, you're wrong. – user253751 Jul 10 '20 at 11:37
  • @user253751 Good point. I didn't add it because it wouldn't solve the bigger problem, though. – Jasper Jul 10 '20 at 12:39
1

This is not a good question, because experience with similar problems makes it easily solvable or very difficult. If IQ is a measure of raw mental power, regardless of experience, then a good question should not be affected by experience on similar questions.

olemundo
  • 11
  • 1
1

I’ll just answer the question itself: Since it is highly debatable, and actually debated, whether D is the correct answer or not, it is not a good question for an IQ test. Any question asked in an IQ test must have a correct answer that is not debated.

gnasher729
  • 776
  • 3
  • 6
0

This is an interesting question and might be a fun riddle / logic puzzle. I don't think it'd be a good IQ test question, though, because it's solvability (and maybe also its solution, if any) depends heavily on the axioms being assumed.

As no set of axioms is explicitly stated, several axiom choices can be considered appropriate. For example, it wouldn't be too strange an axiom to assume that if two answers have the same content, they have the same truth value (are either both correct, both incorrect or both undecidable), irrespective of their order or numbering. However, your question becomes undecidable under that assumption / axiom.

das-g
  • 101
  • 1
0

Here's why only D is correct, using reductio ad absurdum arguments.

Assume A is a correct answer. Then "the option C a correct answer for this question", but C is not a correct answer, since it contradicts A. So the assumption that A is a correct answer has lead to a contradiction, therefore A is not a correct answer. Same for B.

Assume C is a correct answer. Then it is not the case that "the option C a correct answer for this question", again leading to a contradiction.

So none of A, B, or C is a correct answer.

And since C is not a correct answer, the answer to "Is the option C a correct answer for this question?" is NO, and so D is a correct answer.

David Browne - Microsoft
  • 179
  • 4
  • I know this post is a week old by now, but that sort of logic fails if there is no correct answer, e.g. "What is 4+6? (A) 1, (B) 7, (C) 12, (D) 24". It is a very useful tool if you are guaranteed a correct answer, but in the asker's question I don't think you can assume that. – Helen Jul 18 '20 at 20:29
  • 1
    I don’t believe I assumed that there were any correct answers. Once you establish that none of A,B, C is a correct answer by reductio, then you can evaluate the whether D is a correct answer. And it turns out to be correct. The method would also work to show that no answer is correct in your example, as each of them would lead to a contradiction on the form 1=0. – David Browne - Microsoft Jul 18 '20 at 21:08
  • Ah misread on my part, that's embarrassing. I skimmed the last paragraph too hard and I read it as "Since C is not a correct answer, .... so D is a correct answer". – Helen Jul 18 '20 at 21:31
-1

The question does not specify what qualifies as a "correct" option, and is therefore ambiguous and subject to interpretation.

Although IQ tests seem to often favor such questions, this type of question is not suitable for a multiple choice format if trying to guesstimate intelligence.

A descriptive answer will more likely yield a possible glimpse into intelligence.

That said, I consider every IQ I have ever taken to be very biased. If you go by IQ tests, I'm an extraordinary genius. I see no real-world evidence of that being true.

End Anti-Semitic Hate
  • 101
  • 1
  • 5
  • There’s a problem. An “extraordinary” genius is one whose thought processes cannot be followed by others. In other words, your answers are not the ones expected by the test author. If you go by the IQ test, you are just wrong. – gnasher729 Jul 10 '20 at 08:57
  • @gnasher729 If you are going to publicly tell someone they are wrong, you need to provide references for all your claims. Do you have sources for your claims? They appear to likely be fabricated and not based in science. – End Anti-Semitic Hate Jul 10 '20 at 18:07
-2

Jordan Peterson in this video tells how IQ questions are selected and which questions are "bad". According to him a "good" IQ test question is one that people with higher IQ get correct more often that people with lower IQ.

So it's not a matter of opinion, it can actually be measured. And you can also measure if the question is biased. This might be slightly language biased, but maybe not as the terms are rather simple.

The objections (self-referrencing, having seen similar things before) can be said about many of IQ test questions but the fact is that ability to answer such questions usually correlates with IQ. Even trivia questions are (mostly and on average) answered better by people of higher IQ. So even those are at least fine or decent for measuring IQ.

Džuris
  • 121
  • 2
  • 17
    Your definition is circular. If you use an already established IQ test to determine what subquestion is best correlated with a high IQ score, then you are assuming your original IQ test is accurate, but determining the accuracy of an IQ test is the very problem you started off with. But apart from the circularity, the correlation approach is just dumb anyways. Just because something is most-highly correlated, doesn't mean it is justified as a sole indicator, unless you can show that your parameter captures most of the volatility. – Jaood Jul 08 '20 at 23:02
  • 3
    @Jaood it's not my definition, it's an actual method used in psychometry https://en.wikipedia.org/wiki/Item-total_correlation I am just an enthusiast in this field, but as far as I am aware, one could start with whatever questions and iterating this process will always lead to similar kind of "IQ test questions". – Džuris Jul 08 '20 at 23:17
  • @Jaood in no way I suggested this might be the "sole indicator". The question is wether it would be good for inclusion in a test among other questions. The answer is - if it correlates with IQ well enough, then yes, it would be good to measure it (as one question among many). – Džuris Jul 08 '20 at 23:19
  • 2
    Your own wikipedia article disagrees with you. It states that it filters out low-correlated questions, i.e. questions that seem to measure something completely different than what what the other questions are measuring. It does not state that it is justified to pick one question that is "most-highly correlated" and use that as a sole-indicator. This is what you seemed to argue in favour for, or at least Peterson did in the link you posted. He says that the way to create IQ tests is to repeatedly pick out the SINGLE best indicator. This is not the approach your wiki-link outlines. – Jaood Jul 08 '20 at 23:53
  • 4
    Also note that the item-total correlation approach is NOT circular, since it does not assign any judgement on its conclusions. It does not state that the resulting items are all great measures of whatever they're supposed to be measuring. It only concludes that they are consistent measures of roughly the same thing. So again, even if you apply this approach properly (which Jordan Peterson doesn't), you don't get "good" IQ-tests out of it ... you only get "consistent IQ-tests" out of it. – Jaood Jul 08 '20 at 23:57
  • 7
    Also, please do provide evidence that if one starts with "whatever questions", it always leads to the same IQ test questions. This is a very strong statement, and I'd be surprised if it were true. Where's your evidence? – Jaood Jul 08 '20 at 23:59
  • -1 for the reasons @Jaood has given, particularly the one two above mine, which demonstrates why this is not an adequate answer to the question asked. – JBentley Jul 09 '20 at 08:58
  • 1
    @Jaood "consistent measures of roughly the same thing" -> that same thing is what's called IQ. It is a metric that those tests actually measure. Being in line with them is the only way to actually assess a question. – Džuris Jul 09 '20 at 09:25
  • 1
    @Jaood sorry, "whatever questions" was indeed too strong. I meant - any non-homogenous set of questions having something to do with intelligence or mental abilities. You are right. The actual claim is that these tests are consistent. They measure some common factor and that something is often called IQ - an unfourtunate term that invites the debate we are having. – Džuris Jul 09 '20 at 09:57
  • 1
    @Jaood And all those questions are good measurements of whatever-it-is-they-measure, right? – user253751 Jul 10 '20 at 11:39
  • It's not purely about consistency with our questions; it's also about predictive power with regards to overall performance. The final battery comprises questions with similar power, but greater than that of rejected questions. Modern tests therefore use a tiny subset of what's been tried because they achieved this. From that perspective, the OP's question boils down to whether science would show these questions achieve that. I suspect they've not been tried. – J.G. Jul 10 '20 at 17:52