How Select All works

Note that, basically everything about the select all environment is identical to the multiple choice environment, except that multiple answers can be selected by the student to account for needing multiple correct answers being selected simultaneously.

How to generate Select All answers

The select all answer set is generated by an environment, cleverly named ‘selectAll’ along with the choice command to enumerate choices. Indeed, the select all environment is secretly a kind of retooled itemize/enumerate environment and the choice command is secretly a retooled item command. So the following:

            \begin{selectAll}
                \choice{First Choice is Wrong.}
                \choice[correct]{Second Choice is Right.}
                \choice{Third Choice is Wrong.}
                \choice[correct]{Fourth Choice is Right!}
                \choice{Fifth Choice is Wrong.}
            \end{selectAll}
        

Results in:

Notice that the correct answer has the optional argument [correct], more on that below.

How select all validates an answer

Simply. Select all allows the student to click on as many answer choices as they want, and then hit the “check work” button to check their answer. If they have selected exactly every answer choice that is marked as correct (has the “[correct]” optional argument), then they get credit for the problem, otherwise it is marked wrong.

Notice this differs slightly from the multiple choice environment, in that it requires the student to select every “correct” answer and none of the incorrect answers before any credit is given, and doesn’t grey out answers (since it is not clear from an unsuccessful attempt which selected answers were right or wrong).

Optional Arguments

Select All itself (currently) has no optional arguments. The choice command (currently) has only one key-value pair, which is the flag for whether the choice is correct or not. Technically the key is ‘correct’ and the value is ‘true’ or ‘false’, but the key defaults to ‘false’ and simply using the key flips it to true. In other words using \choice[correct]{Correct!} is the same as using \choice[correct=true]{Correct!}, and both of those will mark the given choice as the correct answer. Likewise \choice{False!} and \choice[correct=false]{False!} are equivalent, and both result in the answer being incorrect.

(Additional) Examples

Best Practices

selectAll over multipleChoice

It is generally a good idea to use selectAll over multipleChoice, as multipleChoice is easy for students to spam answer options until they find the correct one, but selectAll has exponentially more possible combinations and quickly becomes unwieldy to try and spam answer configurations for credit. This helps push students to actually consider and answer the question, rather than just trying every combination until they get the page credit and move on.

selectAll is good for equivalence

A classic multiple choice setup asks something like “which of the following are ...”, eg “which of the following answers is equal/equivalent/similar to”. For classic multiple choice questions, this allows students to do a quick assessment and try the most likely answer first, until they find the one answer that is indeed equal/equivalent/similar/etc, at which point they can select that answer and move on. In many cases, this means the student never even reads half of the provided answers, as they hit upon a correct answer early on.

selectAll style problems allow for any, or indeed all, of the presented answers to qualify as correct choices. This forces the student to carefully go through each of the potential answers, since finding a correct one is no assurance that it is the only correct one. As mentioned in the previous comment, this also makes it much more difficult to spam answer combinations for the correct configuration and move on without actually checking any of the potential answers.

4 choices seem to be a good minimum to avoid configuration spamming.

From personal testing, 4 choices seems to be a good minimum number of choices, with the ideal being 5 or 6 choices. At 4 choices, there are $2^4 = 16$ combinations, which is enough for most students to lose track and miss at least one combination when spamming configurations. 5 choices result in 32 configurations, which makes it very likely that the correct configuration is missed by students that are spamming configurations, and seems to be the ideal number of choices. More than 6 choices occasionally seems to overwhelm students and cause secondary issues around math/test-style anxiety. To be clear, this personal and anecdotal observation - although one with surprising consistency within my experience. This might be a good area for a research project or at least some data analysis.

Potential Pitfalls and Problems

Every correct answer must be selected

Since there is no clear feedback as to which answer is correct or incorrect when students attempt a selection and check it, it can be frustrating for students that think they have “tried every possible combination” and still haven’t gotten their answer accepted. That being said, this is an inherent issue with this type of answer in general, and not really a technological barrier/issue, it’s just something to keep in mind.

There are a finite number of options/combinations

Since Ximera allows unlimited attempts at answering (this is a feature we intend to make controllable in some way in the future, but currently all answers get unlimited attempts), a clever-enough-to-be-dangerous student will realize they can just spam the answer possibilities/combinations until they find the right one and move on without even reading the options. With select all, this obviously involves significantly more effort than with just a multiple choice, but I have definitely had students tell me they have done this on a bunch of these problems - so... chalk another one up to “if you had just spent that effort figuring out the answer instead of ’cheating the system’ it would have been less work.” There isn’t much to do about this, but it is important to keep in mind when designing problems and deciding on answer-type variety.

Randomization is ’kind of’ pointless

Although there are ways to randomize the content of the problems and answers (see the randomization section), it is important to note that the LaTeX level content cannot be randomized currently (this is going to be changed in the future edition) which means, although the content of the choice command might be randomized, which select all option(s) is/are correct cannot be randomized - something that the students will pick up on mighty quick. For review this isn’t as bad since students are at least aware enough that they are trying to understand how the correct answer is the correct answer - so knowing “the right answer are the first and third choices” is less important... but it’s still important.

It is worth noting that there is a workaround for this that we have developed, which is addressed in the randomization section, so if you want to make a randomized select all problem, you should followup there.

2024-06-24 13:15:25