Feedback

$\newenvironment {prompt}{}{} \newcommand {\ungraded }[0]{}$

Feedback

How to generate/use the feedback environment.

Feedback can be provided to students using the feedback environment within a problem-style environment. The feedback environment takes some optional arguments (see below) to determine when it is displayed, but by default it is displayed after any attempt at the problem (regardless of whether that attempt was correct or incorrect).

For example, consider the following:

Guess my number! $\answer {\pi }$

$\pi$ . The answer is $\pi$ .

The previous problem is generated by the code:

            \begin{problem}
                Guess a number! $\answer{\pi}$
                \begin{feedback}
                    $\pi$. The answer is $\pi$.
                \end{feedback}
            \end{problem}

It is also worth noting that you can stack feedback environments in the same question. For example, try the following:

Ok, ok, that was unfair. Guess an integer this time. $\answer {31415}$

The answer is 31415. Obviously.

And this is only displayed because you’ve entered the correct answer. Good work!

Optional Arguments

Currently feedback only has two explicit optional arguments, which serve as triggers for when the feedback is displayed:

correct: The optional argument “correct” will ensure that the feedback is only displayed once the question has been flagged as “correct” and locked. This can be used for followup explanation of the problem once it has been correctly answered - in case you want to include information that would be too much information for a student still trying to answer it.
attempt: The optional argument “attempt” will display the feedback after any attempt, regardless of whether that attempt is correct or incorrect. This is the default behavior of feedback if no optional argument is provided, and note that it is the only behavior that gives feedback when a student gives a wrong answer. However, it also doesn’t go away once the student gets the correct answer.

In addition to the two above however, feedback has an open-ended optional argument as well - in particular you can trigger a basic validator check. This requires that you assign an id to the student answer, then call it in some kind of validation - e.g. if the id you assign to the student answer is $y$ , you can include as an optional argument in the feedback command things like: $y = 4$ (display this feedback if the student’s answer is $4$ ) or $y<-4$ (show this feedback if the student’s answer is less than $-4$ ). Keep in mind that this doing a very basic javascript comparison - it is massively more complicated to do something other than numbers here. In particular, you can’t use something like $y=x^2$ as a check to see if the student’s answer is $x^2$ , as there is (currently) no default support for algebraic checks here - it only does decimal value numbers (no fractions or irrational numbers). See examples below for an example of this kind of validation-based-feedback.

(Additional) Examples

For the first (two-part) example, we demonstrate how the feedback can be moved around relative to where the answer, and even the problem-statement, are displayed. This is something to keep in mind when writing feedback.

Example 1.1

The answer to this can be found in the feedback. But look where the feedback is generated, relative to the answer box.

The answer is 14.

$\answer {14}$

Example 1.2

The answer is 141.

The answer to this can be found in the feedback. But look where the feedback is generated, relative to the answer box.

$\answer {141}$

Example 2: Using feedback on correct answers

What is the smallest prime number? $\answer {2}$

The smallest prime is not 1. But you probably didn’t even write 1 did you? That’s because this is shown regardless of your answer, after your first attempt.

Correct! The answer is indeed 2. You should only see this if/when you entered the correct answer.

No, really, my favorite number is $y = \answer [format=integer,id=y]{17}$ . (Type $x$ . Type something larger than $17$ . Type something smaller than $17$ . Type $17$ .

You made a first attempt!

That number is TOO BIG.

That number is too small.

I have always loved the number $17$ .

Best Practices

Feedback versus Hints

There are (currently) two canonical ways to provide supporting information to the student while they work on a problem, either feedback or hints. As a purely anecdotal - but shockingly consistent and ubiquitous - observation, students view feedback as something that (at best) is a followup once the problem is correct, whereas “hints” are used to help get to understand how to complete a problem.

For a precalculus course, I included feedback to help students understand how to complete problems, on every single problem. At the end of the first semester after doing this, in a course of 500+ students, over half of them (just over 300) requested more support for the online problems/homework, despite every problem having feedback to help them figure out how to complete the problem.

For the next semester, I renamed the feedback environments into hint environments and moved them to show up above the problem statement instead of below. I didn’t change the content of the environment, nor did I add any content to any of the environments. At the end of that semester, out of around 450 students, I had less than 10 percent (around 30) students request additional support. Just changing from feedback to hint had a massive impact on the student feedback, even though the content was unchanged.

This could be a population quirk due to something about students attending my university (University of Florida) but I suspect it has more to do with how online educational resources have conformed and built expectations among students as my students are almost entirely first year university students.

Default Feedback is always seen.

Since default feedback is displayed on an attempt, it will be seen by students after their first attempt is made - whether they got it right or wrong. Moreover, it is never suppressed, meaning that, if they get the answer correct on their second or third attempt, the original “on attempt” feedback will still remain - it won’t go away due to credit status change. For this reason it is best to word your feedback in a way that either makes it clear it is only intended to be useful while the student is trying to get a correct answer. Alternatively you can try to word your feedback to be useful/insightful for both correct and incorrect answers - such as advice on why this particular problem is tricky or what some of the most common (incorrect) answers might be, and why. This can potentially help students later if/when they return to review content to study.

Potential Pitfalls and Problems

Limited triggering

The feedback system only has the two current methods of triggering, either a correct answer, or an attempt. There is a (very) limited custom validator written to give targeted feedback based on specific numeric answers, but it uses javascript validation methods, which are ... not awesome.

Content is only hidden, but it exists

Technically the content within a feedback environment exists, but is just hidden until triggered. This means, if you want to do something like put another problem within the feedback environment, and you customize the trigger so that a student completes the page without triggering the feedback, then the problem they don’t see still counts for “page credit” meaning that the student is now stuck without being able to get full credit.

Under the current system, the only way a problem locks is if they get it right, and correctly answered problems always display the feedback under either of the default trigger behaviors - meaning that the above concern isn’t possible under the existing system. But if you want to play with the system and do custom work, be aware that this is an easy pitfall to land in by accident.

2024-06-24 13:21:53

Press...	...to do
left/right arrows	Move cursor
shift+left/right arrows	Select region
ctrl+a	Select all
ctrl+x/c/v	Cut/copy/paste
ctrl+z/y	Undo/redo
ctrl+left/right	Add entry to list or column to matrix
shift+ctrl+left/right	Add copy of current entry/column to to list/matrix
ctrl+up/down	Add row to matrix
shift+ctrl+up/down	Add copy of current row to matrix
ctrl+backspace	Delete current entry in list or column in matrix
ctrl+shift+backspace	Delete current row in matrix

Type...	...to get
norm	$\|\|\blue{[?]}\|\|$
text	$\text{\blue{[?]}}$
sym_name	$\backslash\texttt{\blue{[?]}}$
abs	$\left\|\blue{[?]}\right\|$
sqrt	$\sqrt{\blue{[?]}}$
paren	$\left(\blue{[?]}\right)$
floor	$\lfloor \blue{[?]} \rfloor$
factorial	$\blue{[?]}!$
exp	${\blue{[?]}}^{\blue{[?]}}$
sub	${\blue{[?]}}_{\blue{[?]}}$
frac	$\dfrac{\blue{[?]}}{\blue{[?]}}$
int	$\displaystyle\int{\blue{[?]}}d\blue{[?]}$
defi	$\displaystyle\int_{\blue{[?]}}^{\blue{[?]}}\blue{[?]}d\blue{[?]}$
deriv	$\displaystyle\frac{d}{d\blue{[?]}}\blue{[?]}$
sum	$\displaystyle\sum_{\blue{[?]}}^{\blue{[?]}}\blue{[?]}$
prod	$\displaystyle\prod_{\blue{[?]}}^{\blue{[?]}}\blue{[?]}$
root	$\sqrt[\blue{[?]}]{\blue{[?]}}$
vec	$\left\langle \blue{[?]} \right\rangle$
mat	$\left(\begin{matrix} \blue{[?]} \end{matrix}\right)$
*	$\cdot$
infinity	$\infty$
arcsin	$\arcsin\left(\blue{[?]}\right)$
arccos	$\arccos\left(\blue{[?]}\right)$
arctan	$\arctan\left(\blue{[?]}\right)$
sin	$\sin\left(\blue{[?]}\right)$
cos	$\cos\left(\blue{[?]}\right)$
tan	$\tan\left(\blue{[?]}\right)$
sec	$\sec\left(\blue{[?]}\right)$
csc	$\csc\left(\blue{[?]}\right)$
cot	$\cot\left(\blue{[?]}\right)$
log	$\log\left(\blue{[?]}\right)$
ln	$\ln\left(\blue{[?]}\right)$
alpha	$\alpha$
beta	$\beta$
gamma	$\gamma$
delta	$\delta$
epsilon	$\epsilon$
zeta	$\zeta$
eta	$\eta$
theta	$\theta$
iota	$\iota$
kappa	$\kappa$
lambda	$\lambda$
mu	$\mu$
nu	$\nu$
xi	$\xi$
omicron	$\omicron$
pi	$\pi$
rho	$\rho$
sigma	$\sigma$
tau	$\tau$
upsilon	$\upsilon$
phi	$\phi$
chi	$\chi$
psi	$\psi$
omega	$\omega$
Gamma	$\Gamma$
Delta	$\Delta$
Theta	$\Theta$
Lambda	$\Lambda$
Xi	$\Xi$
Pi	$\Pi$
Sigma	$\Sigma$
Phi	$\Phi$
Psi	$\Psi$
Omega	$\Omega$

Feedback

How to generate/use the feedback environment.

Optional Arguments

(Additional) Examples

Example 1.1

Example 1.2

Example 2: Using feedback on correct answers

Best Practices

Feedback versus Hints

Default Feedback is always seen.

Potential Pitfalls and Problems

Limited triggering

Content is only hidden, but it exists

Controls

Symbols

Settings