I think this system would be valuable for our precoure evaluation. We could set up questionnaires with random math problem and get a better idea on the math background of our students. It doesn't seem possible to start from scratch and provide a good preparation course.

Some Nodes I made on the Tutorial

• focus: high school and undergraduate students
• exercise types: multiple choice, open answer, fill in blanks
• questions are automatically generated (questions may include graphics)
• 2000 questions types from which (a virtual unlimited number of) specific problems can be generated
• answering by formulae, automatic verification
• exercises are algorithmic
• problems include hints and complete solutions
• automatic grading
• purpose: assess and practice mathematical skills
• purpose: illustrating graphs and images
• questions are arranged according to a standard taxonomy (classification of exercises)
• usage: questions are used by high school teachers as homework (giving bonus points)
• usage: for diagnostic testing at the beginning of university courses; this allows to divide students into groups according to their individual learning needs
• benefits: immediate feedback to students and automatic grading for teachers

Student Mode

• Multiple Choice quiz: returns my answer, correct answer, comment and a visual button for correct/ incorrect answers.
• It would be nice if I could ask for hints or an example for incorrect problems. Some questions provide hints, but for learning and understanding these might not sufficient
• Student have to adapt to the maple syntax for entering formula, they have to adapt to the notation of the system e.g. (4;3) is incorrect but (4,3) is correct; however this was never a problem - at the beginning of a course they do test quizzes to get familiar with the system ...
• Students can take the quizzes as often as they wish, the system remembers their attempts and reduces the grade respectively. But this can also be ignored, i.e. taking quizzes becomes a game in which the best score counts.
• For high school math this seems to work; e.g. for questions on graphs of function (intersection points) or simple calculations; for more complex problems student can print the quiz and then entering their final answer. But then the teacher looses the way of solving the problem - here we sometimes give extra points even the result might be wrong.
• It is a better approach to model question types and generate concrete problems from that: this reduces cheating as each student can get different problems and answers can not be copied so easily.
• Student has a grade book: can see his answers and final results

Teacher Mode

• teacher get an overview of the students results and can manually change points, give extra credits, and enter comments.
• teacher can edit assignments: adding/ deleting question types (during runtime these types are instantiated) and modify their points; setting a passing score (students have to take the assignment until a certain number of points)
• creating anonymous practice that are used for grading.

Programmer Mode

• writing questions is more difficult, you actually write little programs (issue with debugging, careful editing)
• one has to write the algorithm using maple; preview is available; from that the problems can be generated
• Question creation is based on the Maple TA System, which provides an editor
• enter formulae in Maple Syntax to generate Presentation MathML or enter Presentation MathML using the editor
• Webalt started in 2005, about 12 people create these question programs in the maple programming language
• any type of question can be modelled (e.g. using free text, but then automatic verification is not be possible)

Presentation Issue
The system is generating Presentation MathML, however, for the presentation it is converted into an image. But their have been critics on this as e.g. enlargement of the fonts will not enlarge the image (for visual impaired people) and conversion to braille will fail.

Some Further questions I have:

• What about alternative solutions? Can you support all problem solving strategies or do you enforce certain practice? Do you really practice all possible mathematical skills and the way of doing math? Is this not too result-focus?
• Which skills do you want to train? Have you ever modelled the competency students gain by solving these exercises?
• Is the taxonomy public? Have you linked exercises to mathematical concepts, e.g. if teachers is talking about concept "A" you can retrieve the best suited questions?
• Are questions interlinked/ related/ have you modelled dependency?
• Are you sure that: "students gain understanding of their learning process and can concentrate on the weak points and practice more" I wonder if students can reflect on the problem solving process while answering the questionnaires. Did you do any evaluations?