'Big data' analysis of flash-card learning data

Flash-cards combined with spaced repetition is a great way of learning some kinds of knowledge - like remembering facts or extending your vocabulary in a foreign language.

The current 'digital exam push' to test also higher levels of maturity with multiple choice questions raises the question whether flash cards also has some extended use for deeper types of knowledge and skills.

It /is/ possible to make short-answer questions testing maturity in a field. But we teachers acknowledge that it is hard to make good questions and unfortunately the reusability is low.

Assume the context of flash-cards technology used in an Ntnu course:

• It is hard to make good questions: We can challenge the students to make good questions. Making a good question requires in itself a good mastery of the topic.
• A good question is one that some but not all manage to answer correctly, starting out.
• ... and has an unambiguous answer - that everyone learns in the end.

Letting students make such questions for each other we can measure things like:

• The students up-front assessment of the quality and difficulty of the question. This again requires maturity to get right.
• The /real/ quality and difficulty of the questions.
• Questions (question groups) that are seen as hard and slow to learn may yield feedback to learning ambitions in the course.
• Correlations between questions - if one student manages question A then it is likely that he also manages B. Leading to clusters of connected knowledge and skills.
• I'll add an 'Etc.' here...

An earlier student has made the web platform/infrastructure that may let us collect these data.