An Assessment Bank for Connexions

On Wednesday of last week (11^th Feb 2009) I participated in a meeting to discuss adding an assessment bank to Connexions. The idea isn’t new and has been floated in a variety of guises on numerous occasions over the last year. In fact, we’ve already moved well beyond discussing it as a possibility and all discussion was focused on what the assessment bank would need to do.

The conversation included a number of the Connexions team, Kathi Fletcher, Manpreet Kaur, Philip Schatz, and Vickie Nguyen, as well as Michelle Smith from the University of Colorado at Boulder, Margaret Beier from Rice University, and Roché Compaan from Upfront Systems. This diverse group ensured that we discussed issues from the technical implementation through to the research aspects of creating assessment items which are effective and useful. Specific use cases differ but collectively we want to build an assessment bank tool that can be used stand-alone or plug seamlessly into Connexions.

From the Siyavula side, our initial interest was in a question bank that:

• allows teachers to integrate questions into their Connexions modules,
• allows for the creation of random exercise sheets/tests quickly and easily that meet a set of criteria (subject, topic, difficulty, etc.),
• allows teachers to add questions to the bank, and
• accepts feedback on specific questions.

Once a critical mass of questions has been achieved, this assessment bank would allow a teacher to create a test or exercise sheet with separate memorandum in a few seconds, saving teachers a lot of time. A simple interface covering:

• subject
• grade
• topic
• number of question for each cognitive level
• and many optional refinements ex. total marks etc.

would allow us to sample the bank of questions and produce a question paper and memorandum which the teacher could download.

This also allows the teacher to make a custom exercise sheet for learners with special needs quickly and easily, be those more advanced or weaker learners.

Given that the alleviation of time pressure on teachers our focus would be on encouraging teachers to add new questions with a focus on originality and quality as well as to provide feedback on questions.

For a tool like this to be useful the questions need to be stored in such a way that we can extract the questions and solutions easily. More sophisticated use requires even more markup. We are currently leaning towards implementing the most common set of question types from the QTI-specification. Examples of the question types can be found here.

If built exposing an API the assessment bank could be integrated into other tools. One example of such a tool has emerged in discussions with Anil Kanjee of the Human Sciences Research Council (HRSC). The HSRC plans to build a diagnostic tool which couples to an assessment bank. With the correct meta-data stored for every question their tool will capture learners performance, like a Learning Management System, and provide feedback to the teacher on learner’s weaknesses and strengths. It will also allow comparison on a class-, school-, district- or national level with additional restrictions based on other relevant meta-data. The tool can then also provide teachers with recommended strategies to address learner’s shortcomings.

The HSRC has been developing questions and the required meta-data and is now busy with development of the assessment bank. The diagnostic tool will be built and trialled over the next few years.

I think that this is a very exciting extension to Connexions and will save teachers a lot of time. Alleviating some of the time pressure on teachers should have knock-on benefits in all areas as an added bonus.