MILLIONS OF MULTIPLE CHOICE QUESTIONS

Might as well smile.

 

UGH—If you think having to pass multiple choice exam to graduation from massage school and become licensed and then finally certified is a pain you should try writing them.  I write textbooks for massage education and each book has a huge test bank.   I also write a review guide for the tests used for licensing.  I am in process of writing 10 practice tests 150 questions each and my brain is exhausted.  I have five of them completed and five to go.  I write really good multiple choice question. I see every question as a little problem to be solved and a little lesson to be learned. I wish I could tell you that I approve of  multiple choice questions as a method for evaluation but I don’t (even though I write really good multiple choice questions).  Doesn’t matter if I approve or not- the licensing exams are based on multiple choice exams.  Multiple-choice questions can be used to measure knowledge outcomes and various types of learning outcomes.  They are most widely used for measuring knowledge, comprehension, and application outcomes.

 I think the biggest reason multiple choice questions are used because scores are more reliable than subjectively scored items (e.g., essays),  scoring is easy, objective, and reliable and they can cover a lot of material very efficiently (about one item per minute of testing time).

 Constructing good questions is time consuming and it is difficult to find plausible wrong answers. My biggest concern is that real world problem solving differs – a different process is involved in a real critical thinking and clinical reasoning processes.    If a multiple choice exam is used that the questions should be well written.

Multiple-Choice Item Writing Guidelines

From: http://ctl.utexas.edu/assets/Evaluation--Assessment/Writing-Good-Multiple-Choice-Exams-04-28-10.pdf

Multiple-choice questions typically have 3 parts: a stem, the correct answer – called the key, and

several wrong answers, called distractors.

Procedural Rules:

• Use either the best answer or the correct answer format.

• Best answer format refers to a list of options that can all be correct in the sense that each

has an advantage, but one of them is the best.

• Correct answer format refers to one and only one right answer.

• Use good grammar, punctuation, and spelling consistently.

• Minimize the time required to read each item.

• Avoid trick items.

• Use the active voice.

• The ideal question will be answered correctly by 60-65% of the tested population.

• Have your questions peer-reviewed.

• Avoid giving unintended cues – such as making the correct answer longer in length than the

distractors.

Content-related Rules:

• Base each item on an educational or instructional objective of the course, not trivial

information.

• Test for important or significant information.

• Focus on a single problem or idea for each test item.

• Keep the vocabulary consistent with the examinees’ level of understanding.

• Avoid overly specific knowledge when developing items.

• Avoid textbook, verbatim phrasing when developing the items.

• Avoid items based on opinions.

• Use multiple-choice to measure higher level thinking.

• Be sensitive to cultural and gender issues.

• Use case-based questions that use a common text to which a set of questions refers.

Stem Construction Rules:

• State the stem in either question form or completion form.

• When using a completion form, don’t leave a blank for completion in the beginning or

middle of the stem.

• Ensure that the directions in the stem are clear, and that wording lets the examinee know

exactly what is being asked.

• Avoid window dressing (excessive verbiage) in the stem.

• Word the stem positively; avoid negative phrasing such as “not” or “except.” If this cannot

be avoided, the negative words should always be highlighted by underlining or capitalization:

Which of the following is NOT an example ……

• Include the central idea and most of the phrasing in the stem.

• Keep the length of options fairly consistent.

• Avoid, or use sparingly, the phrase all of the above.

• Avoid, or use sparingly, the phrase none of the above.

• Avoid the use of the phrase I don’t know.

• Phrase options positively, not negatively.

• Avoid specific determinates, such as never and always.

• Make sure that there is one and only one correct option.

Distractor (incorrect options) Development Rules:

• Use plausible distractors.

• Incorporate common errors of students in distractors.

• Use familiar yet incorrect phrases as distractors.

• Use true statements that do not correctly answer the item.

• Avoid the use of humor when developing options.

• Distractors that are not chosen by any examinees should be replaced.

Suggestions for Writing Good Multiple Choice Items:

• Present practical or real-world situations to the students.

• Present the student with a diagram of equipment  or procedure and ask for application, analysis or

evaluation.

• Use pictorial materials that require students to apply principles and concepts.

• Use charts, tables or figures that require interpretation.

For all my whining and complaining the multiple choice question is here to stay until a better way evolves and right now I do not have a better way for the licensing exam process.  When I write practice exams I will include a few questions that break the rules provided above just so the reader will have experience with it.   By the time anyone trudges though 10 practice exams they should be able understand  how to approach the multiple choice exam format and since licensing exams are multiple choice exams so if you are going to be licensed as a massage therapist you have to pass a multiple choice exam whether I like it or not.

 I do however have a better way for the classroom- Rubrics.

A rubric is an explicit set of criteria used for assessing a particular type of work or performance. A rubric usually also includes levels of potential achievement for each criterion, and sometimes also includes work or performance samples that typify each of those levels.  Levels of achievement are often given numerical scores.  A summary score for the work being assessed may be produced by adding the scores for each criterion.

http://www.tltgroup.org/resources/flashlight/rubrics.htm

Here is a general example

CRITICAL THINKING RUBRICS

Based on a draft from Elaina Bleifield and the Paulus CT Group

http://openedpractices.org/files/Bloom%20North%20Hennepin%20CC%20Critical%20Thinking%20Rubric.pdf

 

CATEGORY ONE: KNOWLEDGE AND COMPREHENSION (understanding the basics)

4—The work consistently demonstrates clear, accurate, detailed and comprehensive understanding of the relevant facts /data / theories/ terms as well as the ability to organize the information for application, presentation, documentation, and/orfurther examination.

3--The work demonstrates an adequate understanding of the relevant facts / data / theories/ terms as well as the ability to organize the information for application, presentation, documentation, and/or further examination

2-- The work demonstrates an uneven and shaky understanding of the relevant facts / data / theories/ terms as well as a limited ability to organize the information for application, presentation, documentation, and/or further examination.

1-- The work demonstrates an inadequate understanding of the relevant facts / data / theories/ terms as well as a limited ability to organize the information for application, presentation, documentation, and/or further examination.

CATEGORY TWO: APPLICATION AND ANALYSIS (attaining the concept)

4—The work demonstrates confident ability to work with the key concepts / information / process / theory -- applying or extending them to a wide variety of new problems or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.

3--The work demonstrates adequate ability to work with the key concepts / information / process / theory -- applying or extending them to a variety of new problems or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.

2-- The work demonstrates uneven and shaky ability to work with the key concepts / information / process / theory --applying or extending them with mixed success to new problems or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.

1-- The work demonstrates extremely limited ability to work with the key concepts / information / process / theory --applying or extending them with very limited success to new problems or contexts, making predictions, recognizing hidden meanings, drawing inferences, analyzing patterns and component parts, communicating insightful contrasts and comparisons.

CATEGORY THREE: SYNTHESIZING AND EVALUATING (going beyond the given)

4—The work demonstrates surprising/insightful ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as to assess discriminatively the value, credibility and power of these ideas (etc) in order to decide on well-considered choices and opinions.

3-- The work demonstrates adequate ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as to assess discriminatively the value, credibility and power of these ideas (etc) in order to decide on well-considered choices and opinions.

2-- The work demonstrates uneven and superficial ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as a limited ability to assess discriminatively the value, credibility and power of these ideas (etc) in order to decide on well-considered choices and opinions.

1-- The work demonstrates little ability to take ideas / theories / processes / principles further into new territory, broader generalizations, hidden meanings and implications as well – as well as a limited and superficial ability to assess discriminatively the value, credibility and power of these ideas (etc) in order to decide on well-considered choices and opinions.

 

Here are some more examples

 

 

http://openedpractices.org/files/Problem%20solving%20Sinclair%20CC.pdf

 

http://openedpractices.org/files/ProblemSolving%20source%20unknown.pdf

 

Well now that I have vented I need to get back to writing multiple questions. UGH

ADVANCED MASSAGE IS ABOUT THE RESULT

Yes that is a dog on the massage table.  To be an advanced massage therapist you also need to be flexible. 

Advanced massage begins with a goal for the outcome of the massage.  I have been observing myself while giving massage (being my own teacher) though the lens of "advanced massage" and identified that I pay attention. That may seem simplistic but I think this is very important.  Regardless of the goal for the massage (or series of massage sessions) my focus remains pretty solid.  Yes my mind will wander -whose doesn't? However because I know what I am trying to accomplish I stay on track.  I have a massage routine that I tend to follow-not stuck on it but the sequence allows me to be systematic about the assessment process.  The general full body massage that will lead to a state of relaxation ( primary goal) but is also the main assessment for the secondary goals of the session.  For example, I may be working on scar tissue limitations during knee movement. I feel strongly that spot work is limited in effectiveness and almost always provide a full body massage that moves the various tissue layers and moves all the joints. Again to me this is assessment  but to the client it feels like an excellent general massage.  During the massage if indicated I may perform some additional assessments such as a muscle strength test or an orthopedic test.  I do "clean up " minor tissue , postural and movement issues I find along my journey through the massage even if they may not be directly related to the identified goals because often -guess what- they are or involved in some aspect of compensation that no longer is desirable.  You know--might as well pick up the clothes on the floor while on my way to do the dishes-as an example. I don't loose my focus though. If the client goal is less back aching then my massage stays on that path.  I won't go off path in another direction to work on something- only address issues I find along the way.  You can't do everything in one massage for goodness sake! Advanced massage is about know when to quit as well as when to do something.

 I do not use a bunch of " methods".  For example - I don't think in terms of lymph drain, myofascial release, trigger point, energy work, deep tissue and the hundreds of other approaches out there.  I think in terms of how can applying mechanical forces to the body support a more normal function. I have to really understand anatomy and physiology and pathology to function this way.  For example, if something is swollen I want to know why.  Swelling can be a really good thing ya know.  If something hurts I want to know why.  Pain can also be a really good thing, If however, there is too much of a good thing that is now a bad thing. I wonder what massage can do to tip the physiology back toward homeostasis. So, if an area is excessively swollen and if I understand the reason why it is swollen, I may attempt to mimic the normal action of the lymphatic system with a rhythmic pumping action. If tissue is dense and lacking pliability and it is causing problems I might attempt to soften it by kneading and rolling it.  You get the idea I hope.  While doing this I position the client's body so I can most easily get to the area I am working on and apply the forces (ie tension force, compression force, shear force) as easily and efficiently as possible. I am really efficient and seldom need more than 60 minutes to be effective.  Rarely is there a need for the 90 minute massage.

Occasionally the client will endure an uncomfortable sensation such as a burning pulling sensation as I pull on bound down scar tissue or a localized tender point that I might focus on with some inhibitory pressure but it isolated, focused directly on the goal, feels right and is familiar. It makes sense to the client.  Sometimes the client may be a little sore to the touch  in an isolated spot but it is important that the massage does not make the client sore to movement or painful and stiff in large areas.

 I do not use a lot of stretching during the massage.   My research indicates that there is often more potential for harm than benefit. Besides most problems that people stretch for are related to hyper mobile  joint function.  The limits in movement are the body's attempt to stabilize the joint.  I may use localized direct tissue stretching but I am not supportive a aggressive stretching methods.  Passive and active joint movement to asses ROM.- yes. Movement of a joint beyond physiological limits and to anatomical barriers--NO! I use simple muscle energy methods, usually contract relax antagonist contract, and then gently stretch only those areas with hypo-mobility and only look for an increase is 5 to 10 degrees. I mess with increasing the range over a series of sessions and remember the GOAL is the GOAL!  Functional flexibility- good but too loose in the joints- REALLY BAD.

Goals for massage outcomes can be clustered into four categories   Relaxation/pleasure--( really really important), stress management , functional mobility, and pain management.  Often there is overlap but the good news is that a really good general massage with moderate pressure will address relaxation, stress and pain issues.  Mobility goals require the little bit of specific work. Not gobs- just some.

Finally-to be advanced you need to really understand the body as a functioning whole- all the anatomy and physiology together- not just individual muscles. Relaxation, stress and pain goals are related to the nervous system and endocrine system for goodness sake.  To be advanced as a massage therapist you need to put in your time and be your own teacher while doing hundreds of massage sessions. It is important to learn from experienced massage therapists who do not rely on gimmicks and complicated stuff  and want to sell you stuff and most important have practiced massage for years and years.  And to be advanced you have to care about your clients.

I hope this series has been  helpful.