- 5 Steps to Design a Questionnaire
I was asked to suggest some tips to plan a survey which would be a needs assessment for a mentoring program within the organization.
Mentorship
Mentorship is a close relationship between a mentor and a mentee and can be formal or informal (Lundin, 2013). Mentoring programs have been used in clinical training and research for improving patient outcome, fostering knowledge transfer, enhancing evidence-based practice, and encouraging professional development (Black, Bungay, MacKay, Balneaves, & Garossino, 2016; Gagliardi, Webster, & Straus, 2015; Norman & Roche, 2015).
Needs Assessment
A needs assessment identifies the gap between a requirement and its fulfillment and prioritizes various needs (Altschuld & Watkins, 2014). Altschuld and Watkins (2014) stated there were three stages of needs assessment: Preassessment, assessment, and postassessment. First, the inquirer should consolidate data and literature (preassessment). Then, the inquirer may need to conduct a survey to gather more information (assessment). After the assessment, an action plan should be devised using the survey results (Altschuld & Watkins, 2014).
Examples of Needs Assessment and Mentorship
In healthcare, there are examples of mentorship needs assessments (Sawatzky & Enns, 2009; Sinclair, Fitzgerald, Hornby, & Shalhoub, 2015; Wan, Cheung, Tsirigotis, & Banks, 2018). Sawatzky and Enns (2009) used a Likert scaled questionnaire to find out the pros and cons of being a mentor in 29 full-time nurse educators.
Sinclair et al. (2015) administered a 47 item survey and received 565 completed response from trainee surgeons in the UK and Ireland. They found that about half of the respondents indicated they had a mentor, although most wanted one. Also, most respondents showed interest to be a peer mentor. Despite they were willing to be trained as a mentor, only 8.7% received mentoring training (Sinclair et al., 2015). In both surveys, the researchers designed novel questionnaires which were peer-reviewed and verified by pilots.
Wan et al. (2018) distributed a 6-question needs assessment before a mentorship experience. They reflected that it was an invaluable tool to prepare the event to meet the participants’ learning needs.
Other examples of mentorship-needs-assessment surveys include:
Designing a Survey
From the above examples, it seems that a survey is the typical means for conducting a needs assessment. Frequently in the form of a questionnaire, it is the best tool to gather perceptions (Lodico, Spaudling, & Voegtle, 2010). A needs assessment survey is used to ask the community what the members need most (University of Kansas, 2014).
A questionnaire is a communication tool which is highly specific to its audience. That is, the author must identify the need for a survey (the why), what information is needed, who will supply the data, and how to ask the questions. It makes sense to conduct a cross-sectional (one-shot) survey (Huber et al., 2015; Lodico et al., 2010; Sinclair et al., 2015). Furthermore, the design and conduct of a questionnaire must adhere to the ethical standards and the inquiry form must include a confidentiality statement (Benini et al., 2016; Lodico et al., 2010).
Steps to Design a Questionnaire
The following is a brief description of the steps taken to design a questionnaire, adapted from Benini et al., (2016) and Lodico et al., (2010).
- Identify the purpose
- What information is needed?
- Only collect the absolutely necessary info (which leads to decision making)
- Identify the source of information
- Who is the target audience?
- How to acquire the info? E.g. Interview, focus group, survey (As discussed before, it is more likely to be a survey)
- How many responses are needed? Please see below.
- Draft the questionnaire
- How many questions? Please see below.
- Questionnaire components
- Demographics (either at the beginning or the end)
- The body of the survey
- Format, for instance
- Likert scale: Choose one of the preset responses (e.g. Strongly agree, agree, disagree, strongly disagree)
- Semantic scale: Choose a response within a range (e.g. 1-10)
- Free text: It is important to allow respondents to express views other than the predetermined answers
- Confirm validity – to ensure that the questionnaire is valid, one or both of the following can be used
- Pilot: Randomly select a few participants in the population to test if the survey measures what needs to be measured.
- An expert panel to evaluate the appropriateness in terms of depth and breadth
- Administer the survey – usually conducted online, but other modes include via phone or mail
Questionnaire Design Practical Points
The anatomy of a questionnaire
How many responses are needed? In our department of several hundred, it makes sense to survey every staff (University of Kansas, 2014). However, we can also refine our population, such as limited to techs only (Kalpana, 2016). SurveyMonkey (2017) provided a formula to calculate the required number of responses to ensure the sample is representative of the population.
How many questions should we ask in a mentorship needs assessment questionnaire? From the previous examples, the inquirers asked between six and 47. Survey experts suggested 10 questions because shorter surveys yield better participation (Chudoba, 2010; Millikin, 2016; Zheng, 2011). Hence, a high-quality questionnaire should get the most answers with the least stretch to avoid survey fatigue (Millikin, 2016).
How to analyze the data and present the results? The goals of conducting a survey are to answer the overarching questions and present the results (Benini et al., 2016). Because a survey is usually considered a quantitative study, the results are analyzed quantitatively and presented as figures (Lodico et al., 2010). However, the free-text responses, if present, will need a qualitative analysis which results are organized in themes.
Quality of Questions
Adopted from Lodico et al. (2010):
- Specific terms must be clearly defined
- Each item collects data to answer one question – otherwise, it will be confusing for the respondent and impossible to interpret the results
- Avoid double negative, e.g., “I will not progress in my career if I do not receive guidance.”
- The list of answers to a question must be mutually exclusive and cover all possible responses – no overlap and nothing is left out.
Acknowledgment
I wish to thank Jean Nash, Medical Imaging Site Manager and Professional Practice Leader of the Joint Department of Medical Imaging, Toronto, for assigning this task to me. I learned a lot from reviewing the survey methodologies.
References
Altschuld, J. W., & Watkins, R. (2014). A Primer on Needs Assessment: More Than 40 Years of Research and Practice. New Directions for Evaluation, 2014(144), 5–18. https://doi.org/10.1002/ev.20099
Benini, A., Bruni, E., Walton-Ellery, S., Garfield, R., Koedam, A., Legoupil, E., … Weihmayer, M. (2016). How to design a questionnaire for needs assessments in humanitarian emergencies. Retrieved from https://www.acaps.org/sites/acaps/files/resources/files/acaps_technical_brief_questionnaire_design_july_2016_0.pdf
Black, A. T., Bungay, V., MacKay, M., Balneaves, L. G., & Garossino, C. (2016). Understanding mentorship in a research training program for point-of-care clinicians. Journal of Nursing Administration, 46(9), 444–448. https://doi.org/10.1097/NNA.0000000000000373
Chudoba, B. (2010). Does Adding One More Question Impact Survey Completion Rate? Retrieved from https://www.surveymonkey.com/curiosity/survey_questions_and_completion_rates/
Gagliardi, A. R., Webster, F., & Straus, S. E. (2015). Designing a knowledge translation mentorship program to support the implementation of evidence-based innovations. BMC Health Services Research, 15(1), 198. https://doi.org/10.1186/s12913-015-0863-7
Huber, J., Nepal, S., Bauer, D., Wessels, I., Fischer, M. R., & Kiessling, C. (2015). Tools and instruments for needs assessment, monitoring and evaluation of health research capacity development activities at the individual and organizational level: a systematic review. Health Research Policy and Systems, 13(1), 80. https://doi.org/10.1186/s12961-015-0070-3
Kalpana, C. (2016). SurveyMonkey: How Many People Do I Need to Take My Survey. Retrieved from https://www.surveymonkey.com/curiosity/how-many-people-do-i-need-to-take-my-survey/
Lodico, M. G., Spaudling, D. T., & Voegtle, K. H. (2010). Methods in educational research: From theory to practice. San Francisco, CA: Jossey-Bass.
Lundin, L. L. (2013). Mentorship. In Salem Press Encyclopedia. Retrieved from https://eds-b-ebscohost-com.ezp.waldenulibrary.org/eds/detail/detail?vid=2&sid=49d3b8b8-4ed8-4679-97c3-e1e25b83d7f7%40sessionmgr101&bdata=JnNpdGU9ZWRzLWxpdmUmc2NvcGU9c2l0ZQ%3D%3D#AN=89677586&db=ers
Millikin, L. (2016). How many questions should I have in my survey? Retrieved from https://www.surveygizmo.com/resources/blog/how-many-survey-questions/
Norman, K., & Roche, K. (2015). Mentors: Supporting learning to improve patient care. British Journal of Healthcare Assistants, 09(3), 132–137. https://doi.org/10.12968/bjha.2015.9.3.132
Sawatzky, J. A. V., & Enns, C. L. (2009). A Mentoring Needs Assessment: Validating Mentorship in Nursing Education. Journal of Professional Nursing, 25(3), 145–150. https://doi.org/10.1016/j.profnurs.2009.01.003
Sinclair, P., Fitzgerald, J. E. F., Hornby, S. T., & Shalhoub, J. (2015). Mentorship in Surgical Training: Current Status and A Needs Assessment for Future Mentoring Programs in Surgery. World Journal of Surgery, 39(2), 303–313. https://doi.org/10.1007/s00268-014-2774-x
SurveyMonkey. (2017). Sample Size Calculator: Understanding Sample Sizes | SurveyMonkey. Retrieved from https://www.surveymonkey.com/mp/sample-size-calculator/
University of Kansas. (2014). Chapter 3. Assessing Community Needs and Resources | Section 7. Conducting Needs Assessment Surveys | Main Section | Community Tool Box. Retrieved from https://ctb.ku.edu/en/
Wan, D., Cheung, C., Tsirigotis, D., & Banks, L. (2018). Facilitating Mentorship: The Canadian Cardiovascular Society Trainee Committee Experience. Canadian Journal of Cardiology, 34(1), 1–3. https://doi.org/10.1016/j.cjca.2017.11.002
Zheng, J. (2011). How Many Questions Do People Ask? Retrieved from https://www.surveymonkey.com/curiosity/how-many-questions-do-people-ask/
Education Issues for Female Baby Boomers 