HCC SPRING 2009
FACULTY COORDINATORS - UTILITY OF KNOWLEDGE SURVEYS
-

For the past four years, many HCC faculty have used Knowledge Surveys as part of their effort to more effectively assess and improve student learning outcomes.

More recently online Knowledge Surveys have been clustered to facilitate multiple course/discipline/program level analysis.

This survey is being administered to HCC Knowledge Survey program coordinators/discipline heads who have used Knowledge Surveys to gauge the effectiveness of the strategy, determine faculty satisfaction with the online Knowledge Survey process, to provide findings for the HCC Accreditation Midterm Report, and to solicit suggestions on how the online process and resulting reports might be improved.

The survey findings will be shared with respondents and the Assessment Committee.

Mahalo for your candid responses!

-

Course/Discipline/Program for which I coordinated Knowledge Survey Assessment:
(Check all that apply)

ANTHROPOLOGY FOCUS - SYMBOLIC LOGIC
COSME FIRE
ENG 22 ICS
ENG 100 FOOD, SCIENCE, & HUMAN NUTRITION
WORLD CIVILIZATION OESM
FOCUS - ETHICS HUMANITIES
FOCUS - WRITING INTENSIVE SOCIAL SCIENCES
FOCUS - ORAL COMMUNICATION ASSOCIATE OF ARTS DEGREE
Other:
-
How many semesters have you used Knowledge Surveys at the group/discipline/program level?
1 2 3 4 5 6 7 8
-
How frequently do you use (plan to use) Knowledge Surveys for this group level analysis purpose?
More than once a semester Every Semester Every other semester Every few semesters
-
In which format(s) do respondents complete the Knowledge Surveys?
Exclusively - paper Some paper - some online Some paper - mostly online Exclusively - online
-
If students complete the Knowledge Surveys online, where do they normally complete the online Knowledge Surveys?
Mostly in Mobile Computer Lab (7-620)
Mostly with laptops in my classroom
Mixed - some in computer labs and some in classroom with laptops
Mostly by providing students with web address of the Knowledge Survey
-
In what ways did you participate in the design process of the Knowledge Surveys you have used? (Check all that apply)
No participation - Knowledge Survey was designed for me by Assessment Researcher
Drafted KS items from existing SLOs
Created new SLOs/KS items by myself
Jointly created SLOs with others in my department/discipline
Worked with Assessment Researchers to create KS items
-
If you participated in designing your initial or revised Knowledge Survey, about how much time did you devote to the design/drafting process?
Less than an hour Between 1 and 5 hours Between 5 and 10 hours More than 10 hours
-
If you have participated in the design of your Knowledge Survey, how often have you revised the initial KS?
Never revised Every Few Semesters Every semester
-
Have you conducted PRE-TEST (Beginning of Semester) and POST-TEST Knowledge Surveys?
Just POST-TEST Both PRE and POST-TESTS
-
If you have conducted PRE and POST-TEST Knowledge Surveys, how valuable have you found the consequent ability to gauge "value added" learning outcome gains?
No value Modestly valuable Very Valuable Extremely Valuable
-

ONLINE KNOWLEDGE SURVEY PROCESS

Very Difficult Fairly Difficult Fairly Easy Very Easy
-
How difficult/easy was it for your faculty to sign up for time slots (or get laptops to use in your classroom) to conduct the Knowledge Survey?
-
How difficult/easy was it for students to find the right evaluation for the class on the evaluation website?
-
How difficult/easy was it for students to complete the Knowledge Survey?
-
How difficult/easy was it for students to reset the screen to "home" (original evaluation website)?
-
How often have you (your faculty) encountered technical problems (network down, laptops out of power or asleep, etc.) while conducting Knowledge Surveys?
Never A few times Often Always
-
If you have OFTEN or ALWAYS encountered technical problems while conducting Knowledge Sruveys, please describe those problems for us (so they can be corrected)
-
How long did it take for the average student to complete the Knowledge Survey?
Less than 5 minutes Between 10 and 15 minutes
Between 5 and 10 minutes More than 15 minutes
-
How would you rate the overall process of conducting the online Knowledge Surveys?
Poor Fair Good Excellent
-
Suggestions for improving the online Knowledge Survey process?

-
EVALUATION REPORTS
-
How satisfied are you with each of the following aspects of the evaluation reports you received?

Very Dissatisfied Somewhat Dissatisfied Somewhat Satisfied Very Satisfied
-

Promptness of report preparation/delivery

-
Format of the report
-
Report statistics
-
Report graphics
-
Comparison (item mean values) of the individual faculty evaluation results with group results
-
Suggestions for improving the online Knowledge Survey report format/content?

-
UTILITY OF KNOWLEDGE SURVEY FINDINGS
-
PLEASE INDICATE THE DEGREE TO WHICH YOU HAVE USED YOUR GROUP/DISCIPLINE/PROGRAM LEVEL KNOWLEDGE SURVEY REPORTS FOR EACH OF THE FOLLOWING ITEMS:

Never Slight Degree Significant Degree Major Degree
-
To determine which learning outcomes seem to be adequately achieved and which seem to require additional attention
-
To gain insights about how different type (Bloom Taxonomy) of learning seem to be accomplished to adjust course emphasis
-
To gauge the degree to which Program/Degree (broad) SLOs are being met in the class/discipline
-
To share/discuss insights about student learning with teaching faculty
-
To work with faculty to devise strategies in an attempt to improve learning outcomes in areas that appear to need attention
-
To compare a semester's findings with the previous semester's findings to gauge learning outcome progress
-
To compare learning outcomes in different formats of the course/discipline (distance education vs... traditional classroom; learning communities vs... traditional classroom; etc..)
-
To revise the Knowledge Survey to improve the process and focus on different learning outcomes
-
Have KS findings caused your faculty to change teaching strategies to improve learning?
No, teaching hasn't changed as a result of KS findings
Yes - minor teaching changes have been made in some areas
Yes - major teaching changes have been implemented
-
Please provide examples of how you have adjusted instructional strategies as a result of KS findings:
-
If you changed your teaching strategies as a result of the findings of your Knowledge Surveys, have subsequent assessments indicated any improvement in student learning outcomes in these areas?
No perceived improvement Slight perceived improvement Major perceived improvement
-
Overall, how valuable have Knowledge Surveys been to you in assessing student learning outcomes in your courses/disciplines/programs?
No value Modestly valuable Very Valuable Extremely Valuable
-
For instructional improvement, how effective are your Knowledge Survey findings compared to the traditional teaching evaluation reports you have received during your career
Traditional teaching evaluations have been more effective
Traditional teaching evaluations and Knowledge Surveys findings have been equally effective
Knowledge Survey findings have been more effective than traditional teaching evaluations

-

Please comment on the usefulness of the "program level" reports you have received?

-
How have you used the findings to improve Student Learning Outcomes (SLOs)?
-
MAHALO FOR PROVIDING FEEDBACK TO IMPROVE THE KNOWLEDGE SURVEY PROCESS!