Assessment: Otero Junior College

Teaching and Learning: Evaluation and Improvement

OJC Assessment

At Otero Junior College our mission is “To be a leader in Colorado's Community College System when it comes to education.” To achieve that mission we will “provide innovative and responsive educational and economic opportunities in an accessible, inclusive environment that promotes success for students, employees and the community.”

Through our assessment process OJC demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement.

 

Overview of the Assessment Program at OJC.

  • I. Philosophy

    The Assessment philosophy at Otero Junior College is based on the principles of assessing skills, reviewing data and taking action.

    Additionally, assessment is a system where faculty use the data collected to improve teaching and student learning. It is not used to give grades, nor is it a faculty evaluation system.  Assessment is used by the entire campus for strategic planning and budgetary decisions and assessment aligns with the Higher Learning Commission Accreditation.

    Criterion Four. Teaching and Learning: Evaluation and Improvement

    The institution demonstrates responsibility for the quality of its educational programs, learning environments, and support services, and it evaluates their effectiveness for student learning through processes designed to promote continuous improvement.

  • II. Definition of Assessment

    Assessment is the systematic collection, review, and use of information about educational programs to improve teaching and student learning.

  • III. Goals of Assessment

    Otero Junior College
    2015-2017 Assessment Plan

    Universal Assessment Activities

    Task Timeline Responsible Parties Status

    Create Assessment cycle in Taskstream for ISLOs, PSLOs and Co-curricular

    Annually in Fall

    Assessment Coordinator

    • Fall 2015 created and implemented a full assessment cycle in Taskstream.
    • Separated assessment for 2016 cycle in to Spring and Fall and added syllabus repository

    Develop “How To” guide for Taskstream

    2015

    Assessment Coordinator

    • Completed and distributed
    • Revised and distributed postcard cheat sheets to faculty Spring 2016

    Create Assessment Handbook

    2016

    Assessment Coordinator and S3

    • Completed first draft of assessment handbook Fall 2016

    Increase Taskstream usage

    Continuous

    Department Chairs

    Update assessment strategic plan

    2017

    S3

    Develop assessment web presence

    2016

    VPI and AVPI

    • Completed Fall 2016

    Create assessment libguide

    2017

    AVPI



    Institutional Student Learning Outcomes

    Task Timeline Responsible Parties Status

    Remap courses being assessed as necessary

    2015 - 2016

    S3, Faculty and Chairs

    • Spring 2016 remapped all ISLOs for Fall 2016 implementation

    Re-evaluate ISLO Outcomes, criteria and assessment process

    Continuous

    S3

    • Fall 2015 S3 adopted Information Literacy VALUE Rubric for Spring 2016 implementation
    • Fall 2015 S3 adopted Critical Thinking VALUE Rubric for Fall 2016 implementation
    • Modified all ISLO score sheets

    Data collection and analysis process

    2015 - 2016

    Assessment Coordinator

    • Created PSLO reporting system for comparison of data across semesters Summer 2015
    • Changed PSLO reporting system to match the ISLO data reporting system across semesters Summer 2016

    Follow through applying longitudinal data

    2017

    All Staff

    • Trainings for inter rater reliability

    Analyze comparison of the Mathematics and Written Communication courses assessed by OJC Math and English, other OJC programs, and concurrent

    2016

    Assessment Coordinator

    IR



    Program Level Student Outcomes

    Task Timeline Responsible Parties Status

    Data analysis

    Continuously

    Assessment Coordinator

    Data Collection Process

    Continuously

    Assessment Coordinator

    Create “to do” for assessment to keep faculty thinking and focused on assessment

    2016

    Assessment Coordinator

    • Completed Fall 2016



    Program Reviews

    Task Timeline Responsible Parties Status

    Process

    Annually

    Department Chairs

    Consistency of the data

    Annually

    IR



    Co-Curricular

    Task Timeline Responsible Parties Status

    Data Collection Process

    Semester basis

    Staff



  • IV. College Assessment Team (S3)

    Assessment at OJC is guided by the Student Success Strategies (S3) committee comprised of department chairs, faculty representatives, assessment coordinator, Vice President of Instruction, Associate Vice President of Instruction, Director of Library & Learning Resources, Director of Instructional Initiatives, Associate Vice President of Enrollment Management and Vice President of Student Services. This committee adopted the ISLOs and developed rubrics and score sheets. Annually, they complete the ISLO assessment cycle, review and analyze ISLO data, oversee the assessment strategic plan, and make recommendations for assessment training.

     

  • V. College Assessment Areas

    Assessment at OJC is completed in the following areas:

    Institutional Student Learning Outcomes (ISLO) Assessment

    – This is general education assessment of: Written Communication, Mathematics, Critical Thinking, Information Literacy and Technology. Courses were mapped to determine where ISLOs are assessed.

     

    Program-Level Student Learning Outcomes (PSLO) Assessment

    – OJC has identified Career and Technical Education (CTE) programs and four academic programs: Arts and Humanities, Mathematics, Social and Behavioral Science, and Natural and Physical Science. The CTE programs are: Agri-Business Management, Agricultural Science, Automotive, Cosmetology, Early Childhood Education, Business, Nursing, Medical Laboratory Technician and Community Health Worker\Health Navigator. Each program developed outcomes and criteria and mapped outcomes and criteria to courses within the program.

     

    Co-curricular Assessment

    – OJC has 16 co-curricular program categories: Business Office, Computer Center, Physical Plant, SCORE Center, Security, Enrollment Services, Instructional Services, Learning Commons, President’s Office, Advising, Athletics, Financial Aid, Fitness Center, Housing and Dorms, International Department, and Student Life. Each program has developed outcomes and criteria.

    References:

    2015 ISLO Assessment Cycle (232 kb pdf).

    2015 Social & Behavioral Science Assessment Plan (359 kb pdf).

    Download 2015 Assessment Cycle.zip (1437 kb zip file).

     

     

  • VI. Assessment History

    The first S3 committee meeting was in November 2007. This committee was formed in response to the Higher Learning Commission accreditation visit recommendation earlier in the fall to begin assessment work across campus.  In February 2008, a group of seven attended the Assessment Academy in Chicago, which laid the foundation for implementing a general education assessment. In fall 2008, a Critical Thinking rubric was piloted with the Written Communication assessment to follow in 2010. By fall 2012, a 1/4 time assessment coordinator was hired and Career and Technical Education (CTE) program staff began developing program outcomes. In spring 2013, work began on creating outcomes and criteria for Mathematics, Technology and Information Literacy. By fall 2013, rubrics for the five Institutional Outcomes were developed, along with mapping of those outcomes across all programs. Assessment on these outcomes began shortly after development.

     

    Also in spring 2013, program outcomes were developed for Arts/Humanities, Social and Behavioral Science, Math, and Natural and Physical Science. CTE programs continued to work on and pilot program assessment plans. By January 2014, all programs were doing assessment each semester.

     

    In the summer of 2014, an assessment management tool called Taskstream was purchased; and in the fall of that year, faculty were trained in the basics of its use. Assessment data and analysis became more structured and easier to maintain. By spring 2015, co-curricular assessments were added to Taskstream. With this addition, assessment was happening across campus and beginning to be tied to budgetary planning.

     

    The S3 Committee grew from 10 participants to 15 with participation mainly from faculty and the Vice President and Associate Vice President of Instruction with co-curricular staff invited to attend on an as-needed basis. The duties of this committee were initially to steer an assessment path and have since grown to encompass aggregating ISLO data, writing the yearly plan for assessment, and making recommendations for assessment across campus.  

     

     

    November 2007

    Leadership:

    • Dr. Tom Armstrong, Vice President of Instruction, from November 1, 1999 to June 30, 2008
    • Mr. Jim Herrell, Associate Vice President of Instruction, from January 4, 2000 to January 31, 2011
    • Organized first assessment meeting with some faculty and staff.

     

    December 2007

    Discussed:

    • Syllabus revision

    ° SLO replace course objectives
    ° Assessment replace method of grading

    • CDHE – GT Pathway

     

    February 2008

    • Group of 7 attend the first assessment academy in Chicago

     

    Fall 2008

    Leadership:

    • Dr. Bob Hayden, Vice President of Instruction, from July 1, 2008 to August 13, 2010
    • Mr. Jim Herrell, Associate Vice President of Instruction, from January 4, 2000 to January 31, 2011

     

    Pilot Critical Thinking Rubric

    • S3 group and Dept. Chairs were pilot group

     

    Spring 2009

    • Full time faculty assess Critical Thinking
    • “Data collection is not the deal but looking at teaching is the deal.” Jim Herrell

     

    Fall 2009

    • Full time faculty assess Critical Thinking again

     

    Spring 2010

    • S3 looked at the English rubric they use to score their written papers
    • S3 began working on a technology rubric
    • S3 Continuing syllabus discussion
    ° Linking Outcomes on syllabus to ISLOs

     

    Fall 2010

    Leadership:

    • Mr. Jim Herrell, Interim Vice President of Instruction and Associate Vice President of Instruction, from September 1, 2010 to January 31, 2011

     

    • S3 adopted English and working on a technical writing rubric to assess writing as an ISLO.
    • Departments will decide which rubric it will use.

     

    Spring 2011

    Leadership:

    • Mr. Jim Herrell, Vice President of Instruction, from February 11, 2011 to November 30, 2012
    • Dr. David Cockrell Associate Vice President of Instruction, from June 15, 2011 to June 30, 2014

     

    • S3 discussed that the “n” was too small. Need to get more full time faculty involved. It has not been mandatory up to this point.
    • Discussing concurrent classes doing ISLOs

     

    Fall 2011

    • Continue to assess Critical Thinking and Written Communication across campus
    • Discussing math assessment with focus on rubrics.

     

    Spring 2012

    • ECE completed accreditation – S3 considering using the process as a guide
    • S3 beginning to discuss CORE content mapping
    • Initial Critical Thinking data aggregated and given to S3
    • Exit Assessment Academy

     

    Fall 2012

    • Assessment coordinator appointed
    • Group of faculty attended state assessment workshop in Pueblo
    • October - Assessment Workshop for all full time faculty. Beginning to write Outcomes for Gen ED – Written Communications, Mathematics, Technology, Reading, Critical Thinking
    • Three people attended assessment workshop in Chicago
    • Developed a 2 year assessment plan
    • Discussion about what a program is and how to assess program outcomes
    • CTE programs began to develop program student learning outcomes (PSLOs)
    • Mapping continues – gaps have been identified
    • Discussion about common rubrics for ISLOs

     

    Spring 2013

    Leadership:

    • Dr. David Cockrell, Interim Vice President of Instruction and Associate Vice President of Instruction , from January 1, 2013 to June 30, 2013

     

    • Five ISLOs are finalized – Written Communication, Mathematics, Critical Thinking, Technology and Information Literacy
    • ISLO rubric development begins

     

    March 2013

    • First draft of academic PSLOs
    • English 121 rubric adopted for all Written Communication
    • Add Critical Thinking to all ISLO rubrics/score sheets

     

    April 2013

    • Mapping for ISLOs continues across campus

     

    May 2013

    • All faculty assessment workday
    • Math PSLOs are being worked on
    • Discussion about whether or not to include Reading as an ISLO

     

    Fall 2013

    Leadership:

    • Mr. Brad Franz, Vice President of Instruction, from July 1, 2013 to April 30, 2015
    • Dr. David Cockrell, Associate Vice President of Instruction, from June 15, 2011 to June 30, 2014
    • Ms. Kim Grimsley, Associate Vice President of Instruction, from July 1, 2014 to June 30, 2015

     

    August 2013

    • Continue to work on mapping of ISLOs
    • Working to map PLSOs in each program
    • Discussion was to drop Reading as an ISLO and do Information Literacy instead

     

    September 2013

    • Decision to drop Reading as an ISLO but will assess Information Literacy  and Technology – both will have the Critical Thinking component

     

    October 2013

    • Information Literacy rubric finalized
    • Technology rubric finalized
    • Mathematics ISLO rubric still in development

     

    November 2013

    • Continue with ISLO/PSLO mapping
    • Mathematics ISLO rubric finalized
    • Mathematics PSLO outcomes and rubric are finalized.
    • Cabinet approved revised ISLOs

     

    Spring 2014

     

    January 2014

    • Begin to collect PSLO data for programs that are ready
    • Still collecting ISLO data on Written Communication and Critical Thinking

     

    March 2014

    • AA/AS/AAS will assess at least one ISLO during program.
    • Discussion of keeping artifacts – do we need to? How may? Which ones?

     

    April 2014

    • Begin researching assessment software
    • Collecting artifacts for the first time.

     

    Summer 2014

    • Purchase Taskstream and begin to build

     

    Fall 2014

    • Taskstream being used for first time
    • Decided that only ISLOs will be represented in the syllabi for now
    • First data on Spring 2014 ISLO complete
    • All rubric and score sheets for all ISLOs complete and ready to go

     

    Spring 2015

    Leadership:

    • Ms. Kim Grimsley, Interim Vice President of Instruction and Associate Vice President of Instruction, from March 1, 2015 to present

     

    • Some programs will complete Findings in Taskstream – not all ready to complete a whole cycle
    • Working to improve Taskstream

     

    Fall 2015

    Leadership:

    • Ms. Kim Grimsley, Vice President of Instruction, from July 1, 2015 to present
    • Mr. Ryan Trosper, Associate Vice President of Instruction, from July 1, 2015 to present

     

    • Implemented a full assessment cycle in Taskstream.
    • S3 adopted Information Literacy VALUE Rubric for Spring 2016 implementation
    • S3 adopted Critical Thinking VALUE Rubric for Fall 2016 implementation
    • Created PSLO reporting system for comparison of data across semesters
    • Developed and distributed “How To” guide for Taskstream

     

    Spring 2016

    • Remapped all ISLOs for Fall 2016 implementation
    • Modified all ISLO score sheets
    • Revised and distributed postcard cheat sheets to faculty

     

    Fall 2016

    • Changed PSLO reporting system to match the ISLO data reporting system across semesters
    • Separated assessment for 2016 cycle in to Spring and Fall and added syllabus repository
    • Completed first draft of assessment handbook 

     

  • VII. Frequently Asked Questions

    How is assessment completed?

    The assessment cycle at Otero Junior College runs from January to December, or from Spring Semester to Fall Semester. When the college decided to purchase Taskstream as the tool to guide assessment, it quickly became evident that it was difficult to match the budget process when data was collected during an academic year; thus the change to the calendar year. Once data was collected over several consistent semesters, it has become easier to connect assessment to budget.

     

    What is the timeline?

    2016-2017 Important Deadlines

    September 2, 2016                        Validate PSLO 2016 spring findings and submit for review

    December 9, 2016                         Fall 2016 ISLO and PSLO score sheets/artifacts in Taskstream

    February 10, 2017                         Complete 2016 findings and action plan and submit for review

    February 10, 2017                         2017 PSLO assessment plan in Taskstream and submit for review

    February 10, 2017                         Program reviews submitted to O:Drive and notify Tami

    May 5, 2017                                       Spring 2017 ISLO and PSLO score sheets/artifacts in Taskstream

     

    Concurrent Deadlines

    December 16, 2016                       Fall 2016 ISLO and PSLO score sheets/artifacts in Taskstream

    June 2, 2017                                      Spring 2017 ISLO and PSLO score sheets/artifacts in Taskstream

     

    How to enter findings into Taskstream?

    Log on to Taskstream at www.taskstream.com. The instructions for completing findings in Taskstream can be found here:

    Findings How To Submit (2142 kb pdf).

    Reviews How To Submit (2060 kb pdf).

     

    What is the Purpose of Program Reviews?

    Program reviews are currently completed on a yearly basis. Information collected for program reviews consists of semester-by-semester enrollment, completion rates and faculty information, such as professional development. Strengths and weaknesses are developed and tied to an action plan. Each program develops its plan, which is then added to Taskstream. As this process is streamlined and data is collected, a rotating system for completing program reviews will be developed.