Facebook Pixel Code

Standard I.B: Assuring Academic Quality and Institutional Effectiveness – Academic Quality

I.B.1.

The institution demonstrates a sustained, substantive and collegial dialog about student outcomes, student equity, academic quality, institutional effectiveness, and continuous improvement of student learning and achievement.

Evidence of Meeting the Standard

The College Demonstrates Sustained, Purposeful Dialogue about Student Outcomes

Mt. San Antonio College (Mt. SAC) demonstrates sustained, substantive, and collegial dialog opportunities. Collegial dialogue at Mt. SAC is founded in Board Policies (BPs) and Administrative Procedures (APs), which outline structured processes for engagement of all constituency groups with academic quality and institutional effectiveness. The College's committee structure allows for three types of committees with membership including students, faculty, California School Employee Association (CSEA) 262 & 651, Classified Senate, Academic Senate, Faculty Association, and managers/administrators as warranted. Governance committees participate in the short- and long-term global planning of the College and make recommendations about rules, procedures, direction, and processes. Operational committees participate in planning of individual departments or cross-department groups and make recommendations affecting local projects and operations. Academic Senate committees make recommendations about academic and professional matters (I.A.1-24).

At Mt. SAC, participatory governance structures are embraced as part of the culture of the College and support campus wide dialogue and integrated planning. Every year, the College committees discuss issues related to their purpose and function such as equity, outcomes, academic rigor, and effectiveness. Committee representatives actively seek input from their constituency groups and use formal processes to report updates regularly (I.B.1-1 pg. 3). For example, the Academic Senate requires a report-out at its meetings regarding each Academic Senate committee’s work. For CSEA 262, those who sit on committees are asked to provide narrative updates about the committees’ work for the members to review. Special events are hosted by committees or groups on campus to allow for dialog on focused issues (I.B.1-2, I.B.1-3, I.B.1-4, I.B.1-5). Campus wide emails are also used to provide opportunities for broader input on committees’ issues (I.B.1-6).

The work of the Mt. SAC Outcomes Committee most clearly demonstrates the integration of committee dialogue in a sustained cycle of improvement. The Outcomes Committee plans for and implements continuous improvements to the outcomes process; however, this is not done unilaterally. The committee works with the Curriculum and Instruction Council (C&I) and the Academic Senate to make sure its work is validated by these external bodies before College wide process changes are enacted. The Outcomes Committee reviews all courses while they are in the four-year, course review cycle and evaluates their outcomes. Results from this work are sent to the C&I for its consideration as it reviews other aspects of the course. Results are also communicated with the affected department chair to be further communicated with faculty. Noncredit faculty do not have department chairs, but there is faculty representation on the Outcomes Committee, so faculty representatives along with Continuing Education Division managers communicate with noncredit faculty on the status of outcomes. The feedback provides faculty with suggested ways to improve upon their assessment work as well as validates work that is exceptional. This cycle of dialogue is foundational to improvement of student learning (I.B.1-7).

At Mt. SAC, measurable objectives (MOs) and student learning outcomes (SLOs) stand as the criteria for success in a course. MOs are all of the outcomes that a student will meet, or learn, in a given course. SLOs may be derived from MOs in order to achieve more specific assessment and analysis (I.B.1-8). A department determines the SLOs through a discussion about what it wants to focus on for strengthening student success. While the content of an SLO is dependent on the substance of the course, program or instructional unit, five primary traits will always be present: (1) alignment, (2) central, (3) feasible, (4) meaningful, and (5) measurable. Once a department has determined its SLOs, a rubric is created and criteria for success is established. The SLOs are then assessed on a regular cycle. The results of the assessments are collected and discussed at a department meeting, and the discussion leads to using the results of the assessment for improvement. The use of results might lead to a reassessment of the current SLOs after a determined pedagogical or curricular change, or it might lead to choosing a different SLO to assess in the next cycle. The end result of each cycle of this process is indicative of the College culture that focuses on improvement in teaching and learning. Even though the content of courses and programs is relatively static, course and program outcomes may be fluid and revisable. Career Technical Education (CTE) advisory committees help curriculum and SLOs stay current/aligned with industry standards.

A program level outcome (PLO) is a statement about what a student will think, know, or be able to do upon completion of an instructional program. A PLO can also describe the overall expected measurable results of a project or program being implemented in areas of service that directly or indirectly influence student success. PLOs are designed and developed by the stakeholders of the course, program, or institutional unit; the assessment of PLOs within the instructional areas illuminate the competencies students attain as a result of a program.

The College's institutional level outcomes (ILO) represent the ongoing evolution of continuous improvement at Mt. SAC. The College transitioned from general education outcomes to institutional level outcomes (ILO), based on the commitment that all Mt. SAC employees contribute to the success of our students by empowering their educational experiences with core competencies. ILOs are statements about the knowledge, skills, abilities, and attitudes that students are expected to develop as a result of their overall experiences with any aspect of the College, including courses, programs, departments, and services (I.B.1-9, I.B.1-10).

The four ILOs are defined as noted below. 

Institutional Level Outcomes – Everyone’s Included!

Institutional Level Outcomes – Everyone’s Included!
Long Description
Figure I.B.1.-1. Institutional Level Outcomes – Everyone’s Included!

Mt. SAC’s course and program outcomes are mapped to ILOs. Through this process, PLO statements are tracked to courses in the program in which the PLO is introduced, achieved, and/or mastered. Data from these course SLOs are compiled to assess student achievement of PLOs. Collegial dialogue about ILOs was facilitated through professional development opportunities designed by Outcomes Committee members to help faculty map courses to programs. Using division/department meetings as a forum, Outcomes Committee members led faculty in a shared outcomes mapping experience and provided common language for deeper discussions (I.B.1-11, I.B.1-12, I.B.1-13, I.B.1-14). The Outcomes Committee Annual Report demonstrates the broad scope of continued improvements being made across the College and the impact of those improvements on students’ learning (I.B.1-15, I.B.1-16, I.B.1-17).

Outcomes dialogue is not limited to the Outcomes Committee, but it is a regular part of planning efforts. For example, Student Services programs have mapped the ILOs to their programs, with the majority aligned with three of the four ILOs: (1) critical thinking, (2) personal, social, civic, and environmental level outcomes, and 3) communication. Student Services retreats provide a forum for ongoing outcomes assessment discussions focused on the broad range of courses, instructional interventions, learning opportunities, and support services that serve to develop students’ overall educational preparation (I.B.1-18).

Adjunct faculty have been an important part of in-depth discussions of outcomes assessment and alignment. The Outcomes Committee, with support from administration, created a process for departments to apply for adjunct faculty to be compensated for up to two hours to participate in a meaningful way in outcomes conversations at department meetings, division meetings, as well as retreats. Outcomes assessment discussion is a regular part of Department meetings and the outcomes assessment process. Funding for adjunct faculty participation supports and augments this important work. In 2015-16, requests for adjunct faculty compensation were approved for departments such as Adult Basic Education, Education for Older Adults, Aircraft Maintenance, Animation, Communications, Psychology, and Sociology. (I.B.1-19, I.B.1-20, I.B.1-21, I.B.1-22, I.B.1-23, I.B.1-17).

Through department and division dialogue across the Instruction Division, outcomes mapping of degrees and certificates to ILOs has been completed.  A summary report indicated that critical thinking was the most common ILO mapped to degrees and certificates (93 percent) followed by communications (85 percent), information and technology literacy (79 percent), and personal, social, civic, and environmental (65 percent) (I.B.1-24).

Dialogue about student outcomes is not static, but is re-examined through continuous improvement processes. For example, the Mt. SAC School of Continuing Education (SCE) provides adult education services and holds a postsecondary school accreditation with the Accrediting Commission for Schools, Western Association of Schools and Colleges (ACS WASC). As part of the ACS WASC Self-Study process Continuing Education faculty, staff, and students collaboratively developed Student Learning Goals (SLGs) in 2009. SLGs are interdisciplinary statements specifying what all students should know, understand, and be able to do by the time they complete their planned program. They provided the foundation for a comprehensive assessment of all programs across the School of Continuing Education. The SLGs developed in 2009 were: effective communication, critical thinking, and lifelong learning. In spring 2016, the School of Continuing Education Advisory Committee reviewed the SLGs as part of an annual retreat, discussed their alignment with ILOs, and recommended replacement of the SLGs with the College ILOs. The process for assessing is the same and will allow for greater integration and alignment of SCE with the College assessment processes (I.B.1-25).

The President has created a culture of continuous dialogue by leading through example. Each week he sends Cabinet Notes to the College with an executive summary of highlights as well as a link to detailed minutes found on his webpage. The Cabinet Notes reflect the many voices of the College, with constituency representatives reporting to Cabinet and participating in discussions on academic quality and institutional effectiveness. The President calls regular town hall meetings to provide a forum for solicitation of input on College decisions from all members of the College community. For those who may be reticent to share their insights in a public forum, the President’s monthly open office hours provide an opportunity for one-on-one dialogue with the President on any topic, with no appointment necessary. Finally, the President has provided strong support of convocation for both faculty and classified staff. He leads the opening meetings of convocation to ensure dialogue opportunities on student learning and achievement initiatives such as the Guided Pathways Project. During the opening meeting, the President also publicly recognizes excellence in outcomes assessment work. Through a yearly nomination and selection process done in partnership with the Outcomes Committee, faculty are chosen to receive a Presidential Award for Excellence and Innovation in Teaching and Learning. Following the opening session, breakout workshops provide opportunities for continuing this conversation on topics such as, Outcomes Assessment for All: Best Practices to Involve All Faculty in Completing the Outcomes Cycle (I.B.1-26, I.B.1-27). Weekly President’s reports to the Board of Trustees ensure that collegial dialogue includes the highest level of Mt. SAC leadership. This dialogue continues at Board meetings where information reports and discussion items provide opportunities for the Board to discuss student programs, achievement, and equity. The President’s transparency in communications provides a venue for employees to be engaged with continuous dialogue on College initiatives (I.B.1-28, I.B.1-29).

The College Demonstrates Sustained, Purposeful Dialogue about Student Equity

Mt. SAC has been a leader in an open analysis and dialogue related to student equity for almost 20 years. In 2005, the College produced an Equity for All report as part of a special project through the University of Southern California (USC) School for Urban Partnerships. The results of this extensive work served to direct The College's development of interventions, services, and continued research to improve student success. In 2012, a cross-section of faculty, managers, staff, and students developed a Student Success Plan (prior to the initiation of the state’s Student Success and Support Program efforts). This plan continued the work of the Equity for All project by focusing on three main areas: (1) student engagement and persistence, (2) development of students’ own goals, and (3) progression through basic skills course sequences. The College held several campus wide dialogue sessions related to student success and student equity, leading up to the development of The College's state-required Student Equity Plan in 2014. For the 2014 Student Equity Plan, three subgroups (Futures Team, Pathways Team, and Interventions Team) worked closely with the Student Equity Committee and Research and Institutional Effectiveness (RIE) to develop strategies and actions based on critical data related to disproportionality in student success.

In 2015, based on the amended plan template for student equity, College administrators, lead program personnel, and faculty department chairs were provided opportunities to submit project/activity proposals to address inequities in student success in their particular areas. Project teams attended meetings to discuss their proposed projects and further refine how to address equity gaps. The 2015-16 Student Equity Plan reflects institution wide planning and collaboration to develop interventions to address disproportionality and inequities in student success, especially for specifically defined student groups. Project activities range from transfer interventions focusing on science education to specialized interventions for foster youth students to the creation of a Science, Technology, Engineering, and Math (STEM) Center to increase the success rates of African-American and Latino students in math and science courses (I.A.1-12). The College institutes programs and interventions to address concerns related to specific student populations such as veterans, low-income, first-generation, Dreamers, and minority males. The faculty, through Academic Senate leadership, has hosted keynote speakers and featured workshops related to student equity issues such as cultural competency in the classroom. All activities in the Student Equity Plan are evaluated on an ongoing basis to measure the impact of each College effort in reducing disproportionality and lack of progress for defined student populations.

The College Demonstrates Sustained, Purposeful Dialogue about Academic Quality

Mt. SAC uses dialogue to lead to a collective understanding of the meaning of evidence, data, and research used in evaluation of student learning. Student learning outcomes assessment work is a key component of the job descriptions for many of The College's researchers. Educational research assessment analysts are trained to help faculty in all aspects of measuring, analyzing, and interpreting learning outcomes. The Basic Skills Coordinating Committee, of which a researcher is a standing member, funds research projects that seek to increase successful basic skills progression for credit and noncredit students. Part of the funding process requires the creation of SLOs and student outcome reporting that relate to committee goals. Project leaders work with researchers to create these assessments and evaluate the efficacy of the project’s impact on basic skills student success. Yearly basic skills reports provide the evidence to support multiple interventions and demonstrate the data is used for action (I.B.1-30 pg. 5-8, 12-16).

The Outcomes Committee conducts an annual evaluation of the quality of learning outcomes work. This work demonstrates the immense growth over time of the faculty members’ creation and use of outcomes assessment. Early efforts demonstrated a beginning understanding of and use of outcomes assessment. Today’s outcomes demonstrate a clearer, more in-depth understanding of the impact of outcomes assessment on students’ learning (I.B.1-31).

Numerous governance committees at Mt. SAC provide opportunities for dialog about academic quality and continuous improvement of student learning and achievement. The Educational Design Committee (EDC) and the Curriculum and Instruction Council review, approve, and administer courses, programs, certificates, and degrees; and they regularly evaluate the processes, improve the processes, and ensure the processes are sustainable (I.B.1-32, I.B.1-33).

Dialog is also maintained during mandatory training for new members of the EDC. Training is done to outline curriculum policies and procedures. Deans and department chairs also attend these trainings. Training focuses on the California Community College Chancellor’s Office requirements as well as Mt. SAC’s local standards of operation, such as the use of WebCMS, The College's local curriculum management system, for the credit and noncredit curriculum approval process. The Curriculum and Instruction Council reviews higher-level issues as part of its ongoing improvement process, such as changes to unit value of courses and stand-alone courses. Students may view the learning outcomes for any course in WebCMS via the built-in link on the Course Outline of Record to the outcomes webpage (I.B.1-34).

The Academic Senate provides open opportunities for discussion during its regularly scheduled meetings. For example, in April 2015, the Academic Senate allowed for open discussion with the Outcomes Committee members regarding its proposal to expand upon general education outcomes (general education course focus) to become institutional outcomes (campus wide focus) (I.B.1-35 pg. 7).

The College Demonstrates Sustained, Purposeful Dialogue about Institutional Effectiveness

There is a continual sense of urgency for Mt. SAC to effectively serve students and the community. What programs are being offered that students need; what new programs should be offered; what programs need to be re-examined? The College stays abreast of national research, trends in the government, and recent higher education research to inform dialogue about institutional effectiveness. Mt. SAC’s annual program review process, Planning for Institutional Effectiveness (PIE), directly impacts institutional effectiveness, outcomes assessment, and student achievement. PIE requires all units/departments to evaluate their accomplishments, examine the impact of internal and external forces on their work, evaluate their effectiveness using data, think critically about outcomes assessment and its impact on student learning, as well as to plan improvement strategies for student learning and support services.

For example, based on dialogue from the Paralegal Program Advisory Committee, the College determined that students needed to be better prepared to conduct online research to compete in the marketplace. At the time of the initial discussion, fewer than 80 students had access to such resources and training. Through a PIE resource request process, all paralegal students obtained licenses for the online research tool Westlaw. In addition, access to Lexis was granted to students in Paralegal 318. Access to these two resources prompted faculty to institute an SLO for all students in the paralegal class to earn official Lexis and Westlaw certification. Because of the importance of these certifications in the legal environment, these were made PLOs for the paralegal program as well. Based on SLO data, in an effort to improve paralegal student proficiency in Westlaw and Lexis, the program now has official Lexis and Westlaw educators conduct the training for the students in the PLGL 31B classes (The Lexis educators come to campus; the Westlaw training is done via conference call/online live in the classroom). Further, the paralegal faculty members took steps to improve their own Westlaw and Lexis skills in an effort to better assist students. The number of certifications in Westlaw and Lexis are documented through TracDat (I.B.1-36 pg. 2, 4, I.B.1-37, I.B.1-38).

Figure I.B.1.-2. Planning for Institutional Effectiveness
Long Description
Figure I.B.1.-2. Planning for Institutional Effectiveness

Administration supports the innovation needs of the College. Managers summarize their unit/departments’ PIEs and provide input back to the units/departments about how well it represents their work; if they made a good case for why and how they need to improve upon their work based on assessment; and how likely they are to obtain what they need regarding resource allocation. Managers also record progress toward strategic objectives in the Strategic Plan which they have responsibility to address through the PIE process. Vice Presidents summarize their managers’ PIEs, highlight projects, and provide a prioritized list of resource requests linked to program improvement strategies. These reports are used to provide the impetus for and evidence of the need for new and continuing resources. This information is also used to advocate for improvement in integration and planning processes.

Vice Presidents’ and President’s PIE reports provide a plethora of information on the activities of the College as well as its strengths, weaknesses, opportunities, and threats. As noted in the College Strategic Plan, sections of the PIE document allow all areas of the campus to reflect on the internal and external conditions that impact their work (e.g., accreditation, Title 5). These factors are used along with the other components of PIE, such as critical decisions and resource planning, to provide a College wide perspective that is used to form recommendations for action. Each year, the Institutional Effectiveness Committee (IEC) evaluates all Vice Presidents’ and President’s PIEs, the College goals, and the effectiveness of the PIE process, and creates the institutional PIE summary for use by the College in resource allocation and general College operations decisions (I.B.1-39). The annual institutional PIE summary report for the College is presented to the President’s Advisory Council (PAC), a governance committee. In accordance with the adopted College procedure described in Administrative Procedure (AP) 3250, PAC reviews the recommendations for changes to The College's goals and PIE process (I.B.1-40, I.B.1-41). Changes to College’s goals are sent to the Board of Trustees for their review and consideration for approval (I.A.4-6 pg. 11-12).

The impact of dialogue on changes in the PIE process was evident when the IEC and the Accreditation Steering Committee evaluated The College's Institution-set Standards (ISS) and short-term and long-term goals for its Institutional Effectiveness Partnership Initiative (IEPI). IEPI indicators were also taken to the Academic/Student Services Planning Summit for feedback on what the goals should be. Committee and summit discussions about student learning and achievement resulted in standards being set, monitored, and added to PIE in 2015-16. Including ISS in PIE supports broader discussions at the unit/department level, with a deeper understanding of the standards and documentation of how the work of units/departments is contributing toward the College achieving the standards. The final recommendations for each metric are carried by IEC to the PAC for review and approval. IEPI indicators are then presented to and approved by the Board of Trustees.

In 2015-16, the College was chosen as one of 30 across the nation to be part of the prestigious American Association of Community College’s Pathways project. Data from this project revealed, among many things, that students were taking extra units before they achieved their degree or certificate. This is unacceptable given that students are given lower priority registration after exceeding the threshold for a high number of units (I.B.1-42). Overall, the ISS and IEPI data along with the AACC Pathways project data provided the College with data to support its new and ongoing activities, including continued growth of directive course-by-course sequencing of programs, student equity projects, and basic skills work.

The College Demonstrates Sustained, Purposeful Dialogue about Continuous Improvement of Student Learning and Achievement

The College offers many opportunities for sustained, purposeful dialogue about continuous improvement of student learning and achievement. For example, many faculty and administrators are involved in Mt. SAC’s dialogue about the continuous improvement of student learning through distance learning mode and how student success measures compare with student learning in traditional programs. The Distance Learning Committee (DLC) provided faculty with two pages of tips and resources for best practices in online education. Professional development training in best practices, the use of Moodlerooms (Mt. SAC’s learning management system), video creation, optimization of graphics, eLearning, and online pedagogy (Skills and Pedagogy for Online Teaching – (SPOT)) are ongoing to support faculty.

There is a two-step process required to use Moodlerooms for teaching online: (1) Completion of Moodlerooms Basic Training (MR1 & MR2, or the MR Basics Online Workshop), and (2) completion of the Skills and Pedagogy for Online Teaching (SPOT) certification process. The Online Learning Support Center (OLSC) is a centralized place for faculty to learn how to use Moodlerooms, receive support in SPOT certification, and learn about updates to Moodlerooms. The DLC identifies best practices and shares them with faculty as they go through the learning process, and distance learning faculty coordinators ensure that assigned faculty are certified to teach online via The College's process. The DLC provides guidance and training opportunities for faculty to ensure student success in distance learning courses. The DLC establishes a level of quality modality and best practices for asynchronous mode for online education. There is a process for ongoing dialogue about the certification process, efficacy of the technology and learning processes, and evaluation of the distance learning endeavor (I.B.1-43). To change the curriculum from traditional in-class to distance learning, the course must be vetted via the DLC as well as the Curriculum and Instruction Council. The same SLOs are used for a course regardless of the course being taught in a face-to-face or distance learning mode. The number of distance learning courses offered is evaluated on an annual basis; data for success and retention are evaluated annually for the same course offered in different modalities (I.B.1-44). Faculty may only teach up to 50 percent of their unit load online.

Dialogue about continuous improvement of student learning and achievement is also precipitated by analysis of student success on examination and licensure exams. Some Career Technical Education programs at Mt. SAC have external licensure examination requirements of their graduates before they may begin working in their field. Exceptionally high pass rates demonstrate high academic quality. For example, the nursing program has over a 90 percent pass rate (I.B.1-45, I.B.1-46). One factor impacting the pass rate is the partnership between the Adult Basic Education Department in the School of Continuing Education and the nursing program (I.B.1-47, I.B.1-48). Prior to this partnership, many students in the nursing program were having difficulty with the mathematics of titration and dosages. The noncredit Adult Basic Education program and credit faculty developed boot camps and a noncredit course with contextualized basic math to teach nurses the skill set needed to obtain their licenses (I.B.1-49).

Several Mt. SAC initiatives clearly demonstrate how well groups at the College collectively understand the meaning of evidence, data, and research use in evaluation of student learning. The 2015-16 Student Equity Plan Report and Basic Skills Annual Reports both represent cross-College collaboration that involves numerous employees conducting research to both propose and evaluate different services and interventions to improve student learning (I.B.1-30 pg. 5-8, 12-16, I.B.1-50, I.B.1-51, I.B.1-52, I.B.1-53, I.B.1-54).

The Honors College developed from the Honors program in 2015 and uses a cohort system to allow students to complete their major preparation, receive Honors Scholar designation, and transfer within two years to a four-year school of higher education. In 2015, the Pathways to Transfer program expanded based on pilot data that indicated vast improvements in student achievement for students in the Pathway versus those who were not in the Pathway. This project will allow for a synthesis of ideas related to cohort pathways for students (I.B.1-55 pg. 8).

Mt. SAC’s high standing with the Accrediting Commission also indicates high standards for its academic quality and improvement processes. Many of The College's students transfer to four-year institutions of higher education indicating that the College has sound articulation agreements (I.B.1-56). The Adult Basic Education Department (ABE) has also raised academic standards for high school coursework above what is expected. For the past several years, ABE’s high school courses have maintained the a-g designation through the University of California Regents Office. As a result, noncredit students taking high school courses can apply them to satisfy a-g requirements for entrance into California State University and University of California institutions (I.B.1-57).

Dialogue about institutional effectiveness is the work of all constituency groups. Outcomes assessment includes all work that is done across campus in every unit/department. Using this model, the continuous improvement of student learning is the responsibility of all employees. The College uses the term institutional level outcomes to denote this broader perspective; this aligns well with work across the nation on making general education the responsibility of all members of the higher education institution. As part of the PIE process, there are many data sources used by the units/departments to advise the College of its work, its effectiveness, and the opportunities for improvement (e.g., course outcomes, program outcomes, institutional outcomes and student achievement data). PIE is used to discuss the data/information trends that are relevant and to reflect on its meaning as the unit/department plans for the future of the area. IEC reviewed (Spring 2016) these sources and is working with Information Technology to provide departments with more user-friendly data (I.B.1-58, I.B.1-31, I.B.1-59, I.B.1-60).

Cross-team dialogue about data occurs in many meetings. This level of collaboration and dialogue via events such as the Institutional Research Day and Academic/Student Services Master Planning Summits needs to be continuous (I.B.1-2). Mt. SAC employees understand the role that they play in students’ success. Facility employees are aware that their work at creating excellent grounds for the College and its students contributes toward students’ success. They play a part in creating an environment to help students want to be here and to be successful. Classified employees are critical to student success and have contributed their voices to this Self Evaluation and the accreditation process. Dialogue Days offered in spring 2016 invited the campus community to attend one of six, 90-minute sessions to provide their feedback to the Self Evaluation. Special visits were made to California School Employee Association (CSEA) 651 and 262 meetings to engage them in conversations about accreditation, and a special session for the evening CSEA 651 employees was organized in spring 2016 for a similar purpose. The Accreditation Steering Committee, a shared governance committee, provides for open dialogue about, learning of, and evaluation of The College's accreditation endeavors (I.B.1-61).

Analysis and Evaluation

Mt. San Antonio College has structures and opportunities for dialogue that provide each constituent group on campus ongoing opportunities for providing input to all College processes and decisions. This dialogue occurs at all levels of responsibility. College structures support this ongoing dialogue to ensure that it is sustained and substantial. Evidence demonstrates that dialogue includes in-depth discussion of topics.

Accomplishments and Outcomes

  • Mt. SAC demonstrates sustained, substantive, and collegial dialog opportunities.
  • Mt. SAC embraces participatory governance structures as part of the culture of the College and supports campus wide dialogue and integrated planning.
  • Mt. SAC has been a leader in an open analysis and dialogue related to student equity for almost 20 years.
  • Mt. SAC uses dialogue to lead to a collective understanding of the meaning of evidence, data, and research used in evaluation of student learning.
  • There is a continual sense of urgency for Mt. SAC to effectively serve students and the community.

List of Evidence

Title Content
I.A.1-12 Student Equity Plan 2015-16
I.A.1-24 Campus Committees
I.A.4-6 Board of Trustees Minutes 2-17-2016 pg. 11-12
I.B.1-1 President’s Advisory Committee Minutes 9-9-2015 pg. 3
I.B.1-2 Academic Master Plan Meeting Agenda 2016
I.B.1-3 Town Hall Meeting November 2014
I.B.1-4 Expanded Presidents Advisory Council Minutes11-18-2015
I.B.1-5 Professional Development Day Spring 2014
I.B.1-6 Cabinet Notes Email 9-8-2016
I.B.1-7 Outcomes Committee Minutes 5-31-2016
I.B.1-8 How to Write an SLO Web Page
I.B.1-9 GEO Reimagined Report
I.B.1-10 Institutional Level Outcomes Defined
I.B.1-11 ILO Mapping
I.B.1-12 Outcomes Mapping Example - Athletic Trainer Certificate
I.B.1-13 Outcomes Mapping Example - Communication
I.B.1-14 Outcomes Mapping Example - GDI Certificate
I.B.1-15 Outcomes Committee Annual Report 2014-15
I.B.1-16 Outcomes Committee Annual Report 2015-16
I.B.1-17 Institutional Level Outcomes Reports Web Page
I.B.1-18 Institutional Level Outcomes Mapping - Student Services
I.B.1-19 Older Adult Program SLO Meeting 1-20-2012
I.B.1-20 Sociology Philosophy SLO/GEO Retreat 4-26-2013
I.B.1-21 Older Adult Program TracDat Results 2015-16
I.B.1-22 Department Meetings Rotation - Music
I.B.1-23 Continuing Ed Advisory Minutes
I.B.1-24 Summary Report for ILO Mapping August 2016
I.B.1-25 SCE Advisory Notes 4-22-2016
I.B.1-26 Faculty FLEX Day 2016
I.B.1-27 FLEX Day Opening Session Fall 2014
I.B.1-28 President's Cabinet Action Notes Web Page
I.B.1-29 President's Board Reports Web Page
I.B.1-30 Basic Skills Annual Report 2014-15 pg. 5-8, 12-16
I.B.1-31 Qualitative Review of Outcomes Report 2015
I.B.1-32 Curriculum and Instruction Council Purpose, Function, and Members
I.B.1-33 Educational Design Committee Purpose, Function, and Members
I.B.1-34 Student Learning Outcomes Web Page
I.B.1-35 Academic Senate Meeting Minutes 4-9-2015 pg. 7
I.B.1-36 Paralegal Advisory Minutes Fall 2015 pg. 24
I.B.1-37 Business Admin PIE - Paralegal
I.B.1-38 Business Admin TracDat - Paralegal
I.B.1-39 Business Div Manager PIE - Resource Allocation
I.B.1-40 AP 3250 Institutional Planning
I.B.1-41 PAC Minutes 3-25-2015
I.B.1-42 Mt. SAC AACC Pathways Web Page
I.B.1-43 SPOT Web Page
I.B.1-44 Success and Retention in Distance Learning Courses
I.B.1-45 Licensure Pass Rate
I.B.1-46 Table of Licensure Pass Rates
I.B.1-47 Nursing Math Grades Fall 2015
I.B.1-48 Nursing Bootcamp
I.B.1-49 Transitional Math Email
I.B.1-50 Basic Skills Action Plan and Report to the Chancellor’s Office 2015-16
I.B.1-51 Basic Skills Coordinating Committee Purpose, Function, and Members
I.B.1-52 Student Equity Committee Purpose, Function, and Members
I.B.1-53 Student Equity Intervention Team
I.B.1-54 Student Equity Committee Minutes 11-23-2015
I.B.1-55 Title V Newsletter Fall 2015 pg. 8
I.B.1-56 Articulation Agreements
I.B.1-57 A-G Status List
I.B.1-58 PIE Summary College-wide 2014-2015
I.B.1-59 Outcomes Committee Report Web Page
I.B.1-60 Outcomes Committee Resources
I.B.1-61 CSEA 651 Dialogue Days Meeting

I.B.2.

The institution defines and assesses student learning outcomes for all instructional programs and student and learning support services. (ER 11)

Evidence of Meeting the Standard

The College Defines Student Learning Outcomes

Mt. San Antonio College (Mt. SAC) defines and assesses student learning outcomes (SLOs) for all instructional programs and student learning and support services. Definitions and assessment results for SLOs, program level outcomes (PLOs), and institutional level outcomes (ILOs) are maintained in TracDat (I.B.2-1). Mt. SAC’s Administrative Procedures (APs) and Board Policies (BPs) guide outcomes development and evaluation of courses, programs, certificates, and degrees. For all of The College's courses and programs (degrees/certificates), outcomes are defined, assessed, and the results used for improvement. For credit courses and programs, SLOs represent what a student knows, thinks, feels, or can do as a result of an intervention. The Outcomes Committee works with the Curriculum and Instruction Council (C&I) to ensure this process is maintained. Credit and noncredit faculty are represented on the Outcomes Committee to facilitate communication across campus. As part of the regular assessment process, the Outcomes Committee reviews the outcomes on all courses going through the four-year course review cycle. Those courses that have not been assessed within a four-to five-year period are noted. Per the Academic Senate’s recommendation and per AP 4020 Program and Curriculum Development, the department is given one additional year to assess course outcomes, and, if not accomplished, the course is deactivated (I.B.2-2, I.B.2-3, I.B.2-4, I.B.2-5).

Faculty at Mt. SAC have a clear leadership role in instructional programs and outcomes definition and assessment. Faculty are responsible for defining and assessing student learning outcomes for courses and programs (e.g., degrees/certificates) as well as institutional outcomes (I.B.2-6). The Outcomes Plan outlines the role faculty play in assessment (I.B.2-7).

C&I and the Educational Design Committee (EDC) review, approve and administer courses, programs, certificates, and degrees; and they evaluate the processes, improve the processes, and ensure the processes are sustainable (I.B.1-32, I.B.1-33). The development of the Unit/department Planning for Institutional Effectiveness (PIE) (program review) provides for discussion and review of instructional programs (degrees/certificates) and includes all SLO assessment from definition to evaluation and cyclical improvement. Programs that cross more than one discipline are reviewed by a taskforce when courses are being added.

Student Services programs and departments also participate in the development and assessment of student learning outcomes. All departments focus on interventions and activities that serve to improve students’ knowledge about College processes and requirements, strategies for student success, and information related to their health and development. Assessment methods and analyses of results occur on an annual basis, which serves to measure program and department progress in meeting students’ needs and effectively providing instruction and support services.

The College Assesses Student Learning Outcomes

Mt. SAC assesses the quality and improvement of outcomes assessment. The Outcomes Committee annually reviews the quality of faculty members’ work on outcomes assessment. The Outcomes Committee uses a rubric to clearly articulate what makes for excellent outcomes assessment. Chairs receive these evaluations and share them with their faculty to provide a collegial way to improve upon the quality of assessment. During this initial review process, the Outcomes Committee noted that outcomes assessment clearly improved when examining the year-over-year assessment being done within one course or program (I.B.2-8, I.B.1-31). The School of Continuing Education uses a process of outcomes review for its courses to ensure the quality is present and improved upon. The English as a Second Language (ESL) Outcomes Team utilizes a data-driven process to design and analyze use of results for program improvement. Specifically, each year, faculty select one of the three student learning goals, which are now ILOs (effective communication, critical thinking, and lifelong learning), to ensure they are integrated throughout the seven ESL levels of the program. Faculty select one goal each year to measure, and results are used to improve curriculum or make decisions regarding effective material and resources (I.B.2-9, I.B.2-10).

Multiple outcomes assessment work is being carried out across campus. As noted above, faculty are reviewing outcomes at the course, program, and institutional level. Outcomes are recorded in TracDat, The College's central repository. It is from here that information is refreshed onto the public webpage for all to read (I.B.1-34, I.B.2-6). In some cases, SLO assessment results indicate that the criteria for success has not been met. Faculty reflection and discussion leads to use of the results for improvement such as a reassessment of the current SLOs after a determined pedagogical or curricular change, or it might lead to an activity or resource request reflected in the unit PIE. An example of this process is found in the SLO assessment of Physics 4A in which students experimentally measure the movement of a falling object. Students had historically been successful in meeting the criteria for this SLO, but an increase in course sections resulted in a decrease in student attainment of this essential skill. Faculty discussions determined that shared equipment among classrooms limited skill development. A unit PIE request for a separate set of equipment for use in each room to adequately support classes and maintain academic integrity and excellence was created (I.B.2-11, I.B.2-12, I.B.2-13). The end result of each cycle of this process is indicative of the College culture that focuses on improvement in teaching and learning.

Through The College's four-year course review process, the Outcomes Committee assesses the quality of the outcomes, provides feedback to department chairs on this work, and encourages in-depth dialogue with faculty (I.B.2-3).

Initially, program level outcomes (PLOs) were created and mapped to courses. In 2013, an Academic Senate resolution was passed to provide assessment training for program level outcomes. A program outcome reporting resolution was also passed. Since that time, regular progress has been made to assessment program outcomes. Regular status reports are provided to the Deans on progress; the Deans share this information with their chairs (I.B.2-14).

The College Assesses ILOs and Mapping

ILOs represent the core competencies students develop as a result of an educational experience at the College. Instructional programs map course-based student learning outcomes (SLOs) with program level outcomes (PLOs) and ILOs. Faculty across all instructional programs mapped outcomes for their course, degrees, and certificates with ILOs. If an instructional area does not offer degrees and certificates, it maps courses or course sequences with ILOs. As an example, the American Language Department (AMLA) identified and mapped broad department-level outcomes for reading and writing, which students practice across multiple AMLA courses (I.B.2-15). Importantly, student and learning support areas also mapped administrative unit objectives (AUOs) and SLOs with ILOs. For instance, the Library was able to map course and service outcomes to department-level outcomes and College wide ILOs. Mapping indicates where in the programs or services learning occurs and competencies are assessed. Mapping also identifies the level to which learning outcomes are being met, from introduction, to practice/application, to mastery. The College's mapping process encourages faculty, staff, and managers to think broadly and deeply about their programs, courses, and services.

Mt. SAC’s School of Continuing Education defines and assesses SLOs in its service areas for student support services and programs. Student learning goals were a global type of SLO and have been renamed ILOs to be consistent with campus terminology. They are interdisciplinary statements about what all students should know, understand, and be able to do by the time they complete their planned program. ILOs provide the foundation for a comprehensive assessment of all programs across the Continuing Education Division: Continuing Education Division will prepare students to be effective communicators, critical thinkers, and lifelong learners. For example, starting in September 2013, the Adult Basic Education Department (ABE) provided an opportunity for its employees to showcase the outcomes assessment work they accomplished as part of their program review (PIE). PIE Day was so successful that it was repeated in fall 2014, 2015 and is planned for 2016. Adult Basic Education was also awarded the President’s award for Excellence and Innovation in Teaching and Learning based on its efforts with outcomes, including PIE Day. It is important to note that it was the staff that were given the authority to create such an engaging session, and it was they who presented their outcomes assessment work (I.B.2-3, I.B.2-6, I.B.2-16, I.B.2-17).

Mt. SAC defines ILOs as the core competencies students should develop as a result of their educational experiences at the College. ILOs were developed with the belief that all areas of the College contribute to the students’ educational experiences, including services, departments, courses, and programs. Instructional and student and learning support areas establish overarching goals and outcomes for their programs and services in alignment with The College's ILOs. They also consider how students will meet the broader ILOs and PLOs through practice and mastery of unit-level outcomes, including AUOs, SLOs, and, course measurable objectives. Results of assessment, such as strengths and gaps in curriculum, instruction, and services are reported on as part as The College's Planning for Institutional Effectiveness (PIE) process by campus units, divisions, and teams. Requests for new resources, such as equipment, technology, and personnel, are also made through PIE based on outcomes assessment. Although it is but one component of the PIE process, outcomes assessment and alignment is persistent across many levels of planning. Comprehensive attention is given to planning and improvement of curriculum, instruction, and services.

Mapping SLOs to PLOs to ILOs provided a new opportunity to review outcomes. PLOs are assessed via a program mapping template process. Programs are being offered direct assistance to help with this work. Both the faculty accreditation coordinator and the outcomes coordinator attended department meetings to assist them in this process.

The College transitioned these maps for the programs to TracDat 5 which helped minimize the workload for faculty as well as provided an opportunity for higher-level discussions on course-to-program SLOs. The roll-out of TracDat version 5 on April 1, 2016, included two versions of targeted training classes developed collaboratively by the Outcomes Committee and Information Technology (IT) and were offered via Professional and Organizational Development, a PowerPoint presentation on outcomes assessment, a user guide with screen shots and helpful hints, and campus wide announcements detailing the system upgrade. The training class, TracDat 5 for New Users, was offered seven times and included an overview of the outcomes and assessment process at Mt. SAC. The training class, TracDat 5 for Seasoned Users, was offered six times over the course of two months and focused on the mechanics of the software. Additionally, the outcomes coordinator scheduled department or division specific training sessions, when requested. Both training classes have been well received, with attendees commenting that they were pleased with the new TracDat 5 interface. Questions and discussions generated during the training sessions centered around how departments choose to assess their outcomes, mechanisms for completing the use of results that focused on closing the loop, and resources available to assist departments, such as the office of Research and Institutional Effectiveness, funding available to departments to engage adjunct faculty in the outcomes process, and support from IT (I.B.2-18 pg. 15-17, I.B.2-19, I.B.2-20, I.B.2-21).

The College has Long-Established Distance Education Policies and Processes

Mt. SAC has long established policies and institutional processes that guide its development and evaluation of courses in distance education. The same academic standards and outcomes regularly assessed hold true no matter the mode of delivery. Faculty who wish to teach online must complete a Skills and Pedagogy for Online Teaching (SPOT) certification. Faculty use the tools at their disposal to conduct their outcomes assessments of students’ work via assignments and discussions boards.

Mt. SAC is a pilot college in the Online Education Initiative (OEI), “a collaborative effort among California Community Colleges to increase student success and completion by working together to increase access to quality online courses and support services for students.” Mt. SAC is active in OEI to speed the process of supporting students to completion by offering access to classes online. Training has been incorporated that allows colleges to join together to improve success and retention rates. The goal is to have online training consistent through the environment offered in Canvas (learning management system). Student and faculty surveys as well as student usability studies within Canvas are taking place in spring 2016. Faculty engaged in OEI are submitting courses for review and receiving supportive feedback. Mt. SAC currently has three faculty engaged in the OEI initiative (Administration of Justice, Geography, and Psychology). The most popular courses were selected to be a part of this effort. Included in this effort are on-demand tutoring specific to the courses and readiness modules to support success and retention of students engaged in online learning.

Mt. SAC’s Distance Learning Committee conducted a mock accreditation review of 17 courses. Faculty had a one-on-one interview with the faculty coordinators after the review was completed. The distance learning coordinators summed up their findings in the document, What Did We Learn From the Mock Review? Sixteen distance learning faculty volunteered to have an outside evaluator review their courses for opportunity to improve courses. Faculty felt empowered and supported by the process and gained insight and useful feedback to support student success. Reviewers were those recently engaged in accreditation processes. They were asked to review courses from an accreditation perspective. A summary report was made public, and distance learning faculty are working to implement suggested changes (I.B.2-22, I.B.2-23).

The College Assesses Student and Learning Support Services Outcomes

Student and learning support services at Mt. SAC also define and assess SLOs. Determining outcomes assessment for services has evolved over the years. There have been numerous retreats across campus in order to learn from each other and to improve outcomes assessment (I.B.1-24, I.B.2-24). All areas have outcomes defined and are being regularly assessed. Student Services has developed an organizational model that guides their support services for students. The model uses a “formula”: Assessment + [Participation + Persistence + Progress] = Success. The APPPS model provides an overall perspective to guide a comprehensive plan to provide support services to students. Access efforts ensure that students have opportunities to enroll at the College and to be informed about College processes and requirements. Participation, persistence, and progress efforts ensure that students have avenues to be engaged and involved, to develop goals and educational plans, and to receive support services to assist them in making progress. Success initiatives focus on assisting students in goal completion in order to graduate, transfer, and successfully obtain career employment. Student Services' academic-based outcomes are reported and housed along with all other SLOs as a part of the TracDat system. Many Student Services areas that do not have instructional components also report their outcomes (SLOs and AUOs) in TracDat. All SLOs and AUOs for Student Services are reported, reviewed, recorded, and tracked through the Student Services Division.

For Mt. SAC’s student support services and programs, the Vice President of Student Services and the Dean of Library and Learning Resources each review the outcomes assessment conducted in their areas. On a yearly basis, they provide outcomes assessment information in summary format for the Accrediting Commission for Community and Junior Colleges annual report. They examine outcomes assessment and hold retreats to facilitate deeper dialogue. Experts from the Outcomes Committee and the Office of Research and Institutional Effectiveness are also invited to engage in these conversations. There is ongoing review and discussion about what outcomes assessment means within these support service areas as well as utilization of a standardized approach across the areas regarding measurement and reporting of these activities (I.B.2-25, I.B.2-26).

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) defines and assesses student learning outcomes for all instructional programs and student and learning support services.

Accomplishments and Outcomes

  • Mt. SAC conducts outcomes assessment to improve or sustain success in teaching, learning, and the delivery of services.
  • Courses are assessed and meaningful to the campus.
  • Mt. SAC defines ILOs as the core competencies students should develop as a result of their educational experiences at the College.
  • Although it is but one component of the PIE process, outcomes assessment and alignment is persistent across many levels of planning.

List of Evidence

Title Content
I.B.1-31 Outcomes Committee Quality Review Report 2015
I.B.1-32 Curriculum and Instruction Council Purpose, Function, and Members
I.B.1-33 Educational Design Committee Purpose, Function, and Members
I.B.1-34 Student Learning Outcomes Webpage
I.B.2-1 How to Record Assessment Data
I.B.2-2 AP4020 Program and Curriculum Development
I.B.2-3 Outcomes Committee 4-year Course Review Report
I.B.2-4 Students Guide to Learning Outcomes Fall 2015
I.B.2-5 Presentation Associated Students - Learning Outcomes Fall 2015
I.B.2-6 TracDat Login Page
I.B.2-7 Outcomes Plan Executive Summary
I.B.2-8 Outcomes Committee Annual Report to the Academic Senate 2015-16
I.B.2-9 School of Continuing Education Outcomes Course Review Example ARCH121
I.B.2-10 Student Services Outcomes Review Example COUN99a
I.B.2-11 Physics TracDat
I.B.2-12 Physics TracDat Equipment
I.B.2-13 Physics TracDat Resource Request
I.B.2-14 Adult Basic Education Faculty Fall Meeting 8-22-2014
I.B.2-15 AMLA TracDat Results
I.B.2-16 Support for Program-Level Outcomes AS Resolution 2013-06
I.B.2-17 VP PIE Student Services Outcomes 2014-15
I.B.2-18 Instruction Manager PIE Library and Learning Support Outcomes 2014-15 pg. 15-17
I.B.2-19 Continuing Education Advisory Group Minutes 4-25-14
I.B.2-20 TracDat 5 Presentation
I.B.2-21 TracDat 5 User Guide
I.B.2-22 Pre-Flight Check for Accreditation
I.B.2-23 Quality Distance Learning Courses: A Report on Mock Review and Preflight Programs
I.B.2-24 AP4105 Distance Learning
I.B.2-25 Library and Learning Resources Division Retreat January 2016
I.B.2-26 Library and Learning Resources Division Retreat Follow-up Email Spring 2016

I.B.3.

The institution establishes institution-set standards for student achievement, appropriate to its mission, assesses how well it is achieving them in pursuit of continuous improvement, and publishes this information. (ER 11)

Evidence of Meeting the Standard

The College Establishes Institution-set Standards

Mt. San Antonio College (Mt. SAC) has established Institution-set Standards (ISS) for student achievement through a collaborative process facilitated by the Academic Senate. The Institutional Standards for Student Completion Task Force, composed of managers and faculty, was called for by Academic Senate Resolution 2013-07, in April 2013, to discuss the Accrediting Commission for Community and Junior Colleges ISS requirements and was charged with three goals: 1) to consider the accreditation requirement of setting various standards, 2) to propose a solution that will result in full compliance with Accreditation Standards, and 3) to forward the completed plan to the Academic Senate for review and approval (I.B.3-1).

The Institutional Standards for Student Completion Task Force discussed initial data points for satisfactory performance of student achievement in learning in five areas: 1) successful course completion rate by percent, 2) fall-to-fall persistence, 3) number of degrees, 4) number of transfers to four-year colleges/universities, and 5) number of certificates awarded (I.B.3-2). Through use of the initial data points, a recommendation for ISS was made and communicated in a report presented to Academic Senate for review. In 2015, additional ISS were established for job placement and licensure examination passage rates. Included in the final report to the Academic Senate was a recommendation by the task force to have the Institutional Effectiveness Committee (IEC) lead the implementation of the ISS. The final set of ISS indicators was reviewed by Academic Senate and sent to IEC for approval (I.B.3-3).

Mt. SAC’s ISS reflect student achievement, which is aligned to the College mission:

The mission of Mt. San Antonio College is to support all students in achieving their educational goals in an environment of academic excellence. Specifically, the College is committed to providing quality education, services, and workforce training so that students become productive members of a diverse, sustainable, global society. The College pledges to prepare students for lifelong learning through the mastery of basic skills, the achievement of associate degrees and certificates, and the completion of career and transfer pathways. The College will carry out this commitment by providing an engaging and supportive teaching and learning environment for students of diverse origins, experiences, needs, abilities, and goals. The College is dedicated to serving our community through improving economic achievement, advancing civic engagement, enhancing personal well-being, promoting critical thinking, and enriching aesthetic and cultural experiences. Board approved February 17, 2016 (I.A.1-2).

The College Assesses Institution-set Standards

IEC has taken on the role of instituting a process for supporting a broad-based understanding of the ISS goals and processes to implement them (I.B.3-4). ISS are publicly available to College constituents through the Mt. SAC website Factbook. However, these were not initially presented in a format that was easy to understand.  At first, there was not widespread awareness or understanding of ISS, nor a process for each College unit to identify how its work contributes to and supports ISS. In 2015-16, IEC included ISS and the Institutional Effectiveness Partnership Initiative (IEPI) outcomes in PIE to support a broad-based understanding of the College priorities and provide a mechanism for widespread dialogue and development of specific strategies to achieve the ISS and IEPI outcomes. As of 2015-16, ISS is in the unit/department PIE; each area is required to address how it is contributing toward these standards.

Mt. SAC’s ISS are articulated so the College can determine the degree to which they have been met. The initial level for each ISS was set using averages for each of the following measures: student success, student course pass rate by percent, fall-to-fall persistence, number of degrees, number of transfers, and number of certificates. Mt. SAC implements ISS through ongoing processes, such as Planning for Institutional Effectiveness (PIE) (I.B.3-5 pg. 5). For noncredit persistence, specific interventions were put into place to help students with their progression to completion. This included early alert systems and embedded support services. For credit course success, the College uses this disaggregated data in its Student Equity Plan to provide focused direction for project funding and goals.

The ISS are designed to support Mt. SAC faculty and staff in analyzing student performance. If student data indicates the level of performance expectation is not being met, this provides the College an opportunity to consider support activities, which can positively impact student achievement and help students reach their academic potential. Targeted support activities for students who are not achieving are being designed and implemented. The ISS were evaluated for the first time as part of the 2014 ACCJC Annual Report (I.B.3-6). Faculty are using ISS as additional assessment tools. For example, the Career Technical Education programs examine their students’ job placement rates annually and, based on the findings, re-set their ISS. It is difficult, at times, to create the ISS due to the low number of program graduates and the fact that some programs require that students complete the four-year program of study before they qualify for employment.

One example related to student persistence includes the Basic Skills Pathways to Transfer, which came about because students were not progressing through the English and math course sequences in sufficient numbers. This project offers cohorts and in-class support resources for students to progress through basic skills English and math classes. Assessment data indicate students participating in a Basic Skills Pathway demonstrate increased student persistence through sequential courses leading to college-level work. Work on Pathways continues each semester as the College continually refines this program to improve upon student outcomes (I.B.3-7).

The College Publishes Institution-set Standards

The ACCJC Annual Report is shared at IEC meetings, President’s Advisory Council, and via internal communication structures at the College. ISS results are published on the website and used in program review.

In fall 2015, Mt. SAC received notification of enhanced monitoring related to job placement rates which did not meet The College's established ISS. The College immediately began dialogue regarding the concerns. It was determined that the College would change how it calculates job placement rates and would use an outside vendor for the Alumni Survey. The Deans also discussed the setting of the job placement rate on a program-by-program basis, implemented the change and provided updated ISS information. Results of this work were communicated to the Commission in an addendum format.

On division and unit levels, PIE informs the College as to the extent it is achieving its goals for students (I.B.3-5). In its role in leading the ISS implementation, IEC uses evidence to demonstrate progress towards achieving the goals. A clear example of this work is in Pathways to Transfer. Through this program, the College is demonstrating its support for improving the transfer data. Similarly, The College's grant-writing efforts and program development such as Degree Works (MAP), demonstrate support for improving student success (e.g., course, degree/certificate).

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) establishes institution-set standards for student achievement, appropriate to its mission, assesses how well it is achieving them in pursuit of continuous improvement, and publishes this information widely for the campus and the community.

Accomplishments and Outcomes

  • Mt. SAC has established Institution-set Standards (ISS) for student achievement through a collaborative process.
  • The College uses multiple cohort tracking to measure the impact of its interventions on programs, services, and technology.
  • ISS is in The College's program review, PIE, in a way that requires all areas to note how they are contributing toward the standards.

List of Evidence

Title Content
I.A.1-2 Mission, Vision, Core Values
I.B.3-1 Academic Senate Resolution 2013-07
I.B.3-2 Institutional Set-Standards Introduction & Overview
I.B.3-3 IEC Recommendations for ISS Spring 2016
I.B.3-4 IEC Minutes 3-23-2016
I.B.3-5 Unit PIE Reports with ISS Data 2015-16 pg. 5
I.B.3-6 ACCJC Annual Report 2014
I.B.3-7 Pathways to Transfer

I.B.4.

The institution uses assessment data and organizes its institutional processes to support student learning and student achievement.

Evidence of Meeting the Standard

The College Uses Assessment Data to Support Student Learning

At Mt. San Antonio College (Mt. SAC), measurable objectives (MOs) and student learning outcomes (SLOs) stand as the criteria for success in a course. MOs are all of the outcomes that a student will meet, or learn, in a given course. SLOs may be derived from MOs in order to achieve more specific assessment and analysis. A department determines the SLOs through a discussion about which measurable objectives it wants to focus on for strengthening student success. Once a department has determined its SLOs, a rubric is created and criteria for success is established. The SLOs are then assessed on a regular cycle. The results of the assessments are collected and discussed at a department meeting, and the discussion leads to using the results of the assessment for improvement. The use of results might lead to a reassessment of the current SLOs after a determined pedagogical or curricular change, or it might lead to choosing a different SLO to assess in the next cycle. The end result of each cycle of this process is indicative of the College culture that focuses on improvement in teaching and learning.

The College uses assessment data to support student learning. Knowledge and competencies are expressed for segments of study or activity through measurable learning outcomes at the institutional, program, degree, and course levels. The outcomes are written as student learning outcomes (SLOs), a means to determine what students know, think, feel or do as a result of a given learning experience. Use of outcomes assessment at the course, program, and institutional levels gives departments’ data on student achievement in relationship to expected student learning results. The College systems are organized and clearly connected with use of SLO assessment data to inform decisions and improve students’ learning, achievement, and certificate/degree attainment. TracDat 5 is The College's central, password protected repository used for housing outcomes assessment work for courses and programs.

Outcomes assessment is an ongoing, evidence-based approach that indicates the extent to which programs and courses achieve their intentions. Assessment uses a feedback loop that involves identifying desired results, collecting and analyzing relevant information (data), and then using the findings to conduct discussions and direct activities that improve instructional delivery, curricula, programs, and/or services. Programs establish rotation cycles, many in a three- to four-year cycle aligned with curriculum. Career Technical Education programs are including outcomes assessment results as part of annual advisory meetings to monitor and inform program planning (I.B.4-1, I.B.4-2, I.B.4-3, I.B.4-4).

In 2014-15, Mt. SAC’s four institutional level outcomes (ILOs) were designed to be broad and inclusive of all activities across campus. These ILOs provide a higher-level view of the students’ learning outcomes. The Outcomes Committee worked systematically with departments to map courses to program level outcomes (PLOs). All instructional and student services division departments use TracDat to map SLOs and PLOs to ILOs. Data is being collected across the College to identify patterns of outcomes mapping (I.B.4-1, I.B.1-11, I.B.4-5).

The SLO process at Mt. SAC has matured. Faculty discussions now include topics related to stability of SLOs and ongoing data collection of these over a period of time. SLOs in which criteria are being met provide a deeper understanding over time of student success. SLO data is collected in various ways at Mt. SAC. For example in some programs, SLO data is collected for the different classroom modalities (distance learning, hybrid, traditional) and include both adjunct and full-time faculty to examine consistency of learning outcomes (I.B.1-15, I.B.4-6, I.B.4-7, I.B.4-8). SLO data is used to make modifications to the SLO, clarify objectives, and refine course content and pedagogy. It also prompts discussions regarding rigor. For transparency purposes, PLO results are pulled from TracDat and made available on the College website (I.B.4-1).

At the management level, instructional divisions use achievement data to guide critical decisions. Divisions use data for enrollment planning and scheduling – provided primarily by Argos reports displaying data such as enrollment fill rates and degree/certificate completion – as well as degree and general education requirements. Campus administrators, faculty, and staff use the Argos reports each semester and within each term to guide planning and scheduling. As new needs and questions arise, additional reports are designed to provide on-demand access to critical data for decision making.

The Outcomes Committee shares best practices across campus via venues for dialogue. The Committee makes a significant effort to include adjunct faculty in the conversations and secured funding for adjunct faculty engaged in outcomes assessment conversations. The Outcomes Committee’s review of the TracDat course-level outcomes assessment indicates that outcomes assessment is being completed on most courses, but a comprehensive review of outcomes assessment is accomplished as part of the four-year course review process. A random sample of courses is selected and a rubric is used to evaluate the quality of outcomes assessment. The findings indicate about 75 percent have their outcomes in place and have assessed them over the past four years. If not, per the Administrative Procedure, faculty are given one year to complete their outcomes assessment. If they do not comply, then the course is deactivated.

Student Services also completes outcomes assessment in their programs and services for students (I.B.4-9). Student Services offers a broad range of programs and services that allocate both financial and personnel resources to support student learning and achievement. These programs and services serve a diverse academic, socio‐economic, and cultural student population. Student Services programs and departments maintain SLOs. Outcomes are designed to measure the breadth and depth of support services offered to students based on their specific educational and developmental needs (I.B.1-24, I.B.2-17 pg. 17-18). Additionally, many Student Services programs also complete administrative unit outcomes (AUOs) that measure program effectiveness and student satisfaction.

The College Uses Assessment Data to Support Student Achievement

The College uses assessment data to support student achievement. Systematic measurement of student learning outcomes is done within classrooms and in student support services. Results from these assessments provide opportunities for examining students’ learning and subsequent student achievement (e.g., course grades). It is through this process of measuring students’ progress in their learning that the College uses resources effectively. For example, English Department discussion of course outcomes resulted in increased use of tutors in the classroom provided through the Writing Center. Writing Center data provides evidence of the increase in student achievement rates resulting from tutoring. These data are presented to the English Department at the end of major semesters for further review and discussion of student needs.

Based on student assessment and achievement data, Mt. SAC has established a broad range of support services to meet the academic development needs of students. Low student assessment scores in math and English led to the establishment of the Bridge program, a learning community which combines counseling, instruction, and other learning support services to assist first generation, basic skills students in successfully passing sequential basic skills courses upon entry to the College. Analysis of data demonstrating low course and program success for low socioeconomic and first generation students resulted in development of support programs that provide both a counseling and instructional/tutorial programmatic effort that supports student learning and student success: ACES TRiO (Achieving in College, Ensuring Success). Data on African-American student achievement was the impetus for development of the Aspire program. Similarly, review of data for Asian American and Native American Pacific Islander Service Institution students supported development of the Arise programs. This pattern of data review and student support program development is evidenced in creation of the Veterans Resource Center, WIN (Athletics), Honors Program, Counseling, and Disabled Students Programs & Services. The Research and Institutional Effectiveness department provides research and evaluation support across the campus. Specific project support is given to basic skills, student equity, enrollment management, Career Technical Education, outcomes assessment, and Student Services to mention a few. It is through this support that the College is able to use different methodologies to measure the impact of interventions to improve student success (I.B.4-10).

Figure I.B.4.-3. Process Map of Integrated Planning
Long Description
Figure I.B.4.-3. Process Map of Integrated Planning

The College Uses Institutional Processes to Support Student Learning and Student Achievement

The College's integrated planning process is systematic and includes its Strategic Plan, focused plans, program review (PIE), outcomes assessment, and resource allocation.

Mt. SAC has a program review process that includes integrated planning. The College's program review (Planning for Institutional Effectiveness (PIE) is updated annually by all units/departments and includes a component on outcomes assessment as well as student achievement.

The PIE process is a regularly established annual mechanism for planning, evaluating, and documenting accomplishments and challenges. Units/departments complete their PIEs. Managers then summarize their unit/departments’ PIEs. Vice Presidents analyze and summarize their managers’ PIEs, and the President submits a President PIE so his direct report areas are included in the College planning process. The institution wide PIE summary is completed by the Institutional Effectiveness Committee (IEC) and reports on the overall compilation of the College wide program reviews.

Annual PIE reports require all areas to evaluate their accomplishments, examine the impact of internal and external forces on their work, evaluate their effectiveness using data, think critically about student learning outcomes assessment and its impact on student learning, as well as to plan into the future as to how they are going to improve student learning and all support services (I.B.4-11, I.B.1-30, I.B.4-12, I.B.4-13, I.B.4-14, I.B.4-15, I.B.4-16, I.B.4-17, I.B.1-58, I.B.4-18).

The budget and integrated planning process is aligned on the same timeline as The College's 2014-15 Strategic Plan schedule (Figure I.B.4.-4). The IEC and the Budget Committee met in fall 2014 to align their program review and budget processes (I.A.3-1 pg. 22, I.B.4-19, I.B.4-20 pg. 2, I.B.4-21, I.B.4-22, I.B.4-23 pg. 2).

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) uses assessment data and organizes its institutional processes to support student learning and student achievement.

Accomplishments and Outcomes

  • The College uses data in numerous ways to support student learning and student achievement including asking all units to describe how they are aligning their work to support The College's efforts to meet or exceed the ISS.
  • The College provides support for research and evaluation through Research and Institutional Effectiveness. The cohort-tracking process used from the program level to the College level provides the College with a data-driven opportunity to use this information as part of its evaluation process for program improvement.

Integrated Planning and Budget Process Calendar
Long Description
Figure I.B.4.-4. Integrated Planning and Budget Process Calendar (continued to next page)

 Integrated Planning and Budget Process Calendar Continued
Long Description
Figure I.B.4.-5. Integrated Planning and Budget Process Calendar (continued from previous page)

List of Evidence

Title Content
I.A.3-1 Strategic Plan pg. 22
I.B.1-11 ILO Mapping
I.B.1-15 Outcomes Committee Annual Report 2014-15
I.B.1-24 Summary Report for ILO Mapping 8-2016
I.B.1-30 Basic Skills Annual Report 2014-15
I.B.1-58 PIE Summary College-wide 2014-15
I.B.2-17 VP PIE Student Services Outcomes 2015-16 pg. 17-18
I.B.4-1 Outcomes Web page
I.B.4-2 Outcomes Rotation Cycle - Electronics
I.B.4-3 Outcomes Rotation Cycle - CNET
I.B.4-4 CTE Annual Advisory Minutes - Electronics
I.B.4-5 Division Meetings Minutes
I.B.4-6 Outcomes Committee Follow-up Reports 2014-15
I.B.4-7 Outcomes Committee Annual Reports 2013-14
I.B.4-8 Outcomes Committee Follow-up Reports on Adjunct Funding 2013-14
I.B.4-9 PIE Appendix A Student Services Division SLOs
I.B.4-10 Research & Institutional Effectiveness Recent Project List
I.B.4-11 Argos Enrollment Reports
I.B.4-12 PIE VP Summary Administrative Services 2014-15
I.B.4-13 PIE VP Summary Student Services 2014-15
I.B.4-14 PIE VP Summary Human Resources 2014-15
I.B.4-15 PIE VP Summary Instruction 2014-15
I.B.4-16 PIE VP Summary President 2014-15
I.B.4-17 Outcomes Reports and Campus Events
I.B.4-18 PIE Email Correspondence 1-22-2015
I.B.4-19 Expanded Presidents Advisory Committee Minutes 11-12-2014
I.B.4-20 President's Advisory Council Minutes 5-22-2013 pg. 2
I.B.4-21 President's Advisory Council Minutes 6-12-2013
I.B.4-22 President Cabinet Action Notes 10-22-2013
I.B.4-23 President's Cabinet Action Notes 1-31-2015 pg. 2

I.B.5.

The institution assesses accomplishment of its mission through program review and evaluation of goals and objectives, student learning outcomes, and student achievement. Quantitative and qualitative data are disaggregated for analysis by program type and mode of delivery.

Evidence of Meeting the Standard

The College Accomplishes Its Mission through Program Review and Evaluation of Goals and Objectives

Mt. San Antonio College (Mt. SAC) has built a culture of decision making that is data driven. The process for annual review of the mission by the President’s Advisory Council (PAC) demonstrates how foundational this is to College work. The process involves asking governance committee members to gather constituency group member insights. The campus community is invited to send input about the mission to the President. PAC reviews the suggested changes and makes recommendations to the College President. The Board of Trustees then reviews the changes and approves them at a Board meeting. In the midst of the recession, data gathered about the mission through this process indicated that College programs were no longer serving “all” students in achieving their educational goals. Instead, the College was turning away students due to budget cuts. Programs with student outcomes leading to increased wages in high demand fields were being prioritized. This led PAC to make the decision to remove the word “all” from the mission. After the recession, as funding stabilized and the College returned to a growth mode, the word “all” was reintroduced to the mission statement. The most recent review of the mission statement necessitated changes to align it with the new Accreditation Standards. The current mission is noted below:

The mission of Mt. San Antonio College is to support all students in achieving their educational goals in an environment of academic excellence. Specifically, the College is committed to providing quality education, services, and workforce training so that students become productive members of a diverse, sustainable, global society. The College pledges to prepare students for lifelong learning through the mastery of basic skills, the achievement of associate degrees and certificates, and the completion of career and transfer pathways. The College will carry out this commitment by providing an engaging and supportive teaching and learning environment for students of diverse origins, experiences, needs, abilities, and goals. The College is dedicated to serving our community through improving economic achievement, advancing civic engagement, enhancing personal well-being, promoting critical thinking, and enriching aesthetic and cultural experiences. Board approved February 17, 2016 (I.A.1-2).

Mt. SAC uses program review and strategic planning to evaluate achievement of its mission. The Mt. SAC Strategic Plan operationalizes the implementation of the College mission through defined College goals, strategic objectives, and key performance indicators (KPIs). Strategic objectives support the attainment of the College goals which, in turn, support The College's mission. Each strategic objective is accompanied by one or more KPI. These indicators measure progress in achieving the objective and advancing toward the goal. Some KPIs also have benchmarks which state an expected level of improvement in the metric (I.A.3-1 pg. 7-21).

Within program review, each unit/department is required to document its achievements and their alignment to College goals. It is through this iterative process of program review synthesis from the unit/department, to the manager, to the Vice Presidents and President, and to the Institutional Effectiveness Committee (IEC) that the College demonstrates a cyclical, systematic process of integrated planning toward improved institutional effectiveness and quality education. Each year, the IEC creates a Planning for Institutional Effectiveness (PIE) summary document that evaluates how well the College is achieving its mission. Within this document is an assessment of the College goals as well as strategic objectives. An evaluation of the process closes the loop on the process and provides final evidence of the impact of program review on the College achieving its mission and goals (I.B.5-1, I.B.5-2, I.B.5-3, I.B.5-4, I.B.5-5, I.B.1-58 pg. 50)

The College Accomplishes Its Mission through Student Outcomes

Each College goal provides evidence of The College's progress toward achieving its mission. College goals are organized thematically to focus the campus community on critical issues. Articulated by the President's Advisory Council and approved by the Board of Trustees, they guide institutional planning and assessment processes.

Under College Theme A: To Advance Academic Excellence and Student Achievement, College Goal #3 is:

The College will utilize student learning outcome and placement assessment data to guide planning, curriculum design, pedagogy, and/or decision-making at the department/unit and institutional levels.

Based on program review accomplishments, it is clear that the College has a myriad information that demonstrates how well it is achieving its mission through student learning outcomes assessment. For example, the Humanities and Social Sciences (HSS) manager’s PIE states that the division has experienced an “Increase in pedagogical discussions in response to outcomes assessment data, via institutionalized bi-annual department outcomes meetings that include all faculty.” The PIE summary report indicates progress being made in outcomes assessment and the continued need to provide supportive resources (I.B.5-6, I.A.3-5, I.B.5-7, I.B.5-8).

 

The College Accomplishes Its Mission through Student Achievement Using Qualitative and Quantitative Data

Mt. SAC uses qualitative data to ensure that students are learning in all programs and through all modes of delivery. Programs and services undergo a cyclical, systematic review of programs, goals, student learning outcomes, and student achievement (I.B.1-30, I.B.1-31, I.B.5-9, I.B.5-10, I.B.5-11, I.B.2-25). In the narrative sections of PIE, each College unit/department completes a narrative section in which it closes the loop on alignment and progress on College goals. This section serves as a reporting function in which each area describes how planning connects to budget allocation and outcomes/results. Through qualitative and quantitative data reporting, units describe how prioritized College resources connected to the area's outcomes over the past year and the progress made with the resources provided. This analysis leads to future planning. Managers and Vice Presidents rely on the expertise of employees at the unit and operational levels to bring forward the current and future needs of the College during PIE development. Vice Presidents (Administrative Services, Human Resources, Instruction, and Student Services) complete extensive reviews of their divisions and draw upon this data to provide a far-reaching review of qualitative data to report in their division PIE summaries (I.B.5-12, I.B.3-5 pg. 7, I.B.3-1, I.B.5-13).

Focus groups are regularly conducted for a variety of projects (Student Equity Futures Team Projects, Bridge, Marketing, Website Improvements, and Outcomes Qualitative Report). Mt. SAC’s library puts out poster boards with broad guiding questions to collect qualitative data from students. Examples include, “What can we improve? Are extended hours helpful?” It has been an effective way to gain feedback in an open learning environment. Student feedback has been used to make changes such as opening a designated “quiet learning space” near the periodicals, providing more Wi-Fi in the library, and requesting facilities remodeling in PIE (I.B.5-14).

Quantitative data is used widely at Mt. SAC to ensure that students are learning in all programs and through all modes of delivery (traditional face-to-face or distance education). Outcomes assessment data at the course, program, and institutional levels allows departments to discover if the students are, in fact, learning what they are expected to learn. Program reviews provide the College units/departments with many opportunities to examine relevant data and to use it for program improvement purposes. The PIE summary report appendix indicates the many different data sources used. A good example of this is from the Basic Skills Coordinating Committee. The committee initiates an annual call for proposals and recommends funding many grants. Metrics of this work are documented in the annual reports, which show both qualitative and quantitative data analysis and use of the data for improvements (I.B.5-15, I.B.5-16, I.B.1-58, I.B.5-11, I.B.1-16, I.B.1-15, I.B.4-8, I.B.1-30, I.B.5-17, I.B.1-44).

The College Accomplishes Its Mission through Disaggregation of Data

The College regularly disaggregates data in an ongoing effort to address disparities in student success. This disaggregation of data has enabled the College to develop and implement critical interventions to address inconsistencies. For example, the highly successful Bridge program was created to improve the success rates of entering freshmen from local high schools who place at basic skills levels. Bridge students are low income, first generation and mostly Latino, and are considered under-prepared for college. The program addresses their unique academic and learning support needs and has resulted in high rates of persistence and course success. For distance learning, data are separated by mode of delivery and success rates are examined by the Distance Learning Committee and reported to the Board of Trustees as warranted. Differences in success rates are addressed in various ways, such as the Are You Ready to Take an Online Course Assessment to help students self-assess their readiness for a distance learning course. In the School of Continuing Education data is disaggregated by program; each program represents a different area of noncredit education approved by the Chancellor’s Office. Data for each program is disaggregated including student success, retention, course completion, and certificate completion (I.B.5-18, I.B.5-19, I.B.5-20).

Data are disaggregated depending on the needs of the program and its students. For example, The College's work on student equity is disaggregated by many criteria, including race/ethnicity, gender, age, disability, economically disadvantaged, first generation, veteran, and foster youth. All funded student equity projects are required to focus on closing the equity gap for targeted student populations. Equity services are asked to examine which student populations are most under served and less successful and to create specific opportunities for those who may need extra guidance. The variety of ways to examine the data and the interventions put into place demonstrate the continued focus on academic quality (I.B.5-21).

Across the College, the Research and Institutional Effectiveness department is involved in many of these and other projects, from providing general consultation to data collection, analysis, and reporting to facilitating appreciative inquiry conversations about the results. For student equity, the College is supporting two full-time researchers and professional experts for this work. In total, there are six full-time researchers, multiple professional experts (at-will employees), and the director in the department. Various budget sources are used to fund these positions to support The College's research and evaluation needs. Many of the positions have pre-defined areas of specialty: basic skills, outcomes assessment, student equity, enrollment management, grants, student services, Career Technical Education, and the Institutional Review Board. All work together to ensure efficiency of their work and create a unified perspective on the Research Office’s impact on student success across the College (I.A.1-12, I.B.5-22, I.B.5-23, I.B.5-24).

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) assesses accomplishment of its mission through program review and evaluation of goals and objectives, student learning outcomes, and student achievement. Quantitative and qualitative data are disaggregated for analysis by program type and mode of delivery.

Accomplishments and Outcomes

  • The College mission guides The College's work.
  • Strong established program review includes clear links to the College mission and goals.
  • Strategic Plan and program review evaluate achievement of College goals and objectives.
  • Student learning outcomes are a strong component of The College's culture.
  • Recent mapping of institutional level outcomes across Instruction and Student Services indicates a strong level of commitment to attainment of these outcomes.
  • Student achievement (e.g., course success) is aligned with The College's mission, College goals, and objectives.
  • Extensive research is occurring across the campus with a strong support structure in place in the Research and Institutional Effectiveness Office.
  • Qualitative and quantitative research and evaluation studies are ongoing and include disaggregation for analysis where appropriate by program type and mode of delivery.

List of Evidence

Title Content
I.A.1-2 Mission, Vision, Core Values
I.A.1-12 Student Equity Plan 2015-16
I.A.3-1 Strategic Plan 2015-17
I.A.3-5 AP3250 Institutional Planning
I.B.1-15 Outcomes Committee Annual Report 2014-15
I.B.1-16 Outcomes Committee Annual Report 2015-16
I.B.1-30 Basic Skills Annual Report 2014-15
I.B.1-31 Qualitative Review of Outcomes 2015
I.B.1-44 Success and Retention in Distance Learning Courses
I.B.1-58 PIE Summary College-wide 2014-15 pg. 50
I.B.2-25 Library and Learning Resources Division Retreat Follow-up Email Spring 2016
I.B.3-1 Academic Senate Resolution 2013-07
I.B.3-5 Unit PIE Reports with ISS Data 2015-16 pg. 7
I.B.4-8 Outcomes Committee Follow-up Reports on Adjunct Funding 2013-14
I.B.5-1 PIE VP Form 2013-14
I.B.5-2 PIE Dept Form 2013-14
I.B.5-3 PIE Appendix B
I.B.5-4 PIE Appendix C
I.B.5-5 Completed PIE Web Page
I.B.5-6 PAC Purpose, Function, and Members
I.B.5-7 Institutional PIE Program Analysis and Review 2014-15
I.B.5-8 TracDat Mapping
I.B.5-9 Outcomes Committee Purpose, Function, and Members
I.B.5-10 Qualitative Outcomes Review Reports 2013
I.B.5-11 Qualitative Outcomes Review Reports 2014
I.B.5-12 PIE Website
I.B.5-13 PIE for Financial Aid
I.B.5-14 Library and Learning Resources Retreat Combined Packet January 2016
I.B.5-15 PIE Summary College-wide 2012-13
I.B.5-16 PIE Summary College-wide 2013-14
I.B.5-17 Argos Report on Distance Learning by Success, Retention, and Ethnicity
I.B.5-18 Distance Learning Report to Board of Trustees
I.B.5-19 Distance Learning Committee Minutes
I.B.5-20 School of Continuing Education Student and Community Profile
I.B.5-21 Math Basic Skills Sequence Details
I.B.5-22 English Basic Skills Sequence - Success and Progression at Each Course
I.B.5-23 Bridge Program Report
I.B.5-24 Honors Completion Study Report

I.B.6.

The institution disaggregates and analyzes learning outcomes and achievement for subpopulations of students. When the institution identifies performance gaps, it implements strategies, which may include allocation or reallocation of human, fiscal, and other resources, to mitigate those gaps and evaluates the efficacy of those strategies.

Evidence of Meeting the Standard

The College Disaggregates and Analyzes Learning Outcomes and Achievement for Subpopulations of Students

The Student Equity Plan demonstrates how Mt. San Antonio College (Mt. SAC) evaluates institution wide data on learning outcomes and calculates disproportionate impact on subpopulations of students. This is not a new process for Mt. SAC. Data-driven decision-making to improve student success based on analysis of learning outcomes is part of the culture of the College. From 2004-2006, Mt. SAC participated in the Equity for All project through USC’s Center for Urban Partnerships. This effort enabled the College to complete a finer grain analysis of which student groups were below equity on key indicators. The findings of this work helped to focus several interventions to improve success rates, especially in math, for Latinos, African-Americans and Pacific Islanders. Although many under-represented students placed below equity into English classes, students were found to have made substantial progress in successfully passing English courses required for graduation and transfer. This was not the same finding with math courses. Students were neither enrolling nor passing transfer level math courses, leading to a marked disproportionality in transfer rates especially for Latino and African-American students.

In 2011-12, years before the state required a revised equity plan, the Mt SAC Student Preparation and Success Council developed a Student Success Plan. The purpose of the plan was to identify gaps in support for students and interventions to improve success. Three main goals and action plans were developed: student engagement and persistence, development of students’ own goals, and increasing successful completion of basic skills course sequences. In May of 2013, the plan was updated and participants engaged in activities to integrate Mt. SAC’s work with elements of the RP Group’s presentation of Student Success (re)Defined. Following the creation of the Student Success Plan, a campus wide student equity convening was held in May 2014 with over 70 faculty, staff, students and managers in attendance. Key campus leaders and constituency groups were able to provide input on the five Student Equity Success Indicators:

  1. Access,
  2. Course completion,
  3. English as a Second Language and basic skills completion,
  4. Degrees and certificates, and
  5. Transfer.

This enabled the committee members and writing teams to draft the 2014-15 Student Equity Plan goals, activities, timelines, and expected outcomes to provide greater, equitable opportunities for specific subpopulations of students to attain academic success (I.B.1-30, I.A.1-12, I.B.1-52).

Beginning in the 2014-15 academic year and continuing through the 2015-16 academic year, the College has committed work of the Research and Institutional Effectiveness (RIE) department to continuously collect and analyze data related to disproportionality in student success for target student groups. The research method selected by Mt. SAC for2015-16 combines the 80 percent disproportionality method together with a velocity approach using current rather than cohort data, along with a gap measurement. In addition to gender and ethnicity, Mt. SAC disaggregates student data by disability, age, veteran status, foster youth, single parents, individuals receiving public assistance, recent high school graduates, re-entry and older adults, and limited English-proficient students. The College is looking by zip code to determine particular communities of students who are under served. RIE staff work directly with College leadership (staff, faculty and administrators) assigned to student success and student equity projects to develop reports and provide research/evaluation services that measure progress and completion for these particular groups of students. The College makes decisions and develops interventions based on this data (I.B.5-22, I.B.5-21).

Using information from this analysis, groups most at risk are the focus of funded projects in the 2015-16 Equity Plan that contribute toward closing the equity gap. Embedded in the plan are clear ties linking specific activities and services for targeted student populations representing disabled, educationally disadvantaged, low income, foster youth, veterans, financial aid, Extended Opportunity Programs and Services (EOPS), CalWORKs students, basic skills efforts, and core services related to Student Success and Support Program (SSSP). In addition, Mt. SAC has identified other targeted student groups whose success rates warrant further study with accompanying interventions, including AB 540/Undocumented/Dream, LGBTQ, first generation college, minority males, and non-native English speakers. Goals and activities are established for the specific populations in need of support and prioritized allocation of resources is aligned with the overall Student Equity Plan (I.A.1-12, I.B.1-52). The effectiveness of the equity plan projects on targeted student populations is measured from qualitative and quantitative perspectives.

The 2015-16 Student Equity Plan is a product of many forces coming together to provide research data, planning information, and sharing of past successes and future plans to continue to address the needs of under-represented and under-served students. Heroic efforts are in place to develop essential services to meet the needs of The College's targeted student groups. Cross-collaboration, including integration with The College's SSSP Plan, Basic Skills Plan, Accreditation, and Strategic Plan has enabled the Mt. SAC Student Equity Plan to represent a comprehensive approach to addressing disproportionality and the five success indicators. Continuing research is a primary focus for The College's student equity efforts. Monitoring more specifically which particular student groups are making progress and which are lagging is critically important. Finer grain access to look at within-group differences will be our focus to more clearly determine the specific needs of students which need to be addressed.

There is overlap between the Student Equity Plan and The College's Institution-set Standards (ISS): Course Completion/Success, Degrees, Certificates, and Transfer. This overlap allows the College to use appreciative inquiry as it examines data. Going beyond ISS, the work that the Research Office does for the Student Equity Committee is also being presented to Deans of academic programs (English, math, LERN) to support data-driven decision making leading to increased student achievement (I.B.6-1).

The Research and Institutional Effectiveness department provides disaggregation of data on a project-by-project basis. Mt. SAC currently disaggregates data for learning outcomes and achievements by various subpopulations, which include: Honors, Bridge, Aspire, Arise, Title V, Basic Skills, Statway, and Pathways to Transfer (I.B.6-1, I.B.6-2, I.B.6-3, I.B.6-4, I.B.6-5, I.B.6-6). In these populations, learning outcomes are measured in the related courses these students take (e.g., Math 51) and improvements in teaching and learning occur as a result of this outcomes work. The coordinators/directors for these programs also measure outcomes for specific workshops/events and use findings to improve students' learning as well as to justify resource allocation or reallocation (I.B.6-7, I.B.6-8, I.B.6-9, I.B.1-30).

When examining disaggregation and analysis of learning outcomes for subpopulations of students, the College focuses on two areas: (1) student and support services and (2) special programs (e.g., Bridge). Many support services programs track learning outcomes for subpopulations of students based on the targeted student populations served. The ongoing Bridge Longitudinal Study enables program leaders to analyze which profiles of students are reaching success more equitably than others. For many student and support services, program and service learning outcomes are apparent within the nature of what students have to do in order to remain eligible for services. For example, eligible low income and educationally disadvantaged Extended Opportunity Programs and Services (EOPS) students learn what they need to do and by what deadline in order to secure services. Students who meet the requirements are considered to have achieved the learning outcome each year. Students are required to meet with their EOPS counselor at least three times each semester. They must complete a progress report and maintain a grade-point average of no less than a 2.0 in order to remain in “good standing” with the program. Students who fail to comply could lose benefits such as a book voucher. On a regular basis, the program evaluates student’s performance and makes adjustments accordingly to improve learning outcomes.

In the aggregate, analyzing how subpopulations of students are meeting learning outcomes and achieving goals is achieved through collectively analyzing specialized support services. Aspire (Umoja), Arise (Asian American and Native American Pacific Islander-Service Institution), Dream, foster youth, veterans, ACES (TRiO), EOPS/CARE, CalWORKs, and Disabled Student Programs and Services student learning outcomes data enable the College to assess how specific subpopulations of students are achieving success and reaching goals. Many of these programs are categorically or grant funded which requires the state or federal reporting of very specific outcomes measures.

Beyond individual student records, some departments use other records to determine student learning outcomes, including the number of student education plans, the number of students who attend orientations, and the reduction in the number of students who are dropped for non-payment of registration fees. Program reviews for programs serving these subpopulation groups provide a wealth of information (including resource allocation) to support the use of learning outcomes (I.B.6-10, I.B.5-13, I.B.6-11, I.B.6-12).

The College Identifies Performance Gaps, Implements Strategies (Allocation or Reallocation of Human, Fiscal and Other Resources) and Evaluates Impact of Strategies

Mt. SAC tracks many subpopulations. They are identified in the Student Equity Plan, Basic Skills Plan, and other plans as areas for focus because of possible at-risk subpopulations. For example, African-American male students were identified in the Student Equity Plan as those who were at-risk for achievement levels below that of the average for all subgroups (gender by ethnicity). Many of the programs at the College (e.g., Arise, Noncredit Adult High School Diploma and High School Equivalency) are designed to meet the needs of subpopulations of students who have already been identified as groups needing support structures to help them achieve their educational goals. Arise students benefited from the program and services as demonstrated in their increase in success and progression rates.

Information on student successes and challenges are identified in different manners. For example, the Basic Skills Coordinating Committee funds proposals targeted to increase student outcomes where performance gaps have been identified. At the conclusion of the annual grant cycle, data on student achievement specific to the proposal are collected, evaluated, and shared in an annual report (I.B.6-13). The efficacy of the intervention is evaluated and used to decide on its funding for the next round of proposals.

The Colleges’ ability to access data, determine how it thinks about data, and consider how it uses data have significantly shifted since its 2010 Self Evaluation because of improved tools providing individual employees with job-specific report access and unitary data (as appropriate). Additionally, the College has purchased add-on systems so that programs/departments can gather and track specific information and monitor student achievements in order to measure the impact of their interventions. For example, Student Services’ TRiO programs purchased a specific student tracking and outcomes measurement system to help them monitor grant goals outcomes, and impacts of specific program interventions on student success.

Mt. SAC uses strategies to mitigate performance gaps that are identified by the analysis of data. Early intervention is one such strategy, as are cohorts. The challenge is understanding barriers to student use of services and targeting strategies for providing timely support. For example, the College looks at patterns of student attendance and aligns service availability accordingly. Additionally, The College's Title V Hispanic-Serving Institutions (HIS) grant supports workshops to link counseling faculty with subject matter faculty in a strategic manner. Mt SAC also engages faculty in understanding pedagogy related to broad equity topics, such as universal design, and also connects students with services in the classroom such as tutoring and health services.

Student equity is a College priority which, through the PIE process, is tied to College goals and resource allocations to support student achievement. Critical resources to support this effort are found outside of instruction and student services. For example, the Information Technology (IT) department is engaged with meeting the needs of students and decreasing achievement gaps. The IT team focuses on equity in three ways: integrating and improving of systems and raw data to Research and Institutional Effectiveness, working with divisions and Deans to expedite access to specialized data they need, and prioritizing projects based on their relationship to the equity plan.

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) disaggregates and analyzes learning outcomes and achievement for subpopulations of students. When the College identifies performance gaps, it implements strategies, which may include allocation or reallocation of human, fiscal, and other resources, to mitigate those gaps and evaluates the efficacy of those strategies.

Accomplishments and Outcomes

  • Mt. SAC uses data to note performance gaps and justify resource requests.
  • Mt. SAC disaggregates and analyzes data related to learning outcomes and student achievement.
  • Data is used to measure impact of intervention strategies.

List of Evidence

Title Content
I.A.1-12 Student Equity Plan 2015-16
I.B.1-30 Basic Skills Annual Report 2014-15
I.B.1-52 Student Equity Committee Purpose, Function, and Members
I.B.5-13 PIE for Financial Aid
I.B.5-21 Math Basic Skills Sequence Details
I.B.5-22 English Basic Skills Sequence - Success and Progression at Each Course
I.B.6-1 Pathways to Transfer Progression 2015
I.B.6-2 Pathways to Transfer Survey 2015
I.B.6-3 Pathways to Transfer Winter-Spring 2015
I.B.6-4 Statway Summary Report
I.B.6-5 Statway Progression Report
I.B.6-6 Basic Skills Coordinating Committee Minutes 03-10-2016
I.B.6-7 Institution Set Standards: Job Placement for CTE Programs
I.B.6-8 Completed PIE Web Page
I.B.6-9 Title V Grant Report Year 2, 2015
I.B.6-10 PIE for EOPS
I.B.6-11 PIE for Disabled Students Programs and Services
I.B.6-12 PIE for CalWORKs
I.B.6-13 Basic Skills Annual Report Web Page

I.B.7.

The institution regularly evaluates its policies and practices across all areas of the institution, including instructional programs, student and learning support services, resource management, and governance processes to assure their effectiveness in supporting academic quality and accomplishment of mission.

Evidence of Meeting the Standard

The College Reviews and Evaluates Its Policies

Mt. San Antonio College (Mt. SAC) regularly evaluates its policies across all areas of the institution to ensure their effectiveness in supporting academic quality and the accomplishment of its mission. Board Policies (BPs) and Administrative Procedures (APs) are impacted by federal, state, and local laws as well as College wide procedural changes.

When new laws or changes are enacted or when District needs change, policies and procedures are evaluated and updated to reflect changes. The BP/AP Revision Process, coordinated through the President’s Office, provides an opportunity for College stakeholders to suggest changes to support effective programs and services. Proposed changes to APs and BPs are submitted to the President’s Office for review by President’s Cabinet. The President’s Advisory Council (PAC), which includes representatives from all constituencies, reviews the AP/BP for a first and second reading, classified unions and the Faculty Association weigh in on any negotiable items, and the Academic Senate reviews items related to academic and professional matters. Appropriate committees and councils are included in this review. The Academic Mutual Agreement Council determines if the policy or procedure is an academic or institutional matter. Then, changes to BPs are added to the Board of Trustee’s agenda for review and approval. Each time a change is approved, the dates of review are noted on the documents. BPs and APs that required changes based on new regulations had been sporadically delayed through the review process. Because the process described above is appropriate and necessary, strategies utilized to reduce the time required to complete the process have been successfully implemented. The President’s Office uses OnBase to track all changes and monitor timelines. Cabinet provides input to policy changes to minimize time to approval. Reduction of delays is a common goal (I.B.7-1).

The College Assures Policy Effectiveness

The effectiveness of policies and practices in supporting academic quality and accomplishment of the Mt. SAC mission is measured by how well they are aligned with federal and state requirements and implemented on campus. In addition to legal compliance review, PAC also analyzes the alignment of policies with the College mission and the appropriateness to serve students related to each revision.

Mt. SAC also evaluates effectiveness of policies and practices in supporting academic quality through PIE (program review). PIE users evaluate the process every year. This feedback is compiled, evaluated, and acted upon by the Institutional Effectiveness Committee (IEC) in order to improve the process. Recommendations for improvement to the process, the evaluation method, and the College goals are submitted to PAC for approval to implement. College goals directly relate to the achievement of the College mission. An example of this process in practice is the revision of the 2014-15 PIE documents in response to feedback from the campus community. Changes include three-year program planning aligned with the College goals. The new format is more inclusive to all outcomes planning, including student support services, course outcomes, program outcomes, and institutional outcomes. Use of resources funded through the PIE process is documented within the PIE cycle to allow for transparent reflection on what resources were allocated and how they impacted the program. For example, based on justification of need, two full-time staff members were funded in Research and Institutional Effectiveness (RIE) to provide student equity research support as requested in the 2013-14 RIE unit PIE.

Academic quality for specialized instructional programs at Mt. SAC is assured through partnerships and good standing with external accrediting agencies. Through the accreditation Self Evaluation process, discipline specific policies and practices are evaluated for efficacy.

  • The Associate of Arts Sciences Interior Design degree is accredited by the National Kitchen & Bath Association.
  • The Air Conditioning and Refrigeration Program is the only Partnership for Air-Conditioning Heating, Refrigeration Accreditation certified program in California.
  • Airframe and Aircraft Powerplant Maintenance is certified by the Federal Aviation Administration to train new technicians to enter the industry as general aviation technicians, corporate aircraft technicians, or commercial aircraft technicians.
  • The Alcohol and Drug Counseling program is accredited by the California Association for Alcohol/Drug Educators.
  • The Associate Degree Nursing, Licensed Vocational Nursing to RN, and Psychiatric Technician to RN programs are approved by the California Board of Registered Nursing.
  • The Emergency Medical Services program is nationally accredited by the Committee on Accreditation of Allied Health Education Programs and by the Committee on Accreditation of Emergency Medical Services Professions.
  • The Fire Technology program is accredited by the Office of the State Fire Marshall.
  • The Histologic Training program is accredited by the National Accrediting Agency for Clinical Laboratory Sciences.
  • The Mental Health Technology Psychiatric Technician program is accredited by the Board of Vocational Nursing and Psychiatric Technicians.
  • The Phlebotomy program is accredited by the State of California, Health and Human Services, Department of Public Health, Licensing and Certification.
  • The Respiratory Therapy program is accredited by the Committee on Accreditation for Respiratory Care.
  • The Radiologic Technology program is accredited by the Joint Review Committee on Education in Radiologic Technology.
  • The Mt. SAC School of Continuing Education is accredited by the Accrediting Commission for Schools, Western Association of Schools & Colleges.
  • The Certified Nursing Assistants (CNA) and Certified Home Health Aides (CHHA) programs are both licensed through the State of California, Health & Human Services, Department of Public Health, Licensing & Certification Section.
  • The Registered Veterinary Technology program is accredited by the American Veterinary Medical Association.
  • The High School Diploma program courses within the Adult Basic Education Department have achieved a-g UC and CSU status and National Collegiate Athletic Association (NCAA) approval.

The creation of an Accreditation Steering Committee (ASC) was based upon College-wide feedback from the 2010 Self Evaluation, which highlighted a need for a more inclusive accreditation process. The Academic Senate provided the Vice President of Instruction with recommendations for improving the process which also included an ASC. As a result, ASC was formed in fall 2013 and includes all representative constituencies. Its purpose is to guide the College in the accreditation process and facilitate communication with all constituency groups. As a result of the work of this committee, accreditation processes for the 2017 Self Evaluation have been more broadly communicated and employees are more engaged (I.B.7-2 pg. 11-13, I.B.7-3, I.B.7-4).

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) regularly evaluates its policies and practices across all areas, including instructional programs, student and learning support services, resource management, and governance processes to assure their effectiveness in supporting academic quality and accomplishment of the College mission.

Accomplishments and Outcomes

  • Mt. SAC regularly evaluates its policies and practices across all areas of the institution as evidenced by its accreditation standing with specialized accrediting agencies.
  • Mt. SAC evaluates effectiveness of policies and practices in supporting academic quality through PIE (program review).
  • Academic quality for specialized instructional programs at Mt. SAC is assured through partnerships and good standing with external accrediting agencies.
  • Accreditation processes for the 2017 Self Evaluation have been more broadly communicated, and employees are more engaged.

List of Evidence

Title Content
I.B.7-1 AP2410 Process for Revision of Administrative Procedure or Board Policy
I.B.7-2 Board Minutes - Student Housing January 2015 pg. 11-13
I.B.7-3 Accreditation Steering Committee Documents
I.B.7-4 Accreditation Steering Committee Purpose, Function, and Members

I.B.8.

The institution broadly communicates the results of all of its assessment and evaluation activities so that the institution has a shared understanding of its strengths and weaknesses and sets appropriate priorities.

Evidence of Meeting the Standard

The College Communicates Assessment and Evaluation Activities

Transparent communication of assessment and evaluation activities at Mt. San Antonio College (Mt. SAC) is an expectation modeled by the President. Weekly Board Reports posted on the President’s Office webpage share current educational trends and report on campus initiative progress. Community presentations done by the President report out the status of campus initiatives. During FLEX Day, the College President and the faculty outcomes coordinator present awards to groups across campus who have excelled at outcomes assessment work. President’s Awards for Excellence and Innovation in Teaching and Learning began in 2011. Presentation of awards, along with a narrative of each award, provides the faculty with best practices examples from their colleagues. Each award comes with a framed certificate as well as a fiscal award. In 2015, awardees were Adult Basic Education, Theater, and World Languages. Since the awards began, the Outcomes Committee has noted an improvement in the quality of submissions per the rubric they use to evaluate the awards. After each awarding process, the Committee evaluates its process and improves upon it (I.B.8-1). FLEX Day also includes presentations focused on outcomes assessment during the breakout sessions (I.B.8-2).

Department chair workshops provide opportunities for chairs to engage in higher level discussions about outcomes assessment. In 2015-16, sessions were offered to allow departments to map the College institutional level outcomes (ILOs) to their programs and courses (I.B.1-24). Departments are also offered financial support to compensate adjunct faculty who wish to engage in outcomes assessment discussions. Follow-up responses from each department documenting how the money was used and the learning that occurred are collected (I.B.4-9, I.B.4-7).

As part of the four-year course review process, the Outcomes Committee completes an evaluation of student learning outcomes and assessment processes for selected courses. Using a rubric, outcomes are evaluated and feedback is provided to the department chairs to pass on to their faculty members. Feedback is directed toward continuous quality improvement. Courses that have not undergone outcomes assessment in the past four years are given one more year to complete. If not done, the course is deactivated as per the curriculum process (I.B.2-2, I.B.5-10).

Evaluation of The College's program level outcomes is available on the Mt. SAC webpage which provides current and potential students and the public an opportunity to see outcomes that are being regularly assessed, criteria for success, and whether the criteria was met. Assessment of the way the evaluation of course and program level outcomes information is presented is ongoing via the Outcomes Committee. The catalog also includes the program learning outcomes (I.B.1-34, I.A.4-1 pg. 46-47).

The College Demonstrates Shared Understanding of Assessment and Evaluation Strengths and Weaknesses

Since learning outcomes assessment began at Mt. SAC, understanding of outcomes assessment has increased. There are numerous pieces of evidence that demonstrate the current understanding and evaluation of outcomes assessment and the on-going discussions.

Department minutes provide evidence of a shared understanding of outcomes assessment. On an ongoing basis, Student Services regularly evaluates its progress toward assessing its effectiveness in providing services to students. Numerous Student Services programs are required to regularly evaluate and report outcomes to state and federal agencies. At semi-annual planning retreats, Student Services managers assess their progress in meeting students’ educational and developmental needs. The Library and Learning Resources Division has annual retreats that focus on improving their outcomes assessment work. Through these meetings, they provide opportunities for collegial evaluation of their work (I.B.8-3, I.B.2-17 pg. 17-18, I.B.2-18 pg. 15-17).

As noted previously, the Outcomes Committee evaluates outcomes assessment as part of the four-year course review process. These findings provide feedback to the chairs as to the quality of the outcomes work and suggestions for improvement. President’s Awards for Excellence and Innovation in Teaching and Learning is another example of how the College evaluates its outcomes assessments, determines its best work, and celebrates it.

The College Sets Priorities Based Upon Assessment and Evaluation Results

Based on assessment results and evaluation activities, Mt. SAC has a shared understanding of its strengths and weaknesses and sets appropriate priorities. Over the years, more resources were added to support outcomes assessment based upon the work being done, evaluation of the quality of this work, and the need for continuous quality improvement. Prioritization was given for adjunct faculty to be part of the outcomes process, for recognition of outstanding outcomes assessment, for mapping programs and courses to institutional learning outcomes, and for off-campus professional development.

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) broadly communicates the results of all of its assessment and evaluation activities so that the College has a shared understanding of its strengths and weaknesses and sets appropriate priorities.

Accomplishments and Outcomes

  • Transparent communication of assessment and evaluation activities at Mt. SAC is an expectation modeled by the President.
  • Since learning outcomes assessment began at Mt. SAC, understanding of outcomes assessment has increased.
  • Based on assessment results and evaluation activities, Mt. SAC has a shared understanding of its strengths and weaknesses and sets appropriate priorities.

List of Evidence

Title Content
I.A.4-1 Catalog 2016-17 pg. 46-47
I.B.1-24 Summary Report for ILO Mapping 8-2016
I.B.1-34 Student Learning Outcomes Webpage
I.B.2-2 AP4020 Program and Curriculum Development
I.B.2-17 VP PIE Student Services Outcomes 2015-16 pg. 17-18
I.B.2-18 PIE Library and Learning Resources Outcomes pg. 15-17
I.B.4-7 Outcomes Committee Annual Reports 2013-14
I.B.4-9 PIE Appendix A Student Services Division SLOs
I.B.5-10 Qualitative Outcomes Review Reports 2013
I.B.8-1 Presidential Awards for Excellence and Innovation in Teaching and Learning
I.B.8-2 FLEX Day Program Fall 2015
I.B.8-3 Associated Students Minutes 2015-2016

I.B.9.

The institution engages in continuous, broad based, systematic evaluation and planning. The institution integrates program review, planning, and resource allocation into a comprehensive process that leads to accomplishment of its mission and improvement of institutional effectiveness and academic quality. Institutional planning addresses short- and long-range needs for educational programs and services and for human, physical, technology, and financial resources. (ER 19)

Evidence of Meeting the Standard

The College Employs Systematic, Broad-based Evaluation, Assessment, and Planning

Mt. San Antonio College (Mt. SAC) uses systematic, broad-based evaluation, assessment, and planning in its operations in order to accomplish its mission which includes improving institutional effectiveness as well as academic quality. Its Process Map of Integrated Planning is used to guide this work (as shown earlier).

Detailed examples of systematic, broad-based evaluation, assessment, and planning work can be found in the College Planning for Institutional Effectiveness (PIE) documents, and the Strategic Plan.

Program Review (known as Planning for Institutional Effectiveness)

Program review, known as Planning for Institutional Effectiveness (PIE), is how the College evaluates each program’s effectiveness. PIE is a holistic approach that uses qualitative and quantitative data, integrated planning, learning outcomes assessment, and resource allocation. The College is unified through its demonstrated connection to the College mission. Driven by the California Master Plan for Higher Education, revised by the President's Advisory Council, and approved by the Board of Trustees, it informs all planning and assessment.

PIE uses a bottom-up process for integrated planning. The multiple-step process is as follows:

  1. All departments/units complete PIE each year using a systematic, inclusive process. Mt. SAC requires a full program review each year (I.B.9-1). Completed department/unit PIEs are sent to the immediate manager. In 2015-16, sections of program review were:
  • Introduction
      1. Institutional Mission
      2. College and Unit Missions
      3. College Themes and Goals
  • Section One: Where We Are: A Summary and Analysis of the Current Year 2015-16
      1. Summary Context – Unit Goals
      2. Notable Achievements
      3. Closing the Loop – Alignment and Progress on College Goals
  • Section Two: Where We Are Going: Planning for the Next Two Years of this Cycle
      1. Planning Context – Unit Goals Assessed and Revised
      2. Conditions and Trends
      3. Institution-Set Standards (ISS)
      4. Plans, Activities, and Resources
  • Section Three: Recommendations: Improving the Planning Process
      1. Additional Information Needed
      2. Suggestions for Improving the Planning Process
      3. Contributors
  1. Managers synthesize all unit/department PIEs and provide their managers’ PIEs to their Vice Presidents or President, in cases where the unit reports directly to the President, such as Marketing. Embedded in the PIE, managers also provide progress reports on their Strategic Plan objectives which are reviewed by the Institutional Effectiveness Committee (IEC) (I.B.9-2).
  2. Vice Presidents and the President synthesize all of their area PIEs and provide a Vice President/President level PIE to the IEC.
  3. The IEC synthesizes the following information into the annual PIE Summary
  • Vice Presidents’ and President’s PIEs (I.B.9-3)
  • Strategic Plan progress
  • Within the PIE Summary, the IEC evaluates how well the College is achieving its College goals and thus its mission as well as how well the PIE process is working. Based on this evaluation, recommendations are made to President’s Advisory Committee to improve the College goals and PIE process (I.B.9-4).

Joint meetings of the IEC and the Budget Committee are held to align and re-evaluate alignment of calendars for program review, Strategic Plan, and budget (I.B.9-5). The Integrated Planning and Budgeting Process Calendar allows for budget allocation to more closely align timewise to the most recent program review, thus improving institutional effectiveness and academic quality (I.B.9-6, I.A.3-1 pg. 21).

It is through the totality of the PIE process that the College demonstrates its systematic, broad-based evaluation, assessment, and planning as it relates to improving institutional effectiveness and academic quality. Examples of an improvement to PIE over the years include the use of an Adobe fillable form, re-alignment of the content to make it easier to understand, and in 2015-16, an addition of a section asking units/departments to close the loop by reflecting on how useful it was to have received the resources they did receive based on their last PIE.

Strategic Plan

The President’s Advisory Council (PAC) led improvements to The College's Strategic Plan. Through this process, the College developed a more robust integrated plan that included all Vice Presidents’ teams that led to improved effectiveness and academic quality (I.B.9-7). Major components of the Strategic Plan include:

  1. Introduction
  2. Planning for Institutional Effectiveness and Integrated Planning
  3. College Mission, Vision and Core Values
  4. Strategic Objectives
  5. College Goals and Strategic Objectives
  6. Appendices
    1. Appendix A - Process Map of Integrated Planning
    2. Appendix B – Integrated Planning and Budgeting Process Calendar
    3. Appendix C - Mt. SAC’s History
    4. Appendix D - Recent History of Integrated Planning at Mt. SAC
    5. Appendix E – Definitions (I.B.9-9)

The IEC operationalized and improved the Strategic Plan 2015-17 using the following steps:

  • In spring 2015, the committee recommended to President’s Cabinet a specific Vice President to be responsible for each strategic objective.
  • In summer 2015, IEC included the Strategic Plan objectives in managers’ PIE for their progress input (I.B.9-2).
  • In fall 2015 and spring 2016, IEC reviewed Strategic Plan progress (I.B.9-8).
  • In spring 2015 and fall 2015, the IEC chair lead discussions during expanded President’s Advisory Council meetings to review the improvement of the Strategic Plan, its operationalization, and its progress.

The IEC also re-evaluated the Strategic Plan, its progress, and how it could be improved to allow the College to improve its institutional effectiveness and academic quality. Through the College PIE summary, IEC recommended changes to the Strategic Plan, such as reducing the number of indicators being tracked (I.B.9-9).

Resource Allocation

Resource Allocation begins in the unit/department PIE. Each Vice President and the President have a system for prioritizing these requests. Based on the final prioritization process, each brings their priorities to President’s Cabinet for College wide prioritization (I.B.9-10).

President’s Cabinet makes the final recommendation on which requests to fund and when. It is worthy to note that there are many different funding sources which have different requirements that are taken into consideration. For example, instructional supplies funded by California Lottery funds have specific funding stipulations which are adhered to. Results of the President’s Cabinet meetings are published and thus transparent.

Allocation of resources may be necessary when policies and procedures are approved. The request for allocation must be included in the annual Planning for Institutional Effectiveness (PIE) document. An immediate needs request is used when unexpected, urgent needs require action (I.B.9-11).

The College Accomplishes Its Mission through Integration of Program Review, Planning, and Resource Allocation

Employees at Mt. SAC participate in program review (Planning for Institutional Effectiveness) which includes integrated planning and resource allocation driven by The College's mission.

Planning for Institutional Effectiveness (PIE) is a continuous, broad based, systematic, comprehensive, integrated planning and evaluation process that drives the use of resources (human, physical, fiscal, technological) and leads to the accomplishment of the College mission and the improvement of institutional effectiveness and academic quality.

Mt. SAC’s PIE process requires the evaluation of accomplishments, the examination of the impact of internal and external forces on the institution, the evaluation of the effectiveness of data usage, and evaluation of student learning outcomes assessment and its impact on student learning. In addition, it requires thoughtful planning into the future focused on how to improve student learning, student achievement, and support services. There is alignment with planning and resource allocation.

PIE is central to resource allocation. Resource requests that are predictable are required to be in program review in order to qualify for resources. For example, PIE facilities requests in 2014-15 become the 2015-16 facilities project lists. Beginning in the 2015-16 PIEs, units/departments are required to document how the resources they obtained because of their program review were instrumental to their success; this is known as closing the loop (I.A.1-26).

The College's Institutional PIE Summary provides evidence of how the College is meeting its mission via the multitude of accomplishments that are linked to the College goals (I.B.1-58).

The College Improves Institutional Effectiveness through Integration of Program Review, Planning, and Resource Allocation

The College's Institutional PIE Summary provides evidence of how the College is improving its institutional effectiveness. Accomplishments are aligned with College goals and the Strategic Plan.

Every year, each unit/department notes its accomplishments and how they align with the College goals. While each accomplishment could align with more than one goal, they are asked to align each to the most appropriate single goal. Over the many years of evaluating PIEs, it has become clear that it is important to see the one-to-one relationship to better demonstrate improvement in institutional effectiveness. Each year, the Institutional PIE Summary indicates the impact the College is having on each College goal and thus on its mission. For example, in past years there were fewer than expected accomplishments related to diversity and equity, while more recently, The College's work is more clearly focused on this area of institutional effectiveness. In addition, the resource allocation process includes funds for student equity which also clearly demonstrates The College's improvement in this area.

The College Improves Academic Quality through Integration of Program Review, Planning, and Resource Allocation

Faculty continually improve academic quality through their iterative improvement process in the classroom. Outcomes assessment provides a way to document this improvement in a systematic process. Faculty assess outcomes at the course, program, and institution level. Faculty also complete mapping of outcomes to show alignment of each level of outcomes assessment. The impact of outcomes assessment work and plans for the future are noted in the unit/department’s program review. Resource requests require justification which frequently comes from outcomes assessment. In 2015-16, units/departments discussed the impact of obtaining those resources in order to close the loop.

There is clear improvement of academic quality through outcomes assessment as demonstrated by the Outcomes Committee’s work. The committee reviewed the outcomes work using the College repository, TracDat. Reviewing beginning outcomes work versus more current outcomes work shows a clear improvement in the quality of outcomes assessment conducted by the faculty. Also, within this work were clear examples of how students were achieving their outcomes as well as how faculty were improving their curriculum and pedagogy. It is through the aforementioned that the College can clearly state that there is improvement of academic quality through integration of program review, planning, and resource allocation (I.B.9-12).

Other evidence of improving academic quality can be found in program reviews for Student Services, Library and Learning Resources, and the Writing Center. Each demonstrates its use of program review, integrated planning, and resource allocation to improve academic quality (I.B.2-17, I.B.2-18, I.B.9-13).

Career Technical Education programs can apply for Perkins funds to improve academic quality of their programs (e.g. computers, foods lab/kitchen). Additional funds are also available for some programs through the Title V Hispanic-Serving Institutions grant. Through these funds, programs such as Hospitality Restaurant Management (HRM) are able to offer students competitions to improve their learning outcomes and employability. The Title V grant recently purchased equipment that allowed students to view the instructor's actions on a large screen in the classroom instead of merely huddling around the work area hoping to see what their instructor was demonstrating. This past fall 2015 term, 153 students took HRM classes with a success rate over 80 percent. Funds from the Title V grant have also helped purchase equipment for the Radiology and Respiratory Technician programs as well as providing the funding to extend tutoring and open lab hours. Students survey results validate the value of these extended lab hours, and the success rates since implementing these improvements speak for themselves. The overall success rate for these students is well over 95 percent with many classes attaining 100 percent success rates.

The School of Continuing Education (SCE) strongly focuses on academic quality and continually strives to improve quality based on the unique needs of the students. Twice annually, the Continuing Education Advisory Team addresses student needs by reviewing the division’s program review data, student profile data, and accreditation Action Plan. Faculty and staff from each department identify student successes, areas of need, and areas of improvement. Utilizing department and division process and various sources of funding, improvements in academic quality are addressed. For example, the advisory team identified in the area of improving student learning that there is a need for supplemental learning, creation of new courses, and contextualized coursework/programs. In response to these needs, Adult Basic Education and English as a Second Language have been able to access basic skills funding for supplemental learning, in-class tutoring, and curriculum development. Outcomes have demonstrated a commitment to improving academic quality for SCE students (I.B.1-30, I.B.9-14).

The College's Planning Addresses Short- and Long-term Institution Needs

Institutional planning at Mt. SAC addresses short- and long-range needs for educational programs and services and for human, physical, technology, and financial resources. The need for management positions must first be documented in PIE, reviewed by the appropriate Vice President, and reviewed in President’s Cabinet before being sent to Human Resources (HR) for posting. The process for faculty positions is:

  • New or replacement faculty positions are extracted from PIE documents and forwarded to the Vice Presidents of Instruction and Student Services and to the Academic Senate. Each Vice President prioritizes the faculty positions and then they combine their lists. The Academic Senate independently creates a prioritized list.
  • The Vice Presidents and the Academic Senate bring their lists to Academic Mutual Agreement Council and collaboratively determine their joint prioritization list.
  • The list is sent to the College President who decides the number of faculty positions to be hired. The President may re-arrange the prioritized list, but has not done so in the past few years.

The need for classified staff positions is also justified in unit/department PIE documents, requested by the department/unit, reviewed by the respective Vice President, reviewed and approved by President’s Cabinet, and sent to HR.

Technology resource planning begins with the College Technology Master Plan which is reviewed and updated every three years by the Information Technology Advisory Committee. Information technology (IT) receives a list of technology requests from the PIE documents to aide in technology resource planning. Using program review allows IT to stay engaged with employees’ needs and to maintain currency with fiscal needs.

Financial resource planning stems from the Budget Committee which has responsibility to oversee the process for allocation of resources. Allocations are based on College priorities (e.g., Facilities Plan, Student Equity Plan, Student Success and Support Plan) as determined by the unit/departments’, managers’, Vice Presidents’, and President’s prioritization processes.

Analysis and Evaluation

Mt. San Antonio College (Mt. SAC) engages in continuous, broad based, systematic evaluation and planning. The College integrates program review, planning, and resource allocation into a comprehensive process that leads to accomplishment of its mission and improvement of institutional effectiveness and academic quality. Institutional planning addresses short- and long-range needs for educational programs and services and for human, physical, technology, and financial resources.

Accomplishments and Outcomes

  • Since learning outcomes assessment began at Mt. SAC, understanding of outcomes assessment has increased.
  • The College integrates program review, planning, and resource allocation into a comprehensive process of its program review called Planning for Institutional Effectiveness.
  • All resource allocation must be driven by program review to obtain the resource.
  • College goals, aligned with accomplishments in program review indicate how well the College is achieving its mission and improvement of institutional effectiveness and academic quality.
  • Planning for Institutional Effectiveness allows the College to plan to address its short- and long-range needs for educational programs and services and for human, physical, technology, and financial resources.

List of Evidence

Title Content
I.A.1-26 Facilities Master Plan
I.A.3-1 Strategic Plan 2015-17 pg. 21
I.B.1-30 Basic Skills Annual Report 2014-15
I.B.1-58 PIE Summary College-wide 2014
I.B.2-17 VP PIE Student Services Outcomes 2015-16
I.B.2-18 PIE Library and Learning Resources Outcomes
I.B.9-1 PIE Forms - Unit Department 2014-15
I.B.9-2 PIE Forms - Manager 2014-15
I.B.9-3 PIE Forms - VP 2014-15
I.B.9-4 Completed PIE Web Page 2014-15
I.B.9-5 PIE Summary Committee Discussions 4-13-2016
I.B.9-6 Integrated Planning and Budgeting Process Calendar
I.B.9-7 Memo to PAC on Strategic Planning Process
I.B.9-8 Strategic Plan Progress Report from TracDat
I.B.9-9 PIE Summary Annual
I.B.9-10 President's Cabinet New Resource Allocations Table
I.B.9-11 Immediate needs Request 2016-17
I.B.9-12 Four-Year-Review and Outcomes Assessment 09-17-2015
I.B.9-13 Writing Center PIE 2015
I.B.9-14 SCE Advisory Team Notes 4-16-15