Theme III - Improving Student Success in Graduation and Learning

The theme of student success has led to research and reflection on a number of areas related to students. On the one hand, we examined graduation and retention rates, looking at this more holistic level of student success. However, we also examined this issue by looking at what students have learned at the programmatic level, in others words, student learning outcomes. In addition, we included two curricular efforts that have been in progress for a number of years that we believe will have a major impact on student learning: the Graduation Requirements Task Force (GRTF) Initiative and the Writing Across the Curriculum/Writing in the Discipline Initiative (WAC/WID). Each of these four areas became topics of focus for the third theme of the WASC review.

Capacity Issue: Are different populations of students succeeding at similar or different rates?

The University began focusing attention on graduation and retention in 2005 within the context of a CSU system wide initiative. At that time the Facilitating Graduation Task Force (FGTF) was created and charged with making recommendations to improve undergraduate graduation rates. [CFR 2.3, 2.4, 2.7] The recommendations from this group included improving the frequency and type of advising on campus; improving the quality and amount of communication with students; addressing curricular bottlenecks; and, overall, creating a "culture of graduation." At the time of the FGTF review, the University did not have an office of Academic Institutional Research, and much of the data for a more granular analysis of graduation and retention were not available. Since then, we have developed this office within Academic Planning and Development, making a more careful and in-depth analysis possible for the WASC review. [CFR 4.5]

Graduation, Retention and Time to Degree

Although there are many ways to define and analyze success, the WASC Graduation and Retention Subcommittee chose to focus on the big picture by examining graduation and continuation rates and other data broken down by gender, ethnicity, Pell Grant status (to identify financial need), first-time freshmen vs. transfer students, remedial vs. college-ready students, and college/department of major.

In 2005 the overall six-year graduation rate at SF State was 43.3%. For non-under-represented minority students the rate was 46.2%, and for under-represented minorities[6] it was 34.1%. One difficulty historically in terms of determining whether our students were successful was that time to degree (TTD) data have been reviewed and judged based on the four- to six-year graduation rates of full-time, first-time freshmen. The CSU has recently expanded its data collection to look at eight- and nine-year graduation rates; this shows that our students do indeed persist, and given enough time, most who persist beyond one year do graduate. There are still discrepancies along race and gender lines – for example, 75% of female transfers graduate within nine years while just 66% of male students do, and fewer numbers of African-American students are graduating than the rest of the population (61.5% graduate within nine years). In terms of first-time freshmen, within nine years, men and women graduate at almost equal rates, although far more women graduated in a shorter time. While we are about average on graduation rates in the CSU system, the entire system falls behind considerably when compared with other universities in our Carnegie category. Because of this disparity, the Chancellor's Office has once again taken up the charge of improving graduation rates with a major new initiative, Facilitating Graduation 2 (FG2), which is a follow-on to the 2005-06 project. The purpose of the current initiative is to increase the six-year graduation rates of first time-full time freshmen and transfer students at SF State by 8% by 2015, and to decrease the gap between URM and non-URM students by 50% by 2015.

To respond to this initiative, in Fall 2009 the University created an FG2 Task Force, which developed a five-year plan for the initiative. The plan includes an extensive set of activities to be implemented over the next five years, and these activities will become a part of the WASC Educational Effectiveness Review. We are already beginning to see an increase in six-year graduation rates between the 2002 and 2003 cohorts, which had a 1.7% increase. This increase is likely related to the changes that were put in place during the first Facilitating Graduation Initiative. As we scale up our efforts in this area, continue to collect data, and target specific populations and high impact activities, we believe that the collective result will be a further increase in graduation. [CFR 2.2, 2.3, 2.4]

To begin this effort, we recently disaggregated the graduation rates by college and by department for both first-time-freshmen and transfer students, and further disaggregated the data by URM and non-URM students. We have distributed this information to faculty chairs, and departments are already beginning to compare their rates and set benchmarks. We also plan to implement a migration study over the next year to determine where students go when they change majors and at what point they change majors, particularly in departments where graduation rates are low. Although much works remains to be done, we believe that taken together, all of these activities will have an impact. [CFR 4.4, 4.6]

In addition to the quantitative analysis, the focus group discussions helped the subcommittee develop questions for future use in campus surveys that will lead to an understanding of what factors contribute to successful students. Some of our questions were inspired by a WASC ARC session attended by the subcommittee chair in April 2008 entitled "Achieving the Dream: Student Success through Evidence." These questions were designed to explore student beliefs about their ability to complete their degree. They included institutional factors such as availability of classes and flexibility of course choices; personal factors such as support from family and friends; university community support such as faculty and other university personnel; and co-curricular activities such as community service, study abroad, independent study, athletics and student clubs. The new questions have been added to the undergraduate exit survey, and student responses will provide direction in focusing on things that matter with regard to student success. [CFR 4.5, 4.7]

Based on all of our analysis we can make four observations with regard to the overall/university-wide graduation rates over time:

  1. Time to degree (TTD) for both freshmen and transfers improved between 2003-04 and 2008-09 for virtually all populations: Female/male; Pell Grant/Non-Pell Grant recipients; ethnicity; and regular/exceptional admission. Possible reasons for this change are that fewer FTF students are requiring remediation, SF State's facilitating graduation efforts are making a difference, mandatory advising initiatives at specific points of students' college careers are having an effect, and the high cost of living and increased fees give students incentives to take more units each semester.
  2. Non-Pell Grant recipients on the whole graduate sooner than Pell Grant recipients. Possible reasons for this difference are that non-Pell Grant students presumably have more parental support and are able to attend full-time. In addition, some grant students may not feel the rush to graduate since they are receiving financial aid.
  3. There were more than 1000 more graduates in 2008 than in 2003, despite increasing enrollments and decreasing budgets.
  4. Once students begin a major, although time to degree might differ from one major to another, different populations are just as likely to persist and graduate in a particular major.

In general, many factors are leading to decreasing time to degree, and the budget crisis is certainly another important factor. Recent budget realities have forced us to review our priorities and put limits on what students are allowed to do because we can no longer be all things to all people. The positive side of this predicament is that it seems to have helped students focus more on making progress to degree. The negative side is reflected in some comments by some students in focus groups; some feel rushed to graduate and some feel that we are more concerned with increasing the numbers of graduates than with their educational experience. We are clearly in a period of transition with regard to the amount of access that students have to higher education and the focus they need to exercise in making decisions. In response to the budget crisis, the University has implemented a set of enrollment and curriculum controls. Other colleges and universities have practiced some of the controls we have put into effect for many years:

 

  • creating a maximum number of times a student can repeat or withdraw from a course
  • requiring high unit seniors to make timely progress toward the degree
  • only allowing double majors or minors if a student can complete all requirements within a reasonable number of units

Although commonplace elsewhere, these changes have affected how faculty and students alike view the educational experience. One positive outcome is that these changes have made us pay more attention to whether students are making progress toward a degree. Some students who might previously have fallen through the cracks are getting the assistance they need because we are forcing them to do so. Hopefully, this will increase the number of successful students. [CFR 2.12, 2.13, 2.14]

Capacity Issue: Are resources being deployed appropriately to ensure that different populations succeed at similar rates?

A great variety of resources exist for students at SF State, and many of these are aimed at specific populations of students. The Educational Opportunity Program is a program that is committed to increasing the academic excellence and retention of California's historically underserved students (low income, first-generation college) through its academic support programs. Student Support Services (SSS) is a federally funded TRIO program, providing intensive academic advising, tutoring, workshops, and financial assistance to approximately 160 students during their first two years of college. The Disability Programs and Resource Center (DPRC) provides resources, education, and direct services so that people with disabilities have a greater opportunity to succeed at SF State. DPRC serves students with mobility, hearing, visual, communication, psychological, systemic (HIV/AIDS, environmental illness, etc.), and learning disabilities. The Advising Center is a university service staffed by professional counselors, interns, and peer advisors committed to providing guidance and information to help undergraduate students enjoy a successful college experience. In addition, several colleges offer student resource centers to assist declared majors with academic issues (for example, BSS Student Resource Center, College of Business Student Services Center, CHHS Student Resource Center). The Advising Center houses programs for new students, which offer orientations for first-time freshmen and transfer students and their family members. Each year over 10,000 people participate in orientation. In addition, the Advising Center staff monitors and advises students needing remediation in order to improve retention.

The Campus Academic Resource Program (CARP) and the Learning Assistance Center (LAC) provide campus-wide tutoring services. Some departments and programs offer major- or course-specific tutoring.

As mentioned in Theme Two, we have made great strides as a campus to address the needs of the increasing numbers of younger, residential students. However, we need to remember that a large number of what we used to think of as "non-traditional" SF State students still attend; they live off campus and commute, have family and work obligations, and are perhaps part time and possibly older or re-entry students. We do not know whether we are serving the needs of these students. The nine-year continuation data now provided by the CSU and the disaggregated campus data may help us at least see whether they are graduating. [CFR 4.5]

As the work for Facilitating Graduation 2 and the WASC EER proceed, we will be assessing these areas and attempting to define where we can target changes that will make a difference. [CFR 2.1, 2.2, 2.3]

Recommendation 14: The University should continue the work of the Facilitating Graduation Initiative 2 as planned and as required by the CSU. [CFR 4.1, 4.3, 4.7]

Student Success in Learning

Student Success must include not only how many students graduate and how quickly they graduate, but also what they learn along the way. Three areas of Theme 3 focused on what students learn: the Graduation Requirements Task Force (GRTF), Writing Across the Curriculum/In the Discipline (WAC/WID), and Student Learning Outcomes Assessment.

Graduation Requirements Task Force

The current general education program at San Francisco State University took effect in 1981. Since then there have been substantial changes in knowledge, curricula, and faculty, yet general education has remained the same. In Fall 2005 the SF State Academic Senate passed a policy (F05-237) initiating a reconsideration of all of the requirements for baccalaureate degrees. [CFR 3.11] This policy called for the establishment of a Graduation Requirements Task Force, a self-study of current programs (conducted in Spring 2006), and an external review of those programs (conducted in Fall 2006). [CFR 2.4, 2.7] Those reviewers concluded:

The current GEP has many problems which may be individually correctable but which in their totality may require rethinking of the entire program from the ground up…. Neither the program nor any of its parts has a clear and sufficiently extensive rationale for its purposes…. Thorough program revision requires first a clear and extensive statement of the purposes of general education that is readily available to both students and faculty members and couched in terms that guide teaching and learning…. More than anything else, SFSU needs a refreshed statement of educational purpose that includes both general education and the major as part of a unified whole that provides clear direction for the undergraduate program.

Following this report, the Task Force reviewed the self-study and the external report and agreed with the reviewers' recommendation that the first step was to develop broad goals for the baccalaureate. In January 2007 at the SF State University Retreat, the Task Force held multiple meetings to invite campus input about the goals. Task force members also reviewed baccalaureate requirements at multiple universities identified as exemplary programs and studied documents from AAC&U, including College Learning for the New Global Century: A Report from the National Leadership Council for Liberal Education and America's Promise (LEAP).

Capacity Issue: How should the University respond to the findings of the 2007 General Education review?

In March of 2008, the Academic Senate approved a one page educational goals statement that had been vetted on the GRTF website for response from the entire university community. [CFR 2.4, 2.5]

Throughout the spring of 2008, the Task Force brainstormed potential curricular structures for baccalaureate degrees that will be consistent with the new SF State Baccalaureate Learning Goals and will adhere to our own system GE requirements contained in Title V and the Chancellor's Office Executive Order 1033 on General Education. The Task Force then drafted requirements for upper division general education and posted them for campus review.

During the fall of 2009, the Task Force completed the draft requirements for lower division general education and developed student learning outcomes for all areas, as well as a structure for implementing the new package. In addition, during the entire 2009-2010 AY, the University Academic Assessment Advisory Committee studied a variety of plans for assessing general education, and agreed on a course-embedded approach that was designed by the SF State College of Education for their NCATE review. [CFR 2.3, 2.4]

During Spring 2010, the SF State Academic Senate reviewed and debated the entire GRTF Recommendations Report, and finally approved the report at the end of the spring semester. Hallmarks of the revised GE package include double-counting units in GE and the major, an integrated nine units of upper division general education, and student learning outcomes that include campus educational priorities such as social justice, life-long learning, civic engagement, and environmental sustainability.

It is expected that recertification of courses will begin during AY 2010/2011, and implementation of the new program will begin Fall 2011 in time for the EER visit. The assessment cycle for the program is expected to go forward in AY 2012-13.

Recommendation 15: The University should finalize the new baccalaureate degree requirements and begin the certification of courses, implementation, and assessment of the program as soon as possible. [CFR 2.1, 2.2]

Writing Across the Curriculum/Writing in the Discipline (WAC/WID)

Attention to the importance of writing at SF State began 17 years ago with the formation of the Committee on Written English Proficiency (CWEP), which was established to encourage and support broad and effective faculty participation in the teaching and assessment of student writing. [CFR 2.4, 2.7] Toward these ends, the committee sponsors a wide array of services and activities in support of curricula and programs that foster the teaching, learning, and assessment of written English. Over the years, CWEP has established criteria for placing students in English writing courses, standards for assessing whether or not they have accomplished the objectives for writing (i.e. the Graduation Writing Assessment Requirements GWAR) at both the graduate and undergraduate level, and general assessment of the writing program. [CFR 2.4]

Capacity Issue: How should the University move forward to improve writing at both the undergraduate and graduate levels?

In 2006, after much research, evaluation, and discussion, CWEP recommended that the Junior Examination for Proficiency in English Test (JEPET) for the GWAR be replaced by a course requirement for writing in the discipline. Following this recommendation, a Writing Across the Curriculum and in the Discipline (WAC/WID) director was hired to establish and implement the program. Over the past two years, the WAC/WID director has worked tirelessly to educate faculty on the value of WAC/WID and to train them in the skills and resources needed to develop quality GWAR courses. The program is currently being phased in, with 60 GWAR courses now approved and being offered. The WAC/WID director is currently gathering data from surveys and focus groups, and is exploring faculty and student perceptions of the value and challenges of this program. Preliminary indirect assessment data indicate that students have difficulty transferring what they have learned in general education English classes to the skills they need to succeed in writing in the disciplines. These findings are consistent with other programmatic assessment findings about the transfer of knowledge from basic skills to advanced application. As we move forward into the EER, the WAC/WID director will examine data on direct learning by comparing student writing in non-GWAR courses with writing in disciplinary GWAR courses. [CFR 2.3, 2.4, 2.5, 2.6]

Recommendation 16: The University should continue to gather data with regard to the effectiveness of the GWAR courses on student writing, and make adjustments to the WAC/WID program based on the data gathered. [CFR 2.3, 2.4, 2.5, 2.6]

In addition to undergraduate writing, much attention has been dedicated to graduate writing. When the SF State program review guidelines were revised in the 6th cycle to focus on graduate education, those guidelines included specific writing requirements at the graduate level. Two levels of writing assessment are now required of all graduate programs. Level 1 assessment of graduate writing requires all departments to include an assessment of writing at the time of admissions for all students. Departments may use the GRE writing score, a TOEFL writing exam score, or a score on an assessment that has been developed by the department. Level 2 graduate writing assessment requires that departments conduct an additional writing assessment during students' graduate experience.

Since the 6th cycle of program review began in 2008, all departments that have been reviewed have successfully initiated the Level 1 graduate writing requirement. While all departments have also initiated the Level 2 requirement, many of them have been using the thesis for this measure. The SF State Graduate Council has been working with departments to develop earlier measures for Level 2 to insure that students who need writing development will receive assistance before they begin the culminating experience. Graduate departments have also been appealing to the WAC/WID director for assistance with graduate writing. [CFR 2.3, 2.4, 2.5, 2.6] The director is working on the implementation of the undergraduate program, and will work with these departments after that. Once the program has been fully implemented, graduate level writing can be phased into the responsibilities of WAC/WID.

Recommendation 17: The University should consider requiring graduate departments to implement the Level 2 writing requirement before the culminating experience begins, and assess graduate writing at Level 2 across the campus. [CFR 2.3, 2.4, 2.5, 2.6]

Student Learning Outcomes Assessment

No WASC report would be complete without a thorough presentation and analysis of student learning outcomes. Three levels of the assessment of student learning exist at San Francisco State University: Institutional Assessment, Academic Program Assessment, and Student Affairs Assessment.

Institutional Assessment

The Office of Academic Institutional Research (AIR) largely carries out data gathering and analysis for institutional assessment. Five types of measures are ongoing and were used for the WASC CPR report:

  • NSSE survey
  • FSSE survey
  • Collegiate Learning Assessment (CLA)
  • SF State CUSP II strategic plan
  • Degrees of Preparation survey
  • Links and analysis of these measures can be found in the references. The NSSE and FSSE surveys are conducted every two years, while administration of the CLA is required every year by the CSU Chancellor's Office. The assessment of the CUSP II strategic plan was carried out over a period of years from 2007 until 2010 by committees appointed by Academic Affairs. The Degrees of Preparation survey was conducted in the fall of 2008. SF State was one of 12 university campuses invited to beta test this instrument. The SF State AIR analyzed the results of this survey against all other participating campuses. If the opportunity arises, we expect to participate in the survey again in the future. A great deal of institutional information can also be found on the SF State College Portrait website, which is updated annually. [CFR 2.6, 4.5, 4.6]

Academic Program Assessment

Capacity Issue: Does the current program level assessment process give us the appropriate information regarding student learning?

The assessment of academic programs has three strands:

  • Program Review
  • Scheduled assessment reports
  • General Education assessment

Program review at SF State is implemented in cycles. College by college, programs undergo the program review process. When all programs have completed their reviews, a new cycle begins and the program review policy is revised. The university is currently in its 6th cycle of review. As a result of recommendations from the 2001 WASC review, the 6th cycle of review is focusing on graduate programs. The program review process follows the common pattern of a self-study, followed by a visit from two external consultants, a faculty review committee (Academic Program Review Committee, APRC) that writes a final report, and a Concluding Action Memorandum that is signed by the Provost, the dean, and the department chair. [CFR 2.7] A compendium of all 6th cycle review materials, organized by department can be accessed in the references to this report.

Scheduled assessment reports are reviewed by the Associate Vice President for Academic Planning and Development, who determines report timelines. Some departments submit reports on an annual basis. [CFR 2.7] Departments that have fully assessed all of their learning outcomes are encouraged to conduct mini-studies on a currently relevant learning issue and are given an appropriate timeline for completion. In the past five years, a number of faculty members have attended the WASC conference to learn about assessment, and several faculty have attended the WASC assessment academies. The Office of Academic Planning and Development regularly offers training sessions on how to write outcomes and construct rubrics.

Whether or not we are obtaining the appropriate information on student learning from program level assessment depends on the department and its attitude toward the assessment process. Most departments that have specialized accreditation are well informed with regard to their students' learning and are making changes based on that knowledge. In addition, just as many departments without specialized accreditation have embraced the value of assessment and are also making changes. The results across the campus are interesting.

Most students are learning the content of their majors, and departments are generally satisfied with student learning in the major. The places where issues arise are in the use of basic skills in the discipline and the level of learning the in prerequisite courses needed to move into the major. Writing continues to be an issue across the curricula, and we expect that over time the WAC/WID initiative, together with changes in our lower division composition program, will yield discernible improvement in the quality of student writing. In a variety of fields, the ability of students to analyze a problem or a situation often comes up in departmental assessment reports. A number of programs, notably Computer Science and the College of Business, have turned their attention to soft skills such as teamwork, self-directed responsibility for timeliness, and communication skills as major areas of focus. The focus on soft skills has come as a result of feedback from employers and alumni. [CFR 2.4, 2.6, 2.7]

Needless to say, there are still a few departments that are resistant to assessment, even when they are allowed to develop qualitative, constructivist approaches and methods. Departments that have demonstrated strong pushback on assessment typically have one of two characteristics. In some cases, departments are composed of sub-disciplines that differ to a large degree in their perspectives and values. In these cases, assessment is difficult because the faculty cannot agree on the skills and knowledge that they value, and so determining what to measure is the sticking point. Another impasse occurs in departments that have an individualistic culture. In these cases, it is difficult for the faculty to view the student experience from a programmatic level, which obviously is necessary for program assessment. We are still working with these departments and have made much progress in the past year due to the exigencies of the imminent WASC visit. The Educational Effectiveness summary grid and the most recent departmental assessment reports for all departments can be accessed on the Academic Planning and Development website.

General Education assessment is in the process of revision. As noted earlier in this essay, a new Baccalaureates Requirements package, which includes General Education, was passed by the Academic Senate in the spring of 2010, and implementation is slated to begin Fall 2012. A structure for implementing and assessing the new General Education program was included in that document, along with specification of course expectations and student learning outcomes for each area of GE that are aligned with the Baccalaureate Learning Goals (pp. 4-46). [CFR 2.2] During the 2009-10 academic year, the University Academic Assessment Committee recommended a course-embedded assessment procedure that will allow faculty members to score students electronically on specific learning outcomes as they enter their course grades. The system will allow for this data to be aggregated in order to assess all GE areas as well as the Baccalaureate Learning Goals. [CFR 2.2, 2.4]

Student Affairs Assessment

Capacity Issue: What processes need to be implemented in order to assess the impact of student services on student learning?

The Division of Student Affairs at San Francisco State University launched its inaugural assessment program in April 2009. Prior to that date, units within Student Affairs focused primarily on individual program improvement efforts. In preparation for the WASC Capacity and Preparatory Review scheduled for March 2011, Student Affairs shifted the focus and began a deliberate effort to bring student learning outcomes to the forefront. The move from a student satisfaction/program improvement model to a student learning outcome-based model resulted in a report that describes the assessment plans that were developed and implemented within Student Affairs units during the 2009-10 academic year. [CFR 2.3]

To begin this effort, Student Affairs directors received a two-day training program conducted by Lori Varlotta, Vice President for Student Affairs at California State University Sacramento. The training program helped Student Affairs directors understand the basics of assessment:

  • Aligning the department mission with the missions of the Student Affairs Division and the University. In some cases, the departments needed to craft new mission statements.
  • Identifying the two to three overarching planning goals to broadly frame their work during the upcoming years.
  • Articulating at least three significant student learning outcome and/or program outcomes to achieve for students who participate in their programs or utilize their services.

Directors were asked to develop instruments and collect data to measure the student learning that occurred. As might be expected in an inaugural effort at identifying measurable outcomes, some instruments and assessment approaches proved to be more valuable than others. The second cycle of developing and measuring outcomes will be greatly improved based on the experience gained in 2009-10. The foundation for evidence-based decision-making and outcome-based assessment will be used to create more robust assessment plans for the next cycle in 2010-11.

The report details the assessment plans created by each unit in Student Affairs. Assessment plans for the following Student Affairs departments are included:

  • Athletics
  • Campus Recreation
  • Career Center
  • Disability Programs and Resource Center (DPRC)
  • Educational Opportunity Program (EOP)
  • Enrollment Management
  • Financial Aid
  • Leadership, Engagement, Action, Development (LEAD)
  • Registrar's Office
  • Student Health Services
  • Student Outreach Services
  • Testing Center
  • Undergraduate Admissions
  • University Housing

In Spring 2010, an associate vice president of student affairs was charged with overseeing strategic planning and assessment for the division. This appointment and the process that has been set in place will provide the division with the capacity to continue to evaluate its work in terms of student learning.

Student Academic Services Assessment

Three offices within Undergraduate Studies currently provide advising and tutorial assistance to undergraduate students are the Advising Center, Learning Assistance Center (LAC), and Campus Academic Resource Program.

The Advising Center is a university service staffed by professional counselors, interns, and peer advisors committed to providing guidance and information to help undergraduate students enjoy a successful college experience. The Learning Assistance Center (LAC) provides skill-based tutoring across disciplines by students for students. Through collaboration with programs, departments, and the larger campus community, the LAC works to respond to the diverse needs of SF State students. The Campus Academic Resource Program (CARP) is student run and offers full-service tutoring to all SF State students. CARP services include evening academic support programs, in-class outreach to advertise the CARP services, and workshops to develop student college success skills and to prepare students for a variety of campus exams.

These three academic service programs have conducted narrative reviews of their work for many years. Nonetheless, the Academic Services Managers were included in the two-day assessment training program conducted by Dr. Lori Varlotta. The Dean of Undergraduate Studies concluded that the training could provide the advisors and tutoring professionals with knowledge and skills that would lead to a more learning-centered evaluation of their services. Subsequent to that training, each program developed student learning outcomes and measures, and collected data. The first set of results provides baseline data for the programs. By the time of the EER, we should have data over several years, allowing for programmatic evaluation. Data on the 2009-2010 assessment of academic student services can be accessed in the references to this report.

Recommendation 18: Student Affairs and Academic Student Services should continue their newly developed assessment processes, making programmatic adjustments as indicated by the results. [CFR 2.3]

 

Supporting Documents