2 Results of a Pan-Canadian Survey of First Year Experience and Students in Transition Programming

Steven M. Smith, Tom Brophy, & Adam Daniels

In 1997, a survey was conducted on what was then called the “first-year experience” for new college and university students in Canada. The monograph which followed from that work (Gilbert, et al., 1997) was comprised of seven chapters which focused on various elements of the responses to that survey. In 2022, we undertook to revise that survey and see how the landscape for first-year experience and students in transition programming had changed (or stayed the same) over the intervening 25 years. In this chapter we will describe the survey, our data collection methods, and the results we obtained. You can see a copy of the survey in appendix A.

We began with the original survey as a starting point and revised and added content and questions as recommended by our colleagues. That survey covered topics such as:

• Background information on students collected by institutions
• Measures of student experiences
• Measures of student outcomes
• Programs offered to first-year students, as well as their delivery characteristics (e.g., method, target population, credit, compulsory or voluntary, timing, responsibility centre, cost, etc.)
• The provision of First Year Experience (FYE) programming and characteristics of that programming
• Institutional climate (e.g., role of student societies, technological supports, barriers encountered, etc.)

In developing the 2022 survey, we used most of the items from the original survey and added several additional sections and topics to reflect evolutionary changes in this sector. The survey (see Appendix A for complete survey items) was designed to both replicate the research conducted by Gilbert et al. (1997) and expand upon it to understand how the landscape of FYE and Students In Transition (SIT) programming has changed in the intervening time.

Procedure

We partnered with the Canadian Association of College and University Student Services (CACUSS) to reach out to their members to participate. An initial email went out to all Senior Student Affairs Officers (SSAO) at all CACUSS affiliated colleges and universities. Data collection occurred across two waves. Initial data collection occurred in the Fall of 2023, and subsequent data were collected in the summer and early fall of 2023 . Both waves of data collection used the same survey. 130 participants started the survey and 51 completed the survey. Among those that started the survey there were many duplicates (i.e., more than one respondent from the same institution). Feedback suggested that once the survey was begun the initial respondent sometimes determined that there was someone else who was better able to respond to the survey. Data were collected via Qualtrics software.

Survey

As noted above, the survey was designed to cover the same topics as the original survey but also to enhance the coverage of FYE and SIT programming currently underway at Canadian institutions. Specifically, we added sections on new kinds of potential programming (e.g., 2SLGBTQIA+, transfer students, military students, students of African descent, Indigenous students, online programming and mature students) as well as additional questions regarding what data were collected from students, and by whom (see below). The expansion of scope  reflects that the complexity of the student experience has evolved to better reflect the diverse attributes of our students which have become better researched, supported and understood since the previous survey.

In the first part of the survey, we asked participants to “Select the background information collected on your admission application that you use for your FYE / transitions programs (i.e., when students apply to the institution).” Specifically, we asked if there were identifiers for 2SLGBTQIA+ status, disabilities, age, ethnicity, first generation status, full versus part-time status, domestic versus international status, and if the student was in the military or a veteran. We also asked if there were measures (or use) of academic attitudes and motivations, career aspirations, parents who were alumni, high school (or other) grades, hours worked per week, intended field of study, languages spoken and/or written, family income, ability to pay, financial literacy, and type of secondary school.

The next part of the survey asked participants to “Select the information that is measured during or immediately following the completion of the first year of study for your students” and whether this was measured by the institution and/or used in some way for FYE and transition programming. Specifically, we asked about academic average after first term, academic performance of scholarship students, class attendance, academic attitude, student involvement, change in full-time/part-time status, change in residency status (i.e., on or off-campus), changes in family financial circumstance, commitment to completion, method of study (virtual, hybrid, or in person), DWF rates (i.e., drop out, course withdrawals, failures) for first-year courses, peer interaction, and student-faculty interaction. In this section, we also asked about withdrawals per month, withdrawals in the first term, withdrawals in the second term, number of program changes, first to second-term retention, first-year to second year retention, institutional satisfaction, study habits, and sense of belonging.

Following a question about the surveys institutions use to assess student transition and success, we asked about the programming offered to different student groups. For each student group we asked if they provided an onboarding program, if participation was required, which unit in the institution was responsible, when it was provided (i.e., in the summer prior to first term, in the week immediately before term begins, at the beginning of term, during first term, or over multiple terms), if outcomes were developed for the program, how any outcomes are assessed, what those outcomes are, and if retention rates for program attendees were measured. These items were asked regarding programs for 2SLGBTQIA+ students, first generation students, direct from high school students, indigenous students, international students, male-identifying students , mature students, minority-ethnic students , student athletes, students admitted on probationary status, students returning after being on probation or suspension, non-traditional students (as defined by the institution) , out of province students, students who are parents, students with disabilities, transfer students, and female students in STEM. We also asked if there were any other programs being offered that we had not listed.

The next section of the survey focused on student advising programs. We asked if academic advising was offered to first-year students, who the advisor was (faculty or staff), and if it was mandatory. Next, we asked if the institution offered peer-led or group mentoring programs, who coordinated them, and how they were organised. We then asked about first-year transition courses: were they offered, were they for academic credit, if they were required, if they were a stand-alone course, if they were integrated into another first-year program requirement, and if they were taught in residence. Following this, we asked if institutions offered peer-support programs to first-year students, and (if yes) if they were segmented by faculty, if they were segmented by major, if they were offered in residence, and if mentors were paid or if they volunteered (or both). Next, we asked about career advising for first-year students: the type of program, if it is mandatory, who offers it, and if it is embedded in a course. We also asked if institutions had senior students acting as peer mentors in their first-year seminars.

In the next section of the survey, we asked if institutions offered faculty development based on first-year transitions. If they did, we asked who coordinates the training and if it is mandatory. Similarly, we asked if faculty development was offered for developing engaging and interactive pedagogy (i.e., active learning), if it is mandatory, and who  offers it. Next, we asked if institutions offered residence life programming to support first-year student transition. The questions focused on programming for academic skills development: if it is offered, and what it is (“please describe”).  We then asked about other potential programs and if they are required: academic integrity, gender-based violence, EDIA (equity, diversity, inclusion and accessibility), and an open-ended question about any other programs, services or centres the institution may have.

As in the original survey, we asked about the involvement of student unions: “To what extent are your student unions / associations actively involved in first-year student transitions?” with options ranging from “1 – Not involved” to “5 – Very involved”. Next, we asked if institutions used “specific technological supports to assist with student transition in first year,” theirpurpose, and which products were used. Specifically, we asked about pre-arrival surveys, online orientation, early alert/assist programs, Client Record Management (CRM) software, and any others. We asked participants to describe any barriers they had encountered when trying to implement new programs for first-year students and to provide examples of those barriers. We also asked whether any programs had been cancelled and why, and how COVID-19 had impacted transition programming for first-year students.

In the final part of the survey, we asked about pan-institutional initiatives: whether they are involved in any collaborative efforts between units, departments, and disciplines to support first-year development and what those efforts are. We asked if the institution has a Strategic Enrolment Management (SEM) Plan, and the three greatest challenges facing their students in transitioning into, through, and out of the institution. Finally, participants were asked if they had any additional information they wanted to share about their institution, their experiences, or first-year programming in general.

Results

Colleges and Universities:

Of the 51 completed surveys received, 15 were from colleges and 36 were from universities. According to the MacLean’s rankings, 19 of the institutions were primarily undergraduate, 12 were comprehensive, and 5 were medical/doctoral institutions. 16 of the institutions would be considered small (less than 5,000 students), 17 would be considered medium (5-15,000 students) and 18 would be considered large institutions (greater than 15,000 students).

Data Collected at Admission:

As can be seen in Table 1, there was substantial variability in the data collected by institutions at time of admission. A notable finding was that some types of data were often only collected by the admissions department, and rarely used for programming (e.g., incoming grades, languages spoken or written, full versus part-time status, type of secondary school) whereas other data was very commonly used for programming if collected (e.g., domestic versus international status, ethnicity, disability status, living on or off campus, intended field of study). Also notable is the data that was rarely collected or used for programming, such as academic attitudes, academic motivations, career aspirations, first-generation status, family who are alumnae, family income, financial literacy, ability to pay for education, and hours worked per week. We examined differences between colleges and universities as well as between types of university (e.g., primarily undergraduate, comprehensive, medical doctoral) to see if there were any differences in the types of data collected by institutions at admissions. An interesting difference we found was that primarily-undergraduate institutions were more likely than others to collect and use data on ethnicity and race (chi-square (9) = 16.80, p =.05) and first-generation status (Chi-square (9) = 18.86, p < .05). In addition, primarily-undergraduate institutions and medical doctoral institutions were more likely to collect and use high school grades (Chi-Squared (9) = 17.63, p < .05). We also found that medical doctoral universities and colleges were more likely to assess students’ ability to pay for their programming (Chi-Squared (9) = 22.42, p < .05).

Data Collected at the end of First Year:

There were also substantial variations in terms of data collected and used at the end of the first year(see Table 2). Most institutions reported collecting data on academic performance, change in full versus part-time status, DFW rates, first-term withdrawals, second-term withdrawals, and retention rates (year 1 to year 2, and by term). Most institutions did not collect data on withdrawals per month, peer interaction, first term grades versus high school grades, class attendance, student involvement, residence status change, family financial circumstance, method of study (i.e., virtual versus in person versus hybrid) or student/faculty interactions. As can be seen in Table 2, it was rare for data collected at the end of first year to be used for programming purposes alone, and many types of data collected were virtually unused for programming purposes at all (e.g., withdrawals, change of program, financial circumstances, satisfaction with the institution, and sense of belonging) (see Table 2 for complete results).

As with data collected at admission, we explored the extent to which data measured at the end of first year differed across institutions.  Interestingly, universities were more likely than colleges to collect data on class attendance (Chi-Squared (2) 7.33, p < .05). Both colleges and primarily-undergraduate institutions were more likely to collect data on the performance of scholarship students than were medical/doctoral institutions (Chi-Squared (9) = 17.63, p < .05). The data also showed that medical/doctoral institutions were more likely to measure student involvement (Chi-Squared (9) = 26.20, p < .05), peer interaction (Chi-Squared (9) = 24.73, p < .05) and commitment to completion (Chi-Squared (9) = 26.28, p < .01).

FYE Courses:

29 of the 51 responding institutions reported that they offered a specific first-year experience course. Almost half (45%) of those offering such a course indicated that it was for academic credit, and a third (32%) indicated that completing the course was a requirement for graduation. Two-thirds (66%) of the FYE courses offered were standalone courses, and 41% were linked with another first-year program requirement (e.g., such as a required first-year course, or some other graduation requirement). Although there were no differences between colleges and universities in terms of the likelihood of offering FYE programs, universities were more likely to offer them for academic credit (Chi-Squared (4) = 10.22, p < .05).

Challenges, Barriers and Program Changes:

As noted above, we asked respondents to indicate what they thought the three greatest challenges were for students entering their institutions. Although many challenges were mentioned, the three challenges mentioned most often were financial needs (e.g., access to housing, employment, and food security), academic expectations (e.g., workload, academic skills, and preparedness) and wellness and engagement (e.g., sense of belonging, stress and anxiety, and opportunities for socialization). Despite the obvious needs of students, two-thirds (67%) of institutions reported facing barriers to creating new programs . The most commonly listed barriers were maintaining active campus partnerships (with faculty, academic units, etc.), lack of financial resources/staffing capacity, and inconsistent student engagement (especially from underrepresented groups). Respondents reported little change in programming despite the needs of students . Only 19% of institutions reported cutting any programming once it was established. In cases where programs were cancelled, budgetary limitations were the most common reason, with capacity being the second most common reason. This did not differ across types of institutions.

SEM Plans:

Strategic enrolment management plans have become increasingly common, and most institutions reported having a SEM plan (69%) with only a quarter (24%) reporting that they did not, and three respondents (7%) being unsure. This did not differ across types of institutions.

Impact of COVID-19:

Respondents almost universally reported that the COVID-19 pandemic had affected their programming and staffing. For example, one respondent noted that in terms of funding: “We have experienced budget cuts over the course of COVID and now, as we are returning to our more traditional in-person orientation programming, cuts and some uncertainty around budgets have impacted what/how we can deliver.”  Other respondents noted that the pandemic had a significant impact on students and their engagement: “Largest impact is orientation volunteers as we lost approximately two cohorts worth of volunteers when offering solely online orientation. Has been challenging to rebuild this to provide the peer-led experience during orientation” and “We saw a significant drop in student engagement with the shift to online, to hybrid, back to in-person then back to all online, etc. with these pivots can a lower sense of community, heightened sense of disconnect from campus and academics and a unique perspective on what students expect from their first year at university.”  Finally, a respondent noted that COVID-19 had influenced how they had to think about delivering programs: “… we have developed a model of blended delivery providing choice to students. This has actually been a ’win’ for our college, a positive change that benefits students.”

Discussion and Implications

Notable Findings

Responses to the survey showed that there is a wide variety of data used by institutions and substantial variability between institutions in terms of the data they use. Perhaps not surprisingly, incoming grades and intended field of study were the most common elements collected by institutions (though they were not as frequently used for first-year programming). Interestingly, despite field of study being a commonly collected, career aspirations – which we would expect to be closely linked – was rarely collected. Additionally, despite academic attitudes and motivations being important factors in student success (Kuh, et al., 2011) they were rarely assessed or used for the development or implementation of first-year student programs. Financial literacy and financial situations were also rarely collected, yet have a clear impact on students’ ability to continue (and complete) their education. Similarly, we know that being a first-generation student is an important factor in success  (Kuh et al., 2011), but relatively few institutions collected or used these data for programming.

The survey showed that there are a variety of first-year programs being offered across Canada to support student success. New initiatives include programs for students with disabilities, 2SLGBTQIA+ students, mature students, and others. Although programs for Indigenous students existed when the 1997 survey was conducted, such programs are now offered by most institutions. In addition, although international students were still relatively rare in large numbers at Canadian institutions in 1997, this was the most common type programming targeted at a specific group being done at Canadian institutions today. It is also worth noting that although most programs being offered by institutions had outcomes associated with that program, evaluation of programs was typically done “in the moment” immediately after the program or session is complete, and few institutions did longer-term assessments of the impact of programming on student success.

It is important to consider that all respondents listed challenges students are facing, and barriers that they have experienced when trying to start or maintain student programming. Students are facing substantial challenges funding their education. Hours worked by students steadily increased throughout the 1990s and 2000s (Statistic Canada, 2010). With the cap on international student working hours having now been removed in the aftermath of the pandemic, it is likely that international students have officially been working more hours (although international students regularly unofficially worked over the federal government limit to begin with) (Yun, 2023). Although additional work hours may ease the financial burden on students, more time working means less time available for schoolwork. Perhaps unsurprisingly, student mental health and wellness was frequently cited as an issue by respondents, possibly as a result thereof.

For student affairs and services staff, budgetary restrictions were frequently mentioned, as well as challenges communicating with students and working effectively with academic staff. In many institutions, student affairs programming is not considered “academic” in nature and the link between academic and co-curricular programming can be tenuous. It is unlikely that there is a simple solution to the budgetary difficulties facing colleges and universities in Canada, as government funding as a proportion of post-secondary institutions’ budgets has consistently declined for the last 30 years (Usher, 2023). This puts even more pressure on students, as tuition dollars must make up the difference .  Policy changes regarding immigration made by the Canadian Government  in 2023 and 2024 have shown a vulnerability in the overreliance on a supply of international students that may not be as reliable as we first assumed.

Respondents also noted that COVID-19 had a significant impact on students and programming. It has been shown in other research that students have been negatively impacted in terms of mental health, career development and other areas (e.g., Kwon, et al., 2023; Kwon et al., in press; Carter-Rogers et al., in press). However, our results also demonstrated that COVID-19 had an impact on student services professionals, both in terms of programming (i.e., having to change programming, change delivery formats, etc.) and student engagement in those programs.

Implications

The results of this survey have many implications. Although the institutions that responded only represent a subsample of post-secondary institutions in Canada (see below) what is clear is that the programs offered and the challenges and barriers experienced have some commonality across institutions.

Importantly, we can see that over the last 25 years there have been substantial changes in the types of programs and services that are being offered to students. Perhaps unsurprisingly given the dramatic increase in international students in Canada, international student programs are the most common. Between 1995 and 1999, 219,000 international students were admitted to Canada (Statistics Canada, 2015) whereas 900,000 were admitted in 2023 alone (Canadian Press, 2024). Institutions are also seeing more diverse student groups in terms of ethnic-minority students,  2SLGBTQIA+ students, students with disabilities, students with mental health concerns, first generation students, and other groups that would have been less common in 1997. As such, student affairs and services staff have developed many new types of programs to support those students (while rarely cutting existing programs).

Respondents also noted that budgets continue to be a challenge in their areas. The cost of education continues to rise, as do the hours worked by students. Post-secondary institutions are relying more and more on student tuition fees to cover the cost of operations that used to be primarily covered by provincial and federal grants (e.g., Usher, 2022). For many institutions, it is likely this will continue to be the case in the coming years given the recent decision of the Canadian federal government to reduce the number of international students accepted to study in Canada by one third (Wherry, 2024) . This decision, which is particularly targeted at Ontario, Quebec, and Nova Scotia, is an attempt to help manage the current housing crisis in Canada which is affected many Canadian students (also commonly noted by respondents to the survey). It is unclear whether limiting international students entering Canada will indeed have a positive impact on the housing issue; it will undoubtedly impact already strained budgets, especially for institutions that have grown to rely on international student tuition fees to make up for increasing costs and relatively reduced government funding.

It is also clear from the results of this survey that a substantial amount of data is collected about students at admission and after their first year of study. However, these data are rarely used in the development or implementation of programming. Additionally, to the extent that programs are evaluated, it is largely through surveys administered to attendees immediately after participating in such programs, with little long-term follow-up. This is not surprising given the strain on resources in most student affairs and services units (Seifert, et al., 2011). Nonetheless, the data collected presents an opportunity for student affairs and services professionals to improve and develop their programs, provided such data could be collected and analysed in a manner that could be effectively communicated to the staff and senior leadership who determine budget allocations.

Most institutions reported that they had Strategic Enrolment Management Plans in place. Although this was not a specific component of the survey, it is clear that strategic planning documents such as SEM plans are often created by institutions to guide tactical approaches to supporting students. However, in many institutions they do not live up to expectations. Plans are often forgotten, events may occur that make them obsolete (e.g., limits on study permits), plans are not adequately funded, or the plans do not consider the reality and breadth of issues present at an institution. Furthermore, student affairs and services professionals often  find that the focus of SEM plans tends to be on recruitment rather than retention efforts, and thus student success initiatives do not get the attention and support they need to be successful.

Although respondents frequently mentioned the importance of student mental health and wellness, it is also important to note that the conditions at many institutions across Canada require that we also consider the mental health and wellness of faculty and staff (Brophy, Ezekiel, Leary, 2024).

Limitations

It is important to recognize that there are limitations to this survey. Firstly, despite reaching 130 respondents (albeit with a significant number of duplicates) over the period of about 8 months, only 51 complete responses were received, which represents less than 1/3 of post-secondary institutions in Canada. There are several factors which likely contributed to this. The survey was long, and took approximately one hour to complete, which may have been daunting for many respondents. However, it is worth noting that when people quit the survey they usually did so early in the process, suggesting that the people who began the survey may not have had sufficient information to complete the responses. Furthermore, our target audience (senior student affairs and service officers) are extremely busy, and simply may not have had the time to complete the survey, nor would have had anyone else in their units with the knowledge necessary to do so. Our survey was also one of many that SSAOs receive each year.  There are some well known surveys that SSAOs would be involved in completing such as the National Survey of Student Engagement (NSSE), the Canadian University Survey Consortium (CUSC) and the Canadian Campus Wellness Survey (CCWS), the International Student Barometer, and related surveys on internationalization coming from the Canadian Bureau of International Educators, which are often connected with or lead in completion through these same positions .  Furthermore, in the last 10 -15 years we have seen increased expectations for government reporting on institutional performance indicators for both the college and university sectors. Government funding in some provinces now is connected to student enrolment related measurements, as are elements of specific funding in areas such as accessibility and graduation rates for specific professional programs. With mounting critical issues and expectations coupled with increased governmental accountability, it is not surprising that longer surveys requiring uninterrupted attention and internal consultation see lower-than-ideal completion rates.

Conclusion

The results of this survey, which will be discussed further by the authors of subsequent chapters as they discuss their specific topic areas, highlight that there is a clear need for transition programs for first-year students and, more specifically, for special populations within the post-secondary environment. The programs that exist are broad and diverse, covering a wide range of topics and delivered in a variety of formats to allow for the individuality of the writers and the topic areas. However, it is also clear that student need exceeds the ability of many institutions to effectively offer those programs. How we implement, assess and ladder these program outcomes into peer-reviewed research are areas of opportunities within this sector.  Furthermore, these programs are typically not integrated into the “academic” side of the house, largely due to the silos that exist within academic institutions.

 References

Canadian Press (2024). New student visa plan expected today from immigration minister. National Observer. Retrieved from: https://www.nationalobserver.com/2024/01/22/news/new-student-visa-plan-immigration-minister#:~:text=Get%20daily%20news%20from%20Canada’s%20National%20Observer&text=More%20than%20800%2C000%20international%20students,number%20just%2010%20years%20ago. Jan. 23, 2024

Carter-Rogers, K., Smith, S.M., Tabvuma, V., Brophy, T., & MacDonald, T. (in press). A year with COVID: Empirical evidence assessing an institutional led intervention while coping with the challenges and successes of the virtual transition to online learning. Online, Open and Equitable Education – Stories of Teaching and Learning During the Global Pandemic.

Gilbert, S, Chapman, J., Dietsche, P., Grayson, P. & Gardner, J. N. (1997). From Best Intentions to Best Practices: The First Year Experience in Canadian Postsecondary Education. National Resource Center for the Freshman Year Experience and Students in Transition. USC. South Carolina, USA.

Kuh, G. D., Kinzie, J., Buckley, J. A., Bridges, B. K., & Hayek, J. C. (2011). Piecing Together the Student Success Puzzle: Research, Propositions, and Recommendations. John Wiley & Sons.

Kwon, E., Kwak, M., Smith, S. M., Zhang, M., Carter-Rogers, K. & Green, M. (in press). The transition to online education amid the COVID-19 pandemic and the impact on international students’ physical and mental health and well-being. In Online, Open and Equitable Education – Stories of Teaching and Learning During the Global Pandemic.

Kwon, E., Kwak, M., Jung, G., Smith, S.M, Tsuchiya, K., Kyeremeh, E., & Zhang, M. (2023). The impact of the COVID 19 pandemic on motivation to stay: A comparison between Chinese and non-Chinese international students in Nova Scotia, Canada. Journal of Ethnic and Migration Studies. https://doi.org/10.1080/1369183X.2023.2270334

Seifert, T.A., Arnold, C., Burrow, J., Brown, A. (2011) Supporting Student Success: The Role of Student Services within Ontario’s Postsecondary Institutions.  Toronto: Higher Education Quality Council of Ontario.

Statistics Canada (2010). Perspectives on Labour and Income. Retrieved from: https://www150.statcan.gc.ca/n1/pub/75-001-x/2010109/pdf/11341-eng.pdf Jan. 22, 2024.

Statistics Canada (2015). International Students who Became Permanent Residents in Canada. Retrieved from: https://www150.statcan.gc.ca/n1/pub/75-006-x/2015001/article/14299-eng.htm, Jan. 24, 2024.

Usher, A., (2022). The State of Postsecondary Education in Canada, 2022. Toronto: Higher Education Strategy Associates.

Usher, A. (2023). State of Post-Secondary Education in Canada. Higher Education Strategy Associates. Toronto ON. Retrieved from: https://higheredstrategy.com/spec-2023/ Jan. 22, 2024.

Wherry, A. (2024). Federal government announces 2-year cap on student permits.  Retrieved from: https://www.cbc.ca/news/politics/miller-cap-international-students-1.7090779, January 24, 2024. CBC News.

Yun, T. (2023). With Canada set to reimpose cap on working hours, international students worry about paying for tuition, living expenses. CTVNews.ca. Retrieved from https://www.ctvnews.ca/canada/with-canada-set-to-reimpose-cap-on-working-hours-international-students-worry-about-paying-for-tuition-living-expenses-1.6669889  Jan. 22, 2024.