The School of Journalism has assessed its courses continuously and made adjustments to the curriculum accordingly since developing an assessment process in 1999. The plan, updated in 2004, 2011 and 2012-13, has been used to collect direct and indirect measures of student learning for the past six years.
Faculty review assessment data to identify areas that can be improved, make changes to the curriculum, and then monitor outcomes to determine if the changes improved learning. Significant curriculum improvements have been made during the past 10 years because of assessment.
The school built on its assessment plan by adding two quantitative measures for assessing whether students are achieving in the 11 core student learning outcome goals. One is the pre- and post-test, described in a later section of this site, and the other is a writing assessment given to students entering the major and on completion of the major.
The school conducted an intensive review in academic years 2005-6 and 2010-11 for reaccreditation through the Accrediting Council on Education in Journalism and Mass Communications, and implemented changes from 2006 through 2012 to the curriculum in response to the findings.
The school will undergo another extensive review in the academic year 2016-17 in preparation for the next reaccredition by ACEJMC.
Students needed or wanted, or both:
- More interactive elements in the writing, editing, and visual communication classes in the core curriculum, along with more multimedia instruction. .Journalism Student Advisory Council students reported that students felt they weren’t getting sufficient education in multimedia. Students surveyed in 2008 in an upper-division course rated their proficiency level with specific multimedia software. Results showed that while all students had a basic proficiency with Photoshop, few had little or no knowledge of software used for graphics, audio, or visual storytelling.
- Additional opportunities to develop visual skills, and experience with producing video-news reports that could be included in a professional portfolio, to meet increasing industry demand for these skills.
- Additional instruction about mathematics and statistics, and opportunities to develop quantitative skills.
- Additional instruction in computer-assisted reporting, which involves database creation and management.
- More professional internships that offered academic credit. Such internships need to be supervised by the School, to ensure that students have significant experiential learning opportunities.
- More opportunities to model professional behavior, to meet professional journalists, and to attend professional conferences.
- Given globalization, students, faculty and alumni saw a need to increase curricular emphasis on global awareness, so the School could enable future journalists to obtain a better understanding of the history, politics, and cultures of other nations and the role and responsibilities of the news media in a global information environment.
- Better instruction in safety procedures for covering stories near the U.S.-Mexico border, or within Mexico, particularly at a time when violence along the border had become prominent in the news.
Better communication with the School of Journalism. A survey of primarily freshmen students in a spring 2011 course (News in a Digital Age, Jour 150) found that relatively few rely on email listservs and email for their university news, but they did rely on their Facebook pages (87 percent reported they checked their page at least daily). Faculty also noticed that students were not reading the periodic listserv messages closely, a key communication tool between the school and students. School events for students were not well-attended because students were not finding out about them.
In addition, students and recent graduates expressed a desire to provide the School with input about instructional and curriculum issues that went beyond what University course-evaluation forms could provide. As undergraduate enrollment rose sharply, and the School increasingly began to rely on part-time faculty, there was a greater need to orient them about learning-outcome objectives and effective instructional strategies for each class in the core curriculum.
Changes made based on the findings
- In 2013, the school hired a fulltime faculty member to teach multimedia, web apps, mobile apps and to develop a course in entrepeneurial journalism. The School began offering visual communication, magazine photography and entrepeneurial journalism courses regularly, to supplement the required class in beginning photojournalism.
- The School has increased the number of interactive instructional computer laboratories from one to five, with additional computers and printers available for students in the department's reading room. Each year, using course fees, the labs are refreshed, software updated and other equipment assessed. The school has state-of-the-art Mac labs for students, along with computers in the undergraduate Reading Room and the graduate student lounge.
- New required class on multimedia storytelling. In fall 2009 students began taking a new required course, Principles of Multimedia (Jour 307), concurrently with Advanced Reporting (Jour 306). Seniors will be surveyed spring 2012 to assess whether they are more efficient with multimedia practices. As a result of this work, graduating students now report easy familiarity with many of the multimedia tools used by the profession today, including Final Cut Pro X, PhotoShop, Adobe Creative Suite, Wordpress and other programs. The multimedia sequence is consistently re-evaluated as the industry adops new software, new methods and new journalistic initiatives.
- Instructional technology support. In 2008 the school hired a full-time staff member to provide instructional technology support. The school provides instructional support for students learning multimedia, and is paid through the school’s program fee.
- Converged school media. The faculty developed a new course to coordinate the web presence for the separate school media courses, to start spring 2014. This will result in a converged school media where every student graduates with hands-on experience in producing web content and social media.
- The School also created a course in computer-assisted reporting to teach students how to gather government data, analyze it, post it online and integrate their findings into news reports. A unit on computer-assisted reporting is being added to all the Advanced Reporting classes, as a foundation for public affairs reporting and the capstone classes.
- The School added a video-news capstone course, in which students produce news segments shown on local television network affiliates. The School added another broadcast writing class for students beginning in '15-'16 and has initiated a class for all majors in video-photography techniques and tools.
- The School started interdisciplinary International studies, in collaboration with the Center for Latin American Studies, the Center for Middle Eastern Studies, and the Department of Near Eastern Studies, which give students the opportunity to engage in dual majors and specialized regional studies.
- Faculty developed additional in-class exercises and out-of-class assignments that involved the use of mathematics and statistics. There are units on numeracy in the beginning journalism class, the editing class and the advanced reporting class.
- The School Director hired an internship coordinator. The coordinator maintains a file of available internships, posts notices about internships on a listserv, matches students with internships, and acts as a liaison with internship directors at news media and other information industries. The School has been a leader in 100% student engagement, a core strategic initiative of the University of Arizona. Our students have all had engagement experiences through their capstone courses, and a majority of students now have one or more professional internships.
- The School is requiring more out-of-class assignments in lower-level reporting classes, so students have additional opportunities to work in a professional context by producing written, visual, and multimedia reports about real events and issues on campus and in the community.
- The School started student chapters of the Society of Professional Journalists, the Native American Journalists Association, the National Association of Black Journalists, the Asian-American Journalists Assocation, the National Association of Hispanic Journalists and National Press Photographers' Association.
- The School Director developed an orientation program for new faculty and a Faculty Handbook explaining course guidelines, grading policies, syllabi preparation, and other instructional issues. The handbook is revised every year.
- The School started an introductory course for pre-majors, Jour 105 – Principles of Journalism, to better prepare them for their future courses and to help students decide whether journalism is the best major for them.
- Beginning in 2010, the School has been teaming with the Dart Center for Journalism & Trauma at Columbia University to provide border safety workshops for students, educators and journalists. Also, the school developed policies that require permission to conduct journalism for classes in Mexico and that safety measures be taken.
Assessing the program
The Faculty Undergraduate Curriculum Committee has developed learning goals and objectives for every requred class, along with measurable outcomes. These are included in all syllabi and they are integrated into the assessment program by ensuring that incremental outcomes for each standard are set for each requred class. A student would show proficiency in writing by their advanced reporting class, but might not be expected to show mastery until their capstone and their Reporting Public Affairs classes.
The school assesses student learning through a dozen instruments that include more than 40 different learning measures – direct and indirect, quantitative and qualitative – expanding on the 2005-06 self-study for re-accreditation. Each assessment measure is gathered annually from assigned staff and faculty, coordinated by an Outcomes Assessment Committee, and combined into a final report. Faculty review the report at a fall retreat to discuss potential curriculum improvements. A summary of the school’s annual assessment indicators is posted on the school website for students and the public (www.journalism.arizona.edu/report-card), and results are incorporated into the re-accreditation self-study.
Assessment instruments are divided into two main categories
1. Direct instruments are based on student performance on assessment tests, writing samples and surveys of internship coordinators assessing how well they meet each outcome.The pre-test/post-test questionnaire includes several different measures for different learning outcomes, such as an index measuring media law knowledge or support for press independence. Some measures directly assess student competency, such as the assessment test and the writing exercise, required of all students as they begin the major and as they finish it.
2. Indirect measures include alumni surveys, course evaluations, achievement (awards, GPA, etc.), student feedback, and indicators of performance. This includes faculty instruments are based on faculty assessment of student performance, including analysis through committees, retreats, and day-to-day testing and grading. And it includes industry instruments are based on perceptions of student quality by employers, intern supervisors, alumni and other working professionals.Some measures indirectly measure achievement, such as a survey of interns to measure their own perceived competency – they might not have actually learned even if they say they have learned. Measures can be quantitative or qualitative.
Journalism Assessment Test (pre-test/post-test). The school has been administering this test for the past two years, refining it before full implementation in fall 2012. The multiple-choice test is given to students in Principles of Journalism (Jour 105) and to students in the required senior school media courses. Results are compared from beginning to end of the program to measure learning in 10 of the 12 learning outcome categories. The results of this test for 2012-13, 2013-14, and 2014-15 are here.
Portfolio Assessments Members of the class of 2014 began developing their own e-portfolios in the fall of 2012 and will continue to build their portfolios of work as they complete more advanced classes as well as internships with various professional organizations. Beginning in fall 2015, these portfolios will be evaluated by external professionals so that students, and the faculty, have a clear sense of how well the 10 learning outcomes are being met and whether the outcomes need to be adjusted to the changing workplace.
Writing test The school gives students entering the major a writing assessment, and it administers the same exercise to those graduating. The test was first given in Fall 2012 as a pilot, refined and edited, and then given in Spring 2013. The writing test was given to all students in 2013-14 and 2014-15. It has four parts -- fact-checking, writing a news story, developing ideas for a follow-up and discussing any legal and ethical issues encountered in the materials provided. A copy of the rubric for scoring the writing test is contained on this page.
Intern ratings by supervisors. After each student completes an internship, the student’s supervisor completes a survey to assess the student’s capabilities and skills, including written skills, use of technology, critical thinking and ethical principles. These surveys have been conducted since 2005 (and have demonstrated a consistent increase in supervisor ratings of UA journalism students).
Course-specific studies. During the past six years the school has conducted studies of specific courses to assess overall student learning, based on a combination of surveys, focus groups, grade analysis and other methods. These targeted studies have resulted in significant curriculum change.
Faculty assessment. Faculty directly assess student learning through grading, testing, portfolio review, personal contact, and through discussion via committees (e.g., Faculty Undergraduate Curriculum Committee), retreats, regular meetings, an annual faculty assessment questionnaire, and informal conversations. These are direct observations by faculty of student learning, and initial observations usually lead to more quantitative, systematic examination of the curriculum.
Course evaluations. Students fill out course evaluations for every class, and those ratings are available for analysis. One question is “How much do you feel you have learned in this course?” This question in particular is examined to see if students perceive they are learning. This becomes an important part of the feedback loop in refining the curriculum.
· Intern self-evaluation. At the end of an internship, students fill out a questionnaire asking how much they learned. This survey has been conducted each semester since 2005.
· Senior exit survey. When students fill out their paperwork to graduate they also complete a survey asking their perceptions of what they learned, and soliciting suggestions for improving the program. These forms have been collected since 2009.
· Alumni survey. Starting in 2012, alumni six months to a year out of school have been surveyed to assess their attitudes toward what they learned and to find out about where they are employed.
· Student awards. Student awards and honors are tracked and publicized weekly on the school website and through an e-mail “kudos” on the school listserv.
· Industry feedback. Faculty travel the state and meet with editors, publishers, television news producers and others, collecting suggestions and perceptions as they go. Editors at community papers, in particular, often comment about their appreciation that our students know how to cover local government when they graduate. (All students are required to take a senior-level reporting public affairs class.)
Feedback loop: Improving curriculum from assessment data
It is important to note that no single measure can “prove” that the school causes students to learn the core competencies, and every measure has a certain amount of “noise” and imperfection. Also, because there are no national assessment standards in journalism education, results are compared within the school over time, and between students starting and ending the program. Results must be interpreted carefully, and measures can be removed or added depending on their usefulness and resource commitment. Despite the limitations of these instruments, a broad range of indicators can help the school make better informed curriculum decisions based on substantial information rather than anecdotes and hunches. The assessment process is not static. It is a feedback loop where data are gathered and analyzed, changes are made to the curriculum as a result of the analysis, performance is measured to see whether the changes resulted in improvements, and further changes are made if needed.
Journalism and mass communication professionals are involved in student learning outcomes assessment in several ways:
· Involvement as teachers. All journalism skills courses are taught, and all assignments are graded, either by permanent school faculty who have substantial professional experience or by adjunct instructors who are working journalists. No instruction in skills classes is provided, nor is any grading performed, by graduate teaching assistants. Adjunct faculty members also participate in curriculum development activities. They contribute to discussions about how to achieve learning objectives through revisions in course guidelines and assignments. They offer advice about hardware and software, which enables the school to provide instructional technology that is relevant for professional training.
· Involvement as mentors and advisors. Professionally trained faculty members serve as mentors and informal academic advisors for students. Formal advising and additional mentoring is provided by Academic Advisor Renee Schafer Horton, who has an extensive background as a journalist, most recently as a reporter for the Tucson Citizen newspaper before it closed.
· Involvement as alumni professionals. The school maintains an extensive network of alumni. Those who work for news organizations, government information offices or other information businesses throughout the country tell the school about internships and jobs. These alumni provide continual feedback about student learning outcomes from a professional perspective.
· Involvement as members of the external advisory council of professionals and educators. The school’s Journalism Advisory Council comprises professionals from the news media and other information businesses, as well as journalism educators. The local component of the Advisory Council includes journalists, executives and educators from Arizona. The national component includes members who work for national news media. These professionals provide information about what these industries are looking for in terms of students’ general knowledge and skills. This enables the faculty to update the curriculum and learning-outcome goals.