Thoughts on Teaching and Learning

Thoughts on Teaching and Learning

TLE Bloggers

  • Technology Changing How Students Learn, Teachers Say

    I had a meeting with my TA this afternoon, and we were discussing questions that students had problems with on the last exam. One of the questions asked students how Baddley's model of memory differed from the traditional information processing model. The correct answer is that Baddley argues that information goes from sensory memory to long-term memory and then into short-term or working memory. It turns out that the students she talked with during her office hours had problems with the question because the lecture slide showed information from sensory memory going into implicit and explicit memory, which are two types of long-term memories, but the slide didn't have the words long-term memory written on it. So although I stated more than once and in more than one class that Baddley's model proposes that information from sensory memory goes first into long-term memory, they did not process that fact because it wasn't written on a slide. I find myself incredibly frustrated with this expectation that the only information necessary to attend to is that which is printed on the slide. I only put bullet points on my slides to serve as prompts for my lecture notes, and I emphasize from the very start of the semester that there is more to notetaking than just bullet points. In fact, I stop speaking at one point during my first lecture and ask students to look at their notes and compare what they have written down with my version of what I would have written had I been listening to the same lecture, and yet here we are halfway through the semester with that lesson unlearned.

    I then sat down with the New York Times and came across this recent article:

    There is a widespread belief among teachers that students’ constant use of digital technology is hampering their attention spans and ability to persevere in the face of challenging tasks, according to two surveys of teachers being released on Thursday. The researchers note that their findings represent the subjective views of teachers and should not be seen as definitive proof that widespread use of computers, phones and video games affects students’ capability to focus. Even so, the researchers who performed the studies, as well as scholars who study technology’s impact on behavior and the brain, say the studies are significant because of the vantage points of teachers, who spend hours a day observing students.

    The timing of the studies, from two well-regarded research organizations, appears to be coincidental. One was conducted by the Pew Internet Project, a division of the Pew Research Centerthat focuses on technology-related research. The other comes from Common Sense Media, a nonprofit organization in San Francisco that advises parents on media use by children. It was conducted by Vicky Rideout, a researcher who has previously shown that media use among children and teenagers ages 8 to 18 has grown so fast that they on average spend twice as much time with screens each year as they spend in school. (more)


    Comments: 1
  • Technology in Education: Revolution or Evolution?

    For all of the technology that I use in the conjunction with my teaching: PowerPoint, clickers, podcasts, and online resources, I don't believe that technology is sort of indispensible teaching tool. A good teacher is a good teacher with or without technology, and no amount of technological gadgets are going to make a bad teacher any better. This may be heresy to say aloud, but I don't believe that technology is the key to student success. What is most important is student engagement and effort both in and out of class, and a good instructor can nurture that engagement and increase student motivation. Adam Falk, the president of Williams College, spoke to these issues in an address this past April. He has allowed me to share his comments on the TLE site.

    Presented at “The Future of the Liberal Arts” Conference
    Lafayette College
    April 10, 2012

    It seems that you can’t pick up a newspaper or magazine–the New York Times education section, the Chronicle of Higher Education, the EDUCAUSE Review–without encountering the passionate assertion that information technology has changed everything about our students and how we must educate them. Streaming video! Chat rooms! Laptops! iPods (Remember when Duke gave every freshman an iPod, right when these devices first appeared?)! Course management systems! iPads! The list goes on, and it will continue to go on, as a seemingly endless stream of new technologies arrives on the scene.

    There is no doubt that all this technology is fun, and interesting, and presents new and innovative ways to do our work of preparing young women and men to engage the world as educated and thoughtful adults. But there are those who would make a stronger statement, that these new technologies have changed, or will soon change, the very fundamentals of our profession. Even at a place as seemingly secure as Williams, with our 229 years of history and our enduring metaphor of a faculty member on one end of a log and a student on the other, I hear concerns about our coming obsolescence in the face of the computer and internet revolutions. Will Williams still matter? Will a liberal arts education, offered in the mountains of rural New England, be irrelevant to the Twittering students of the 21st century? Too slow, too stodgy, too boring? Do we at Williams, and all of us, at liberal arts colleges, need to become something completely different if we are to survive?

    My response to this question is unambiguous: notwithstanding the very real changes that technology has brought, the core fundamentals of education, both the education that we offer at Williams and education as a larger practice, remain intact. Moreover, we should fiercely resist the reflexive conclusion that because our students come to Williams with different modes of encountering and absorbing information (multitasking, multimedia, instant access, short attention spans) we must as a consequence become like them if we are to reach them and educate them. Rather, I believe our task to be the opposite one, namely to understand both the advantages and the deficits that this new world of continuous information flow leaves them with, and use the brief opportunity of their time in college to reinforce the capacity and disposition for slow, reflective, and difficult engagement with material. In fact, our students are, more than ever, hungry for just this sort of experience.

    Our current situation is hardly a novel one. The invention of the printing press itself might have been thought to presage the end of the university (“Why bring all these students to Oxford when we can just send them all the books by horseman? That would be much cheaper and more efficient, and they could study at their leisure at home, when most convenient.”), but no such thing occurred. Quite the contrary, of course. And there are more recent examples of the same sort, times where innovative uses of technology didn’t end education as we know it. Let me recall three from my own experience.

    When I was seven years old, my favorite Saturday morning activity was to get up early and watch reruns of Gilligan’s Island. But if I got up too early, in those days where there were only four channels, I had to put up with the tedium of both the Farm Report and Sunrise Semester, which ran at 6:00 a.m. on CBS. Sunrise Semester, which lasted from 1957 to 1982, was New York University’s first experiment in distance education. Real courses were offered, with NYU faculty broadcasting from a studio in New York. According to the NYU web site, the first course offered was “Comparative Literature 10: From Stendhal to Hemingway,” taught by Prof. Floyd Zulli. Students could receive college credit by paying $75: 700 applied, 177 completed the course, and 120,000 followed on television without signing up. (It is amusing to note that these numbers not all that different from those for “Machine Learning”, offered in the spring of 2012 by a Stanford professor under the auspices of Udacity.)

    What’s my point? Sunrise Semester was a great success. It ran for a quarter of a century, won an Emmy Award, and was viewed by literally millions of people. It began in 1957, at the dawn of the modern era in which television sets became ubiquitous in American homes. Certainly the pioneers of Sunrise Semester must have entertained the idea that with access to the best lectures for every student in every home, the days of the expensive residential college would soon be at an end. But why did this exciting new technology of television not have this effect? Because, of course, college education isn’t simply about the most efficient, or most engaging, means of transmitting information. It’s about the creation and nurturing of a community of students, in a particular kind of social and physical environment. It’s about learning things together, not just alone at home in your bathrobe and fuzzy slippers.

    But of course, we do sometimes learn things on our own. When I was a freshman in high school, I was bored in my regular mathematics class and my teacher gave me what was called a “programmed” book on probability and statistics. The book worked by having a series of fold-out pages. On each page was a discussion of a concept, and on the fold-out flap was a series of questions. The answers were on the back of the flap. You were meant to go on to the next page only once you’d answered the questions correctly. The whole thing was very engaging, and very effective. It was the only course in probability I ever took, and it served me well through a professional career in theoretical physics. Working in an entirely self-directed way, I really learned the material, and I loved it. In its rigor and its appeal, I have to imagine it was the equal of any self-paced text you can find for your iPad today.

    This was a very efficient and cost-effective way for me to learn probability, and I learned it. But the world of mathematics teaching was never taken over by programmed textbooks. We didn’t do away with all the math classes, hand the students these books, and send them to the library. These independent methods always had their place, of course, but why wasn’t education transformed when the day came that programmed texts could be produced cheaply? Because, I would maintain, it’s hard for even the best students to learn everything on their own, no matter how good the materials. It’s more fun, and far more effective, to learn as part of a community of students, supported by real human interactions.

    Finally, when I was in college, I taught calculus by mail for Duke University’s Talent Identification Program. The girl I taught (her name was Jane) was in the ninth grade and had no opportunity to learn calculus in her high school. I sent her the text and the problem sets. She sent the worked problems back to me every week, and I would correct them and return them to her. We handled exams the same way. Jane did marvelously, earning a 5 on the AP Calculus BC exam (and an 800 on the SAT math test, back when that meant something). A brilliant kid, for sure, and the TIP Math By Mail program was just what this student needed in a school that couldn’t meet her needs. And yet, once again, the existence of high quality distance education programs, more than a quarter of a century ago, didn’t keep schools and parents from thinking that the more expensive option of offering calculus in the classroom was actually the best way to enrich their math curricula. Today, almost all good American high schools offer calculus to their students.

    The point of these anecdotes is that, in very real ways, effective (and cost-effective) technologies to support distance education and self-paced learning have been with us for many decades. Printing, television and the postal service are remarkable tools. And, in fact, they have been used since their inventions to enhance and deepen education. What none of them has done is change the fundamental fact that at its heart education is a social activity that takes its highest form within a real community of students and faculty. Neither books nor video nor chat rooms have made colleges obsolete.

    Nonetheless, there are many who argue that our present moment is different from all those that have come before. Perhaps they are right. Perhaps the new tools of information technology provide something deeper than just new modes of content delivery. Perhaps our students are so accustomed to online interaction that the virtual communities erected within modern course management systems are all that they need to be connected to their fellow students. Perhaps their brains really are wired differently now.

    Perhaps. Personally, I doubt it. But even if some or all of this has happened, and we really have entered a Brave New World, I would argue that some simple, interrelated, principles remain:

    1. Education is still as much about human interaction as about delivering content. More so, in fact.
    2. New technology is expensive, especially technology on the cutting (or bleeding) edge. And immediate obsolescence is a perpetual danger.
    3. The introduction of technology does not generally drive down overall costs; in fact, it is likely to increase them.

    I’ll illustrate what I mean with an example from a program with which I have personal experience, Johns Hopkins University’s Advanced Academic Programs (AAP). These are professional master’s programs in a variety of fields, offered largely by part-time faculty who work in the relevant professions. About half the 2,000 students are in a variety of programs in biotechnology, many of them working in the biotech industry hub in Montgomery County along the I-270 corridor. Initially, AAP offered these courses, extensively and successfully, in a modern glass-and-steel facility in the heart of “DNA Alley.” One could not imagine a better location, and the programs, which were of high quality and high relevance to the target audience, flourished.

    And yet, by 2005 it became clear that there were good reasons to consider offering some of these courses, and perhaps entire degrees, on line. Certainly, the combined brand of Johns Hopkins and biotechnology was a powerful and global one, and there were excellent opportunities to offer these courses to students outside the Washington area. (Since the AAP programs generated net revenue, extending their reach was desirable.) But equally, the lives of the local students, battling Beltway traffic while balancing demanding jobs and family responsibilities, presented challenges that could be met by presenting the courses in an asynchronous and nonlocal format. That is, on line, using a course management system. (Even today, a significant majority of the online learners actually live within fifty miles of that glass-and-steel building, which turns out to be entirely typical.)

    In those days when much of this was new, AAP was concerned to demonstrate that the delivery of courses on line was the equivalent of what was referred to as “on site.” So they did a controlled study, in particular, with the offerings in bioinformatics. A faculty member who was to teach on line needed first to have taught the same course in the conventional classroom. The content of the online and on site courses was identical, and students were given the same final exams. Both student satisfaction and student learning outcomes were assessed and compared.

    The study showed that an online course is every bit as effective as an on-site course, even, perhaps, slightly more so. But this happy outcome is the direct result of a material investment in the online course that is no less than the investment in teaching on site. A core component of the online course, both educationally and in terms of student satisfaction, is the extensive interaction between the faculty member and the students, both in moderated chat rooms and on an individual basis. Sustaining such virtual conversations, at a high level, is enormously time consuming for the professor. As a consequence, the student-faculty ratio in the online course must be approximately the same as (or, if anything, lower than) in the conventional course. There are no simple economies of scale in teaching on line, if it is to be done well. By a similar token, the preparation time for faculty who teach on line is higher than for those who teach face to face; although the preparation is by far the heaviest the first time the course is taught, online courses must be updated as much as on site courses, and faculty turnover is just as common. It is no easier to “package” online courses so they may be taught by “anyone,” than it is for face-to-face education.

    Yet although there were no savings of money or faculty time in teaching on line, there were great benefits to AAP and its students in adopting this model. By 2010, half the enrollments in AAP’s biotechnology program were in online courses. The flexibility in time and place afforded by online delivery was of critical value, both to the local population of working adults, and to the developing global student body. (When the chat room is asynchronous, and you live in India, you don’t have to get up at 4:00 a.m. to be part of the discussion.)

    And although they were not inexpensive to deliver, the courses were educationally successful because, among other things, they made it a core priority to create and nurture a virtual community of deeply engaged students. The critical importance of this aspect of the online courses was clear in student satisfaction surveys and in interviews.

    So what are the lessons in all this for Williams and other liberal arts colleges? Our core educational mission, and our defining structure, is to educate students in a personal, intimate, collaborative environment. Young women and men come to us because they want to be in a place where they will know the faculty and their fellow students, and the community will know them. We know, and they know, that for the great majority of them, this is the sort of education that will prepare them best to be purposeful in their lives and effective in the world. I would argue that the great potential of the new technologies is not to upend these core values, but to allow us to fulfill our existing educational mission more effectively, especially by giving us new strategies to transcend our limitations of scale and location.

    In particular, distance education holds the promise, if thoughtfully deployed, of extending our curricula into areas not covered by our relatively small faculties. For example, in our new multi-polar world, there are many more languages that students will want to learn than we can possibly offer in our classrooms. French, Spanish, German, Russian, Japanese and Chinese simply aren’t enough, anymore. Internet-based instruction could be a vital solution to this challenge. In a similar vein, we can now use efficient, high quality, low cost video conferencing technologies to bring our students into truly meaningful, collaborative contact with students around the world, enhancing the traditional international dimensions of our curriculum, such as study abroad and courses on global topics, with substantive virtual experiences. (At Williams, we have a longstanding relationship with the American University of Cairo, by which we use videoconferencing to teach a course jointly each year. During the Arab Spring of 2011, this was powerful educational experience for our students, indeed!) New technologies will certainly bring us richer, multimedia teaching materials, even if they won’t be any cheaper than conventional textbooks. Some of those materials may even allow us to change, in exciting ways, the very modes in which we teach. And we may even find that computerized, programmed delivery of some standard, elementary topics (Calculus, perhaps?) becomes of such high quality that we can realize some modest efficiencies while maintaining our educational standards.

    Educational innovation comes in many forms, and not all of them rely primarily on integrated circuits. If you visit the Williams campus today, I will proudly show you our new library coming out the ground. This new facility will bring together our extensive book collections in the humanities and social sciences, our wonderful rare book library, and our Center for Media Initiatives, and surround them with dedicated spaces for interdisciplinary and group work. Our plan is to create a vibrant academic hub that will excite students’ passions, nurture their curiosity, and above all else, bring them together in space, not cyberspace. Academic work has changed over the years: it is more collaborative, more disciplinarily fluid, more eclectic in its sources and methods. A modern library is no longer simply a box for books and carrels; rather it is the crossroads of the campus both physically and metaphorically, where students and faculty meet, and where technologies both new and old come into conversation. Great academic architecture brings people into contact with each other, with tools, and with ideas. Information technology is surely a part of this story, and our new library will support it as never before, but it is only one strand among many in the tapestry we are weaving at Williams.

    This brings me back to the other question with which I began. Are students now so fundamentally different that they can only learn, or at least learn best, via the new modalities implied by our current technological revolutions? There’s no question that a teenager raised in a world of Wikipedia and ubiquitous multimedia stimulation doesn’t encounter the world, or have the same relationship to information, as did today’s college leaders when we were that age. But it simply does not follow that the only education that will seem relevant to these students, that they will be able to absorb and willing to embrace, is one that foregrounds the most disjunctive and hyperkinetic features of the modern world. Speeding along the Information Superhighway can be cognitively exhausting, and for many students it is ultimately unsatisfying. If their experiences before college have not encouraged them to slow down and think carefully about a coherent set of information, rather than surf a relentless wave of disconnected facts, then college is certainly the time to start. This deeper connection to ideas, I believe, is what they are really hungry for.

    Educational innovation, as I have said, need not necessarily involve semiconductors. At Williams, we offer some courses in a “tutorial” format, in which a faculty member sits with two students at a time, weekly, and facilitates a discussion in which the students alternately critique each other’s work and defend their own. Over the past decade, we chose to make significant investments to deepen this program, to make tutorials available to students early in their careers, to extend the model to the sciences, to provide this opportunity for any student who wants to have it. Tutorials are now offered in almost every department of the college, most students graduate from Williams having taken a tutorial, and many students take them often. The expansion of this program required a very significant investment by the college, an investment in people that seemed to some critics to represent a retrograde commitment to a pedagogy of the past, during an era of great technological advances. After all, tutorials represent a deeply personal, interactive, and challenging mode of learning, and with no particular emphasis on technological sophistication.

    Yet student satisfaction with the tutorial program is, by far, the highest of any part of our formal curriculum; and the faculty reports that student learning in tutorials is the deepest, as well. We consider this program one of our great successes, a hallmark of a Williams education, and an expression of our fundamental, and enduring, educational values.

    I have spent decades asking alumni, both old and young, about what mattered the most to them in their college educations. Fundamentally, the answer has never changed. Students and alumni point to a small number of critical relationships with individual faculty, faculty who taught them something deep and important, faculty who mentored them in theses and research, faculty who helped them personally, faculty who woke them from their slumbers. Upon reflection, alumni express little preference for those who were, in their time, the flashiest professors, or those who were the closest to the cutting edge. Bringing faculty and students together and giving them the space and time to interact is what we, at liberal arts colleges, do best. That is our core purpose. Technological evolution will make it possible to do this better, in all sorts of important ways. But if the printing press and television didn’t cause a revolution that brought us down, the iPad and the Internet won’t do that either.

    posted with permission of the author, Professor Adam Falk


  • In Defense of the Living, Breathing Professor

    As we move into the 21st century, there has been an increased demand for college educated workers and for a more informed and engaged citizenry. There is also an expectation that those educated workers and citizens have higher levels of learning and knowledge, as well as new and different skills and abilities. While our institution and others across the country face declining federal and state financial support for higher education, there have been increased calls for accountability from lawmakers, parents and students as these groups look for evidence that an education matters. 

    If all we wanted of our students at the end of a college education was a collection of facts, we could give them access to wikipedia and call it quits. However, learning is something that goes beyond the retention of information, its important for students to be able to apply that knowledge and to demonstrate the application of that knowledge. 

    In a recent opinion piece in the Wall Street Journal, the president of Williams College had this to say about the value of education and I wanted to share his comments with the UW Madison teaching community. Professor Falk has kindly allowed me to republish his remarks below, and I have added emphasis to several important points that he raises. 

    Published in the Wall Street Journal (Aug. 28 online and Aug. 29 print edition)

    ‘Crowd-sourcing’ the grading of an essay online is no substitute for thoughtful evaluation by a trained educator.

    As classes resume on our nation’s campuses, amid anxiety about high tuition, student debt and other concerns, it’s worth examining what we value in college education. The question warrants consideration, in particular, following a recent recommendation by distinguished economists, appointed by the National Academy of Sciences, proposing to define the “output” of higher education as a combination of credit hours awarded and degrees earned.

    That reduces the work of colleges to counting how many students they push through the system—a bit like defining a movie studio’s output as the number of feet of raw footage shot, with no consideration of whether the resulting movies are any good.

    Most of us in higher education take the long view about the value of what we do. Sure, students graduate with plenty of facts in their heads. But the transmission of information is merely the starting point, a critical tool through which we engage the higher faculties of the mind.

    What really matters is the set of deeper abilities—to write effectively, argue persuasively, solve problems creatively, adapt and learn independently—that students develop while in college and use for the rest of their lives.

    At Williams College, where I work, we’ve analyzed which educational inputs best predict progress in these deeper aspects of student learning. The answer is unambiguous: By far, the factor that correlates most highly with gains in these skills is the amount of personal contact a student has with professors. Not virtual contact, but interaction with real, live human beings, whether in the classroom, or in faculty offices, or in the dining halls. Nothing else—not the details of the curriculum, not the choice of major, not the student’s GPA—predicts self-reported gains in these critical capacities nearly as well as how much time a student spent with professors.

    What follows from this finding is obvious, but apparently in need of saying these days: What we do is expensive—and worth it—because these rich, human interactions can’t be replaced by any magical application of technology.

    Technology has and will continue to improve how we teach. But what it cannot do is remove human beings from the equation. Coursera, one of the new purveyors of massive, open online courses, proposes to crowd-source the grading of essays, as if averaging letter grades assigned by five random peers were the educational equivalent of a highly trained professor providing thoughtful evaluation and detailed response. To pretend that this is so is to deny the most significant purposes of education, and to forfeit its true value.

    It follows that standard economic models that define productivity as total output per unit of labor are ill-suited to analyzing the activities of colleges. Understandable concerns about the cost of college have spawned a spate of high-stakes considerations of productivity; in Texas, “low-productivity” programs at flagship universities have been targeted for elimination. Yet the only way to achieve higher productivity, as the National Academy would define it, is to reduce each student’s time with the faculty. We know that while such approaches may allow us to deliver some facts to some students more efficiently in the short run, the approaches will undermine the fundamental purpose of education in the long run.

    Equally misguided is the common practice of judging a school’s success by measuring the net worth of its alumni. Is a Williams graduate who is teaching elementary school less successful, less influential, less transformed than she would be if she had become a banker? There’s no reason to think so, and anyone assessing colleges or setting public policy on that assumption is being mischievous.

    Education is not a commodity. It’s a social process, and its value, including its economic value, both to the graduate and to society is unquestionable. It is equally true that this value cannot be reduced to a single number, however much the measurers of productivity—or those who rank us in magazines—may wish it otherwise.

    Professor Falk is president of and professor of physics at Williams College.

    Copyright 2012 Dow Jones & Company, Inc. All Rights Reserved


  • Trying to Find a Measure for How Well Colleges Do

    A recent article in the New York Times regarding the increased pressure on colleges and universities to demonstrate their effectiveness. The interest in accountability asks us as educators to think about what the goals are of a college education.  I think that many of us would say that college is more than just the accumulation of knowledge, but how do we measure achievement of those goals?

    How well does a college teach, and what do its students learn? Rankings based on the credentials of entering freshmen are not hard to find, but how can students, parents and policy makers assess how well a college builds on that foundation? What information exists has often been hidden from public view. But that may be changing.

    In the wake of the No Child Left Behind federal education law, students in elementary, middle and high schools take standardized tests whose results are made public, inviting anyone to assess, however imperfectly, a school’s performance. There is no comparable trove of public data for judging and comparing colleges.

    Pieces of such a system may be taking shape, however, with several kinds of national assessments — including, most controversially, standardized tests — gaining traction in recent years. More than 1,000 colleges may be using at least one of them.

    “There’s a real shift in attitudes under way,” said David C. Paris, executive director of the New Leadership Alliance for Student Learning and Accountability, a coalition of higher education groups. “We used to hear a lot more of, ‘The value of college can’t be measured,’ and now we hear more of, ‘Let’s talk about how we can measure.’ ”

    In January, the New Leadership Alliance released guidelines calling on colleges to systematically “gather evidence of student learning” — though not explicitly advocating standardized tests — and release the results. The report was endorsed by several major organizations of colleges and universities.

    Advocates say the point is not to measure how each college’s students perform after four years, which depends heavily on the caliber of students it enrolls in the first place, but to see how much they improve along the way. The concern is less about measuring knowledge of chemistry or literature than about harder to define skills like critical thinking and problem-solving.

    That vision still faces major obstacles. Colleges that use standardized tests vary widely in what they test, how and when. And many of them that use those tests or national surveys keep the results to themselves.

    “I’d love for all the data to be public,” said Jennifer Carney, director of program evaluation at the Jack Kent Cooke Foundation, which conducts education research. But, she added, that would inevitably lead to some colleges manipulating the figures in pursuit of a higher standing, just as some have done with existing ranking systems.

    In the best-known college rankings, by U.S. News & World Report, up to 40 percent of a college’s score is based on its reputation among educators and its selectivity in admitting students. Other factors include several indirect indicators of what happens in classrooms, like student retention, graduation rates and class sizes, but no direct measures of learning.

    Critics of standardized tests say they are too narrow and simplistic.

    “I’m not sure any standardized test can effectively measure what students gain in problem-solving, or the ability to work collaboratively,” said Alice P. Gast, president of Lehigh University.

    In 2008, the Consortium on Financing Higher Education, a group of some of the nation’s most prestigious colleges and universities — including all of the Ivy League — issued alengthy manifesto saying that what its students learn becomes evident over decades and warning against a “focus on what is easily measured.”

    Many of those same colleges participate in the National Survey of Student Engagement, which measures many factors that educators say are good, though indirect, indicators of learning, like how many hours students spend studying and how much they interact with professors. The survey, which began with a handful of colleges in 2000, had more than 700 participate last year. Each college can see its own results and those of a group of comparable colleges, but the scores are not made public.

    The view from state-supported colleges has been shaped in part by pressure from policy makers to show what taxpayer dollars are accomplishing, through standardized tests. Texas, a leader in the movement, has required its state colleges to administer tests since 2004, and it makes the outcomes public.

    Testing advocates have also gained ammunition from books calling into question the quality of American colleges, notably “Academically Adrift,” by Richard Arum and Josipa Roksa, published last year. It was based on a study showing that more than one-third of students show no significant gain in critical thinking skills after four years of college.

     But the concept of universal assessment got its biggest boost in 2006, from the findings of a commission appointed by Margaret Spellings, then the education secretary. The report said that learning “must be measured by institutions on a ‘value added’ basis that takes into account students’ academic baseline,” and that the results must be made available to everyone “to measure the relative effectiveness of different colleges and universities.”

    That prompted talk that the federal government might mandate standardized testing, as it did for public schools with No Child Left Behind in 2001.

    “That’s what gave this issue urgency,” said Christine M. Keller, executive director of theVoluntary System of Accountability, an alliance of more than 300 state colleges that was created in response to the Spellings Commission. “No one wanted the government imposing a standard.”

    Her group has approved three competing tests and asks public colleges to use them and post scores on the group’s Web site.

    They are the ETS Proficiency Profile, from Educational Testing Service; the Collegiate Assessment of Academic Proficiency, produced by ACT; and the Collegiate Learning Assessment (used in the “Academically Adrift” study), from the Council for Aid to Education, a research group.

    “These instruments are not without controversy,” Ms. Keller said. “It’s still very much a work in progress.”

    While more blanks are filled in each year, the “college portraits” remain spotty. Some member colleges have not posted test scores so far, while others first posted scores from two or three years ago and have not updated them. Typically, only a small sample of students takes a test, and the ways the samples are chosen can vary, making comparison harder.

    Some major systems, like the University of California, do not participate in the organization, while others, like the State University of New York, are represented by only a few campuses.

    Selective colleges, public and private, complain of a “ceiling effect” in standardized testing, making it especially hard for their students to show improvement.

    “It does not, in my opinion, measure value added very well for our kind of institution,” said Neal E. Armstrong, a vice provost of the University of Texas at Austin. “Our freshmen come in with very high aptitude and critical thinking skills.”

    Even so, the use of the three tests in the voluntary system has boomed, with each company claiming several hundred client colleges, most of which do not make their scores public. Test authors acknowledge that colleges often use them in less than optimal ways, like skipping some sections, testing only once every few years (often to satisfy an accreditation agency) and not measuring student growth over time.

    If officials and experts in higher education can reach a broad consensus about how to measure learning, it will become routine, with results being made public “only if consumers demand it,” said Ms. Carney, of the Cooke Foundation.



  • How 'Flipping' the Classroom Can Improve the Traditional Lecture

    This is from the Chronicle of Higher Education and discusses how instructors are reinventing the lecture. A number of faculty across our own campus are exploring these approaches, and I look forward to hearing what they have learned regarding implementation and the sorts of student outcomes they've found.

    How 'Flipping' the Classroom Can Improve the Traditional Lecture by Dan Berrett

    Andrew P. Martin loves it when his lectures break out in chaos. It happens frequently, when he asks the 80 students in his evolutionary-biology class at the University of Colorado at Boulder to work in small groups to solve a problem, or when he asks them to persuade one another that the answer they arrived at before class is correct.

    When they start working together, his students rarely stay in their seats, which are bolted to the floor. Instead they gather in the hallway or in the aisles, or spill toward the front of the room, where the professor typically stands.

    Mr. Martin, a professor of ecology and evolutionary biology, drops in on the discussions, asking and answering questions, and hearing where students are stumped. "Students are effectively educating each other," he says of the din that overtakes his room. "It means they're in control, and not me."

    Such moments of chaos are embraced by advocates of a teaching technique called "flipping." As its name suggests, flipping describes the inversion of expectations in the traditional college lecture. It takes many forms, including interactive engagement, just-in-time teaching (in which students respond to Web-based questions before class, and the professor uses this feedback to inform his or her teaching), and peer instruction.

    But the techniques all share the same underlying imperative: Students cannot passively receive material in class, which is one reason some students dislike flipping. Instead they gather the information largely outside of class, by reading, watching recorded lectures, or listening to podcasts.

    And when they are in class, students do what is typically thought to be homework, solving problems with their professors or peers, and applying what they learn to new contexts. They continue this process on their own outside class.

    The immediacy of teaching in this way enables students' misconceptions to be corrected well before they emerge on a midterm or final exam. The result, according to a growing body of research, is more learning.

    While the idea is not new, the topic of flipping has consistently cropped up during discussions at recent conferences about teaching and learning—and often when the subject turns to science, technology, engineering, and mathematics, or the STEM disciplines.

    The recent interest is driven by the convergence of several trends.

    The first is technological innovation, which has made it easier to distribute lectures by the world's leading instructors. Some faculty members wonder whether it still makes sense to deliver a lecture when students can see the same material covered more authoritatively and engagingly—and at their own pace and on their own schedule. The supply of such offerings, at low or no cost, is increasing, as demonstrated by recent news of the Massachusetts Institute of Technology's founding of MITx and a Stanford University professor's start-up of Udacity.

    At the same time, policy makers, scholars, advocacy groups, and others who seek to improve higher education want to see more evidence that students are truly learning in college. As pressure mounts to graduate more students, and as cognitive psychology produces new insights into how students learn, these observers say professors can no longer simply pump out information and take it on faith that students understand it.

    Adding to these forces is economic reality. Strained budgets make it difficult for colleges to decrease class sizes and create more seminars in which low student-to-professor ratios allow a high degree of personal attention. Even advocates for new approaches to teaching concede that the lecture is not going away. The lecture model—putting dozens, hundreds, or even thousands of students in a room with a professor—endures because it makes economic sense.

    Flipping allows colleges, particularly large research institutions with big classes, to make the traditional lecture model more productive, says Harrison Keller, vice provost for higher-education policy at the University of Texas at Austin, which held a recent seminar on course flipping for its faculty. "If you do this well, you can use faculty members' time and expertise more appropriately, and you can also use your facilities more efficiently," he says. More important, "you can get better student-learning outcomes."

    Those forces are coming together to prompt a rethinking of the faculty member's role in the classroom. "I see a paradigm shift, and it's coming soon," says Michael S. Palmer, an associate professor of chemistry and assistant director of the Teaching Resource Center at the University of Virginia. "Content is not going to be the thing we do. We're going to help unpack that content."

    Identifying Key Concepts

    Professors have flipped courses for decades. Humanities professors expect their students to read a novel on their own and do not dedicate class time to going over the plot. Class time is devoted to exploring symbolism or drawing out themes. And law professors have long used the Socratic method in large lectures, which compels students to study the material before class or risk buckling under a barrage of their professor's questions.

    The way STEM disciplines are traditionally taught makes them particularly ripe for change, Mr. Palmer says, because of their "long tradition of very didactic teaching, which involved disseminating content." By contrast, he says, the humanities and social sciences have been about exploring ideas.

    Still, flipping has been adopted in isolated precincts of STEM disciplines, particularly physics. Some of the most notable examples illustrate the different forms the technique can take.

    At the University of Michigan at Ann Arbor, for example, the math department has flipped its teaching of calculus since the mid-1990s, says Karen Rhea, a lecturer and director of the introductory mathematics program.

    Michigan offers up to 60 small sections of introductory calculus, with a maximum of 32 students in each class, which meet for 80 minutes three days a week. Faculty members receive intense training: a weeklong course at the end of August, followed by weekly meetings and regular classroom visits throughout the semester from more-experienced instructors.

    Consistent with the flipping model, students at Michigan do their reading before class. The instructor gives a brief lecture, asks them about the reading, and goes through an example from the textbook. Students take turns going to the board to present their answers or working in groups, which might be followed by another short lecture.

    As the students work on the next problem, the instructor circulates. Rather than sending students home to struggle with a new concept, the instructors can hear—and correct—misunderstandings as they arise. "We're asking them to solve problems that are not template problems," Ms. Rhea says. "In your presence they're learning how to think, and we're learning what they're struggling with."

    Class size is not the most important factor in teaching this way, Ms. Rhea says. What's more critical is teaching and testing a set of basic principles of differential calculus that are articulated in a test called a calculus concept inventory. This 22-question test focuses not on whether students can run through calculations but on whether they understand the underlying concepts.

    "It's easy to measure if they can take derivatives out the wazoo," Ms. Rhea says, "but it's kind of harder to see what they're getting underneath."

    Research by Ms. Rhea and two colleagues suggests that Michigan's teaching methods have led to greater gains in conceptual understanding. The techniques have been lauded by the Association of American Universities, among others.

    In 2008, Michigan gave concept inventories to students before they started calculus and after they finished, and calculated the difference relative to the maximum gain they could have made. Students in Michigan's flipped courses showed gains at about twice the rate of those in traditional lectures at other institutions who took the same inventories.

    The students at Michigan who fared worst—a group of 12 who were at risk of failing the course—showed the same gain as those who demonstrated the largest increase in understanding from traditional lectures elsewhere.

    A View From the Lecture Hall

    Michigan's program did not randomly assign its own students to courses using different teaching models, as conventional education research would dictate. But the gains in learning that were observed at Michigan correspond with similar findings about teaching methodologies in physics, which have been documented by Richard R. Hake, a professor emeritus of physics at Indiana University at Bloomington.

    In fact, the project at Michigan was modeled on similar work by physicists, who have been among the most innovative STEM scholars in trying new approaches to teaching and testing the results.

    One of the most outspoken physicists is Eric Mazur of Harvard University. He has been flipping courses for 21 years using a method he calls "peer instruction," in which students work in small groups to answer conceptual questions during lectures. Mr. Mazur recently established a network of practitioners in the technique.

    He began to use peer instruction after testing his own students on the force concept inventory, which predates the calculus concept inventory, and which tests understanding of the foundations of Newtonian mechanics. Despite his consistently high ratings from students, Mr. Mazur saw that they were not learning as much as he thought they were.

    "We put a lot of emphasis on the transfer of information," Mr. Mazur said at a recent conference at Harvard on teaching and learning. But that model is making less sense as sources of information grow more plentiful. "Simply transmitting information should not be the focus of teaching; helping students to assimilate that information should."

    At the conference, he demonstrated how his methods help students absorb information and transfer concepts. He briefly explained an aspect of thermodynamics: When molecules are heated, they move away from one another.

    After asking if there were any questions on this concept, he told the attendees to pick up their electronic "clickers" to answer a question. It was not a simple test of comprehension; he asked people to apply the concept to a new context.

    Imagine a rectangular sheet of metal with a circle cut out of the middle, he said. What would happen to the diameter of the circular gap if the metal were to be heated uniformly? Would the diameter of the hole get bigger, stay the same, or shrink?

    The attendees entered their answers on their clickers. Mr. Mazur told them to find someone sitting near them who had chosen a different answer and try to persuade them that their answer was correct. The room quickly grew noisy.

    I answered that the gap would get smaller, figuring that the material would melt and the hole would start to close. Behind me, a psychologist explained how he thought it would remain the same because the interplay between the expanding metal and the air in the middle would balance each other. We went back and forth, failing to change the other's mind.

    Mr. Mazur ended the discussion and began to move on to a new point when people in the audience started protesting. As it turns out, both my neighbor and I were wrong: The hole would expand, as happens when a jar's metal lid is heated.

    "Once you engage the students' minds," Mr. Mazur said, "there's an eagerness to learn, to be right, to master."

    Active Learning

    But such eagerness is not much in evidence on students' evaluations, says Melissa E. Franklin, chair of Harvard's physics department. While she does not defend the traditional lecture and lauds Mr. Mazur for advancing the cause of teaching, she views flipping with some skepticism.

    Harvard colleagues have tried flipping, Ms. Franklin says, but few have stuck with it. It demands that faculty members be good at answering students' questions on the spot, even when their misconceptions are not yet clear because they are still processing the information.

    It can also be very labor-intensive for faculty members who do not have teaching support, she adds, if it requires a professor to read questions that students submit before class (which is characteristic of just-in-time teaching). "For a normal, straight-ahead professor, there's a steep learning curve," Ms. Franklin says.

    But her chief critique is based on the intensity of students' responses. The average score on a student evaluation of a flipped course is about half what the same professor gets when using the traditional lecture, she says. "When the students are feeling really bad about required courses, it doesn't seem like a good thing."

    Mr. Mazur concedes that some students resist participating to the extent his technique demands. Many students have done quite well receiving information and spitting it back out, he says. But while some come to embrace the flipped classroom, others never do.

    Liking the class is ultimately beside the point, Mr. Mazur says. He says his results from using peer instruction show that, on the force concept inventory, nonmajors who take his class outperform physics majors who learn in traditional lectures.

    "You want students to like class, but that's not the goal of education," Mr. Mazur says. "I could give them foot massages and they'd like it."

    Matt C. Hudson, a senior who is double-majoring in physiology and evolutionary biology, learned to appreciate the flipped classroom while taking Mr. Martin's class at Colorado, just as Mr. Mazur says his students sometimes did in his classes at Harvard.

    "I really was caught off guard at first," says Mr. Hudson, who was initially adamant that students taking a lecture class should be lectured to. About three weeks into the course, his view changed. Mr. Martin split the students into small groups to discuss the heritability of beak sizes in finches, and how that trait related to a bird's chance of survival.

    When a fellow student explained the relationship to him, the link became clear. "Having six or seven ways to think about a problem is better than just having your own way to think about a problem," Mr. Hudson says.

    As both Mr. Mazur's and Mr. Martin's classes indicate, the cognitive strain that flipping imposes on students accounts for much of its success—and the resistance it engenders. Ultimately that strain is what is most important, not whether the course is flipped, says Carl E. Wieman, associate director of the White House Office of Science and Technology Policy. He has documented gains when relatively inexperienced physics graduate students and postdoctoral researchers lecture hundreds of students but stop intermittently to quiz and give feedback on the students' understanding of key concepts.

    Whatever method a faculty member attempts, Mr. Wieman says, he or she should start by defining the underlying concepts to be taught and the learning outcomes that will be demonstrated. And it is not enough, he says, to simply declare that the learning outcome is to cover the first four chapters of a textbook.

    "It's a whole different paradigm of teaching," says Mr. Wieman, likening the professor's role to that of a cognitive coach. "A good coach figures out what makes a great athlete and what practice helps you achieve that. They motivate the learner to put out intense effort, and they provide expert feedback that's very timely."

    Comments: 1