CONFRONTING THE DILEMMAS OF TEACHER PORTFOLIOS
AND NEW TECHNOLOGIES
"Technology is the campfire around which we tell our stories," says musician and performance artist Laurie Anderson. If that be so, what tales will be shared in the flickering glow of teachers' electronic portfolios? Will web and multimedia technologies be used to help teachers capture their knowledge of practice and share it in ways not possible with older technologies? Or will the audience gathered around electronic portfolios hear few meaningful stories because we have failed to see the dilemmas posed by these new technologies and made the complex decisions necessary to use our tools wisely?
In this article I argue that we must recognize the dilemmas of teacher portfolios--those authored with traditional tools as well as with new digital technologies and make careful decisions to address problematic issues. The rapidity with which electronic portfolios are being adopted by colleges of education, school districts, and other agencies gives us special reason to pause, step back, and consider the implications of web and CD technologies. If we fail to recall the purposes for which portfolios were proposed, and instead allow new hardware, software, or evaluation mandates to drive our use of electronic portfolios, we may find they have become "empty containers" (Barrett, 2002c), not the warm, "living history of a teaching and learning life" (Wilcox & Tomei, 1999, p. 5) that proponents had hoped they would become.
My insights on these portfolio issues come from various personal experiences, primarily from case study research I conducted on six preservice teacher portfolios, but also from my experiences as a teacher educator initiating an electronic portfolio program at a small liberal arts college, and as an assistant professor who is herself authoring an electronic portfolio for tenure review.
The focus in this article will be on reporting findings from my research that illuminate what I see as the key decisions and dilemmas educational institutions seeking to adopt electronic portfolios must face. Other articles in this collection will consider these same issues from other perspectives. Before discussing decisions and dilemmas, I will review the purposes for which teacher porfolios were proposed, so that we may evaluate our options in the light of our goals.
Those participating in early efforts to develop teacher portfolios (i.e., Shulman, 1998) noted that portfolios were a very powerful learning experience for the teachers who constructed them. Thus, portfolios also began to be recognized and endorsed as tools for reflective thinking about one's teaching practice and for professional development.
Finally, teaching portfolios can play an assessment role. Through the portfolio, teachers can demonstrate achievement of high professional standards--the idea of portfolio as credential.
These three roles are reflected in the various ways portfolios are currently being used for teachers. Although they have been adopted by state licensing agencies, K-12 school districts, and professional organizations (i.e., NBPTS), portfolios have been most widely adopted by college of education for preservice teachers. My research on preservice teacher portfolios gave me some insights into how portfolios function to achieve these three purposes and what problems might be associated with their use when authored with various technological tools.
I argue here that wise decisions must be made in three key areas if one wishes to devise effective requirements and rubrics for portfolio assessment systems, and that each of these decisions entails one or more dilemmas--many of them tool-related. The three key areas for decision-making include:
1. purpose/audience2. ownership3. focus
As I discuss decisions that must be made in each of the three key areas, I will refer to evidence from my case studies that illuminates the issue at hand, speaking first about findings that cut across both traditional and electronic cases, then drawing out the implications of the technological tools used for electronic portfolios.
Decision 1: Defining Purpose and Audience
Each portfolio program must define the purposes for which a portfolio is to be authored. In my case studies I found that conceptualizations of purpose and audience were two sides of one coin--for that reason, I examine them concurrently here. I begin by considering the most common purposes for portfolios, then discuss the audiences these purposes entail, and finally explore the dilemmas inherent in each option.
Purposes for Portfolios
As the preservice teachers at the university where I conducted my study began a portfolio seminar during the final term of their program, several possible purposes for doing a portfolio were suggested. Each author had to decide which of these purposes, or combination of them, to embrace. How one defined purpose in large measure determined audience. This reciprocal purpose/audience decision proved to be the most important factor influencing portfolio form and content.
Helen Barrett (2001b, quoting Ken Wolf, 1999) has noted that portfolios can have three general purposes:
The syllabus also suggested three other possible purposes: the portfolio could be the beginning of a program of continuing professional development (formative), a device to get a job (marketing), or a way to learn more about portfolio processes (formative). How these purposes were appropriated and prioritized by the portfolio author I found to be highly significant.
Most of my study participants reported that they were resistant to doing the portfolio when they first began. Their resistance may be more pronounced because of the nature of this teacher education program--at the time of my study, the entire portfolio was produced in the fifth quarter, after student teaching. As way of making the huge task more palatable, those in charge of the TEP portfolio program tried to emphasize the portfolio's multiple purposes--especially its usefulness for the job search. As a result of this decision, five of my six participants tried to achieve two or more purposes in doing their portfolio. Having multiple purposes meant writing for multiple audiences--and this is what made things particularly difficult for the authors. A dilemma lurks in this decision to make the portfolio multi-purpose.
The Multiple-Purpose Dilemma
Teacher education programs having their students construct portfolios face a dilemma when it comes to purpose and audience: In suggesting teacher portfolios might accomplish multiple purposes, we make it likely they will accomplish none of them well; yet suggesting the portfolio is for one purpose only--to meet a degree requirement, for example--arouses a certain amount of recalcitrance among preservice teachers, especially among those who are not naturally reflective or who find writing to be laborious. Let us explore this dilemma.
Two purposes are most commonly advanced for preservice teacher portfolios: to demonstrate teacher competencies (for credentialing or employment) and to foster reflective thinking. These two purposes may be incompatible if they imply audiences with quite different needs and perspectives, as they generally do.
Telling teachers that their portfolio will be useful for getting a job, a degree, or some other credential, motivates them to accomplish the task, but can undermine the portfolio's usefulness as a reflective tool and honest representation of self. Conceiving of one's portfolio as a high-stakes assessment that will be read by potentially critical readers induces portfolio authors to gloss over the deficiencies of their teaching to present evidence of mastery. Yet, it is in reflecting on ones pedagogical problems, not one's successes, that teachers are most apt to learn. Portfolio authors must be willing to engage in critical exposure for portfolios to be effective as devices for reflective practice. By critical exposure, I mean statements that expose weaknesses or problems in the author's teaching and subject them to critical analysis.
Critical exposure. In my study, the one participant who was unconcerned about his portfolio as either job artifact or high-stakes credential was the most willing to engage in critical exposure. Roger, a preservice teacher being certified in physics, produced a traditional paper portfolio. This portfolio is the leanest of the six I studied, and the one with the fewest frills--entirely black and white text, with no graphics, except on a few scattered worksheets that had been used with students. When asked what was significant about his portfolio, Roger said, "Anything significant about it? No. It's just a lot of words" (Carney, 2001, p. 137).
Constructing a portfolio was not something Roger was eager to do. His purpose was simply to meet program requirements and get his degree. For that reason Roger said he didn't make any special efforts to dress up his portfolio and make it attractive to the reader: "I don't picture mine as really visually attractive or pleasing, I mean it's very bare-bones black and white, just a lot of text because I wasn't motivated or inspired to come up with a theme and really turn it into something that would be exciting, interesting to look through--at least for me it's not" (Carney, 2001, p. 138).
Why was Roger not motivated to make his portfolio something exciting and interesting? Because he was not worried about it as a requirement for getting his degree or for getting a job. As Roger said, " mean, I guess my priority was to meet the requirements, and I met the requirements so I didn't have to do anything above and beyond that, and I really didn't see it as something that I would need in the near future to get a job" (Carney, 2001, p. 138).
Roger didn't think it likely that a prospective employer would take the time to read his portfolio. Being a physics and math teacher, with school districts eagerly calling him with job offers might be why impressing a principal with his portfolio didn't seem necessary. As a result, Roger was able to write for a single audience, an audience he didn't seem to consider terribly worrisome. His purpose was simply to meet TEP requirements, and he knew just what a TEP audience would want to see: evidence of reflection; he gave the reader 10 critical comments identifying problems from his student teaching and suggesting alterations in instructional methods.
Roger was the only participant in my study who limited himself to achieving just the dual TEP purpose for doing a portfolio, and thus was able to write for one audience. The others had multiple purposes and tried to adapt their portfolios to the needs of multiple readers. This decision to write for multiple audiences, especially if the author didn't clearly prioritize one group of readers, made the authoring task extremely difficult and resulted in a portfolio that was not well-adapted to the needs of any group of readers. I found that portfolio authors were not able to fully achieve the multiple purposes these audiences represented.
Conceiving of the portfolio as a marketing device and writing for an audience of school administrators was particularly problematic--participants who wrote for this purpose and audience had the fewest and least substantive critical self-exposure remarks or, in the case of one electronic portfolio author, used affordances of the tool to "hide" his most candid assessments of self (Carney, 2002).
I suggest it is not simply the school administrators per se who are the problem here; it is that they generally represent a high-stakes audience whose needs and perspectives are contrary to the demands of another high-stakes audience--evaluators from the authors' teacher education program. The portfolio authors in my study were being asked to write at cross-purposes. Roger solved the multiple-purpose dilemma by appropriating just one portfolio purpose for himself, and by deflating the high-stakes nature of even that task.
These issues of purpose and audience cut across the paper/electronic divide, but for the electronic portfolio authors (who posted their portfolios on the web), audience posed some special concerns, which I will now consider in discussing the implications of the technological tool.
Implications of the Technological Tool for Purpose and Audience
What effects did the technology used for authoring electronic portfolios have on these purpose and audience issues? I found that the affordances of the technology exacerbated portfolio authors' concerns about self-revelation, creating a new dilemma--yet, at the same time offering the potential for dealing with the multiple-purpose dilemma.
Two of the three electronic portfolio authors expressed strong concerns about revealing personal information or speaking candidly about the problems they might have faced during student teaching. Both indicated they had left important things out of their portfolios as a result of these worries. (The third electronic portfolio author was unabashedly writing for the purpose of obtaining a job, and didn't feel the same tensions; he was intentionally avoiding personal revelations.) Maggie, a language arts electronic portfolio author, spoke of her unwillingness to reveal mistakes in her portfolio because she felt this would be like "advertising" herself as a finished product to an imaginary person on the web:
It's not like I'm afraid to say, I made a mistake here, and this is what I would do differently; there are just like different levels of mistakes, you know there are some that are so fresh and awful...you don't just open those up to people who might not treat that wound kindly...I probably wouldn't do a portfolio in the future on the web, because it's just a spot where I am now, it's not, and I'll be changing from that spot, and I wouldn't want...I guess I'm just really hypersensitive to some imaginary person thinking that I think that where I am now is good enough, that I think that where this portfolio is right now is, shows the best teaching that I can ever do. That's something I really don't like about publishing it on the web, you know...like advertising this is who I am. And because, I guess I feel very fetus-like still (Carney, 2001, p.71).
My study shows that preservice teachers who post their portfolios on the web are particularly worried about potentially critical readers. Publishing on the web offers portfolio authors the great affordance of being able to communicate with a wide audience, anywhere in the world. Paradoxically, this affordance is actually a constraint for preservice teachers, however. Insecure in their knowledge of a new profession, they are reticent about revealing to unknown readers how they struggle with problems of practice--their "wounds," as Maggie called them.
This poses a personal-revelation dilemma for teacher education programs that encourage their preservice teachers to share their portfolios widely--especially if it means posting a portfolio on the web. The technology that affords teachers the opportunity to share teaching knowledge widely may prove too revealing for novices to deal candidly with problems of practice.
In contemplating personal revelations, participants who authored traditional portfolios seemed most concerned about prospective employers who might later be readers. The participants who posted their portfolios on the web, on the other hand, worried not only about an audience of school administrators, but also about other unknown, potentially critical readers who might be part of a vast audience on the Internet. Unlike traditional portfolio authors, who could at least decide who had access to their finished portfolio, those who posted their portfolios on the web gave up that control. (I will return to this and other issues of control when I discuss the second key area for decision-making--ownership.)
The issues underlying this personal-revelation dilemma have not gone unrecognized by colleges of education. A number of technological solutions to this problem have been widely adopted: many universities have created password-protected environments and posted their portfolios where only cohort peers or university and school teacher mentors can access them. In fact, this has become general practice since my study. Another technological solution to the personal revelation dilemma is to produce the complete portfolio on CD, uploading only a showcase version to a web server.
The hypertextual capacities of web-editing software can address both the multiple-purpose and personal-revelation dilemma. Using hyperlinks, it is possible in electronic portfolios to take a "growth" portfolio compiled for formative purposes, perhaps with documents accumulated gradually throughout one's teacher education program, and create multiple paths through the body of artifacts--each path appropriate for a particular audience. In fact, a teacher could have any number of portfolios on CD--a formative portfolio showing deep critical reflection, a summative portfolio demonstrating achievement of standards, a marketing portfolio displaying one's finest moments. In this way, teachers no longer have to take on multiple purposes and write for audiences with different needs and perspectives.
Although current technology does not entirely solve the personal-revelation dilemma, it can ameliorate it, by allowing the author to exert control over who sees the most sensitive parts of his or her portfolio. However, these solutions do not address another dilemma related to purpose: the cognitive-overload dilemma.
The Cognitive-Overload Dilemma
Colleges of education have expected electronic portfolios to accomplish many of the same purposes as their traditional portfolios--but they very often have one additional purpose as well. Constructing an electronic portfolio is frequently viewed as a means of developing advanced computer skills among preservice teachers. Many teacher education programs see electronic portfolios as a way to help their students achieve the competencies specified in the National Educational Standards for Teachers (NETS*T).
Does the process of authoring an electronic portfolio develop advanced computer skills? I would suggest it may depend on what kind of electronic portfolio one constructs--that is, what kind of technology is used to author it. If the teacher is using generic software tools (GT) for web editing and multimedia authoring, one would develop a considerable number of the NETS competencies in successfully authoring an electronic portfolio. If, however, the electronic portfolio is part of a customized systems approach (CS) that relies on a server with specially-designed software and database functions, the skills required may be little more sophisticated than word processing.
In my study I did not compare the technology skills my participants had as they began the authoring process versus their skills as they completed the task; however, from observation, as I have taught classes in electronic portfolio design in two teacher education programs, I am able to say unequivocally, that constructing an electronic portfolio with generic tools is an excellent example of authentic, problem-based learning. Students must learn a great deal about technology to complete the task. Using GT can make the portfolios a device that drives the development of advanced technology skills. We must consider whether we are willing to forego this authentic learning activity in favor of systems that do not require preservice teachers to develop new technology skills, or only limited additional skills.
As we make decisions about what technologies to use for electronic portfolios, we ought to keep our goals in sight. Is one of our purposes in having preservice teachers author electronic portfolios the development of advanced technology skills? If so, we may have to counterbalance the advantages of a customized system with the learning opportunities of generic tools. Yet, even this question about skill development is a bit more complicated than it appears; another dilemma intrudes as we attempt to answer it.
Both in the case study research central to this article (Carney, 2001) and in a small previous study where I analyzed three electronic portfolios for evidence of reflective thinking (Carney, 1999), I saw evidence that made me wonder if perhaps learning the technology as one is trying to reflect deeply about one's practice may result in a "cognitive overload." In both studies, the preservice teacher who took on the most challenging technological tasks, relative to his or her capabilities, produced the least successful portfolio in terms of content.
In my 3-part case study, two experienced evaluators found the reflective thinking in the most technologically-advanced portfolio to be below minimum standards for the program; one called it "just plain inadequate" (Carney, 1999, p. 14). In my larger case study, I analyzed for evidence of portfolio authors' pedagogical content knowledge, and noted that the electronic portfolio author who had the most limited technology skills produced a portfolio that was deficient in both number and quality of representations as well as strategies for teaching. While there are a host of other explanations that could explain these inadequacies, it is certainly possible that students who are working at the limits of their technological competency may have little time or cognitive energy available for thinking deeply about their teaching practice.
This problem is most acute when using generic tools for portfolio authoring. Not only is the author trying to learn the technical aspects of the software, but she/he is also trying to deduce principles of information design--for a genre that is just emerging. Electronic portfolio authors who use GT have to think through an enormous number of complex issues involving organization, layout, color, graphics, typography, and so on. They have to become information designers as they master the hardware and software. To create documents for a new and unfamiliar medium, where even specialists are trying to deduce principles of effective design, can be a daunting task.
When authoring traditional portfolios, or those done using the most fully developed customized systems (CS), not only are the technological demands on the portfolio author significantly reduced, but they also don't have to attend as much to design. Highly literate preservice teachers have absorbed most of the conventions of traditional printed text.
The cognitive-overload dilemma can be stated in this way: the technology that is most effective in helping preservice teachers develop high-level NETS competencies, may at the same time make it difficult for the portfolio authors to achieve important portfolio goals related to content. Yet on the other hand, using the most fully-developed customized systems or traditional authoring tools will mean giving up an excellent opportunity for developing preservice teachers' technological skills--itself an important electronic portfolio purpose for many programs. What shall we do?
The issues implied by this cognitive-overload dilemma clearly need additional research. Are preservice teachers whose institutions have adopted CS for portfolio authoring as successful in developing NETS computer competencies as those who use generic tools? Are preservice teachers whose institutions use generic tools as successful in achieving portfolio content goals? Leaders in the field of electronic portfolios have called for a national research agenda in this area (i.e., Barrett, 2002a), but institutions designing teacher portfolios are finding they must choose tools for authoring without clear evidence of the how different systems impact portfolio content and competency goals. I point out this dilemma so that decision-makers are at least aware of the implications of their choices.
If we expect teacher portfolios to achieve the goals for which they have been proposed, we must make careful decisions about the purposes and audiences for which we ask teachers to write. In large measure, content and format will be determined by these choices. Yet our decisions will not be easy ones; we must grapple with at least three dilemmas: the multiple-purpose dilemma, the personal-revelation dilemma, and the cognitive-overload dilemma. The affordances and constraints of different technological tools are inherent to these dilemmas.
Besides defining the purposes for which a portfolio is to be authored, portfolio programs must also negotiate ownership--the second key of decision-making, which I will now discuss.
Decision 2: Negotiating Ownership
Ownership means a number of different things when we are speaking about portfolios: generally, we think of it in terms of who is allowed access for reading the portfolio, what tools are used for its construction, and where the portfolio will be stored. I believe there is a more basic ownership issue involved also--to what extent the portfolio is an expression of the individual who has authored it. I will begin with this point, which I think is a crucial decision for teacher education and professional development programs to make, and one that is often ignored.
Who Am I as a Teacher?
Portfolios have been proposed as devices to help teachers conceptualize and represent their own practice. In selecting artifacts, a portfolio author is saying, "This is who I am as a teacher--me doing what I think is good teaching." With this portfolio role in mind, I would argue that one of our goals in designing portfolio programs ought to be giving the portfolio author sufficient control over process and product to ensure that the portfolio is truly an expression of me. I believe this is the most basic issue of ownership.
of my study participants spoke of how her traditional, language arts, paper
portfolio played the theoretical role of helping her define self as teacher:
I really came to believe through the process of going through it all that it really did help me figure out who I am as a teacher. It really did. One of the things in your student teaching is that you know, a lot of it isn't you, and I think it took me several weeks after I was done to kind of sort which was me and which wasn't me (Carney, 200l, p. 43).
I think the reason I'm so pleased with my portfolio is that I think that it really looks like, it really has items in it that are true to me, and I'm so relieved because when I turned it in, I frankly was just glad that it was finished and it was turned in. And then, when my mentor gave it back to me yesterday, and I looked through it, I thought, "Oh! This is right, I was honest, it's true," and that was a really nice feeling (Carney, 2001, p. 85).
What I found even more striking was the manner in which portfolios represented self as teacher in a metaphorical sense too--the authors thought of the portfolio as actually being me. To read the portfolio was a way of interacting with the author and experiencing the author's pedagogy. For example, Kyle wanted the reader to have multiple paths and have to "figure out answers on your own," just as he did in his teaching. Doris' portfolio was a "Metaphor to Action." Mark, a social studies teacher who authored an electronic portfolio, found ways to incorporate the multiple perspectives he believed central to his pedagogy. Maggie felt herself to be "close in on the spiral" and produced a portfolio that was self-consciously rudimentary as well.
The actions of these portfolio authors are in accord with Mary Diez's (1994) conception of a portfolio as mirror, map, and sonnet. These portfolios allowed the owner and reader to view the self over time, to show a plan for learning and development, and to provide a venue for expressing self artistically. I believe portfolio programs need to make decisions that will allow authors sufficient rights of ownership to fully express themselves in all of these areas. The requirements and rubrics devised by this teacher education program had allowed portfolio authors discretion in the choice of artifacts, and control over portfolio format. They had sufficient ownership to express themselves theoretically, artistically, and metaphorically. I suggest that all programs ought to consider their own portfolio requirements and rubrics to determine if they allow authors freedom of expression; they should also carefully consider whether the technological tools provided or mandated allow that same freedom.
Implications of the Technological Tool for Ownership
All of the electronic portfolio authors in my study used generic software tools for web page design. Thus, they had full control of the organization and graphic design of their portfolios. Will portfolios authored within customized systems allow students the freedom to express themselves artistically, creating metaphors for who they are as teachers? We find ourselves with another tool-related dilemma--the self-expression dilemma.
The Self-Expression Dilemma
To be able to express who they are as teachers, portfolio authors need some facility with the tools they will be using for communication. When the author has adequate skills, generic web editing tools can enable him to create something very true to his intentions. In my study Mark noted:
Originally I thought that doing it on the web would make it less possible to do attractive things and innovative things; I thought that I would be more restricted creatively by doing a web portfolio, but once I got into it, I realized that the opposite is true, that I would actually be more confined had I done a paper portfolio because...I guess because it's so easy to change a color and have it be exactly the way you want it, you know, you don't have to go to the paper store and buy a certain color paper, and then if you're dissatisfied with it you've gotta buy more paper. On the web there's software where you can have the exact color you want. Or you can have the exact background you want. I couldn't have found...paper that has old Spanish looking galleons watermarked, but on the web, you can do that. And so, it's made it more creative, not less. Once I realized that, that's when I got really excited about doing it this way (Carney, 2001, p.120).
Customized portfolio software systems raise another concern related to this self-expression dilemma. Using fully-developed customized portfolio systems might result in teachers lacking the skills to author their own portfolios outside the CS. By producing teachers who lack the technology skills to do web authoring and multimedia design with off-the-shelf tools, we may be, in effect, disempowering them. Might this be yet one more example of how a top-down effort to simplify teachers' work takes ownership away from them? This possible disempowerment has implications for continuing professional development, and lead us to one final dilemma related to ownership.
The Dead-End Dilemma
Teacher portfolios have been proposed as a vehicle for continuing professional development--for that reason we would want authors to conceive of their portfolios as the beginning of a process rather than a finished product. Unfortunately, the three participants in my study who authored traditional, paper-based portfolios saw them as a finished product--something to put on a shelf or in a drawer; none had any plans for further development. If one intends the portfolio to be a working document and part of a teacher's program of professional development, these traditional preservice portfolios represent a dead-end. On the other hand, the three preservice teachers in my study who used generic web authoring software to construct electronic portfolios indicated they were likely to continue theirs. Hannah herself noted the differences between paper portfolios like hers and the electronic ones done by her classmates:
I was terrified to do an electronic portfolio, and I thought, with the stress in doing this, to also have to learn a new technology on top of it, I'll kill myself, or I'll kill my family, or I'll explode. I was just afraid. And then I look at Mark's on the web, because I can pull it up...oh it's so COOL!! It's alive. He can adjust it, change it, and you can link it to other things that are of value and of interest to him. It's bright, it makes noise...His feels alive, like something that will live and change and adjust...it feels more accessible to more people. It IS...more accessible to more people. This will sit on my shelf. In a place of glory, but on my shelf nonetheless. His will be out there alive, that's really cool. I wish I was brave (Carney, 2001, p. 96)
School districts may face another complication: they employ teachers who have graduated from many different teacher education programs; if the customized system approach predominates, teachers will most likely be coming into a district with portfolios constructed using widely different software systems. This limits school districts' abilities to use these portfolios for common assessment and continuing development. School districts will also have no way of ensuring that their teachers' portfolios, which are mounted on other institutions' servers, will be supported.
School districts will most likely have to develop their own customized software systems so as to standardize district teachers' portfolios. Thus newly-hired teachers may have to begin anew, rather than continuing to develop their preservice portfolios. If this scenario were to play out, preservice CS portfolios would be nearly as much of a dead-end as traditional portfolios. With a portfolio constructed with generic tools, on the other hand, teachers can move from college of education to school district, or even from state to state, and still be able to continue development.
Yet, on the other hand, as school districts and states ponder the types of tools they will choose for their inservice teacher portfolios, they must consider just how many of their practicing teachers have the technological skills or the time to author portfolios with generic tools. And with limited district professional development funds, how long would it take to develop those skills? Here is where large-scale customized portfolio systems could hasten the adoption of portfolios as a common device for teacher assessment and professional development. Minnesota's customized portfolio system (Avenet) is a good example of how a fully-developed CS, by removing the technological-competence barrier, can enable teachers at all educational institutions within the state to share artifacts of practice.
A discourse of practice. Knowledge is developed in a profession as practitioners talk about problems of practice with others in their professional community. Teachers have traditionally lacked the tools and venues for communicating their professional knowledge. Kerr (2000) draws on studies of teacher video clubs (Frederiksen, Sipusik, Gamoran, & Wolfe, 1992; Gamoran, 1994) to suggest that the multimedia capabilities of web technology may allow the teaching profession to develop a new "language of practice." Work done by Lampert and Ball (1998) on the M.A.T.H. project gives us some idea how the languages of video, audio, text, and graphics can be brought together in a multimedia computer environment to create rich representations of what teachers do and know.
By providing a structure for discourse about artifacts of teaching and learning, electronic teachers' portfolios are one place where a teachers' language of practice could develop. Customized electronic portfolio systems might enables us to establish that discourse in online communities of teachers. Wineburg (1997) saw portfolios as opportunities for social learning--these web portfolio environments may extend that social learning beyond the confines of the local setting.
Yet, on the other hand, fully-developed customized systems like Minnesota's Avenet, implemented by additional states, could remove ownership of portfolios from both teacher education programs and teachers themselves. This concerns me. The long, sad history of top-down professional development for teachers makes me think we ought to be cautious about efforts to systematize portfolios at the state level. For years, the "experts" have tried to do something similar with curriculum and instruction--trying to make the complex decisions at the heart of good teaching "simple." I don't think we want to "teacher-proof" the portfolio.
Thus, the final dilemma of ownership--dead ends or continuing professional development--can be stated in this way: large-scale customized electronic portfolio systems offer us the potential for sharing teachers' wisdom of practice widely and providing for long-term professional development; but, if we don't make wise decisions in designing and using our tools, customized systems might inflict yet more rigid requirements on teachers and make the promise of portfolios yet another dead-end.
This concludes our discussion of the decisions and dilemmas related to ownership. The personal-revelation dilemma and the dead-end dilemma make our choices difficult.. Once again, making wise decisions will call for an awareness of the affordances and constraints of our technological tools and a willingness to face problematic issues of portfolio ownership.
Decision 3: Determining Focus
A final area crucial for decision-making about portfolios involves a question of focus: What shall we focus on in devising our requirements and rubrics? I found in my study that small decisions about requirements and rubrics can have big implications for the content of a portfolio.
In an earlier section, I noted that the portfolio authors in my study made few comments that exposed weaknesses or problems in their teaching. Despite being in a teacher education program that emphasized reflective thinking, these preservice teachers identified few troubling puzzles of practice and did minimal critical analysis of the problems they did identify. Why so little willingness to admit deficiencies in their novice practice?
Requirements and Rubrics
I suggested one reason for a paucity of critical exposure comments had to do with audience concerns and the high-stakes nature of the assessment. A second reason, I believe, is due a lack of focus on student learning in portfolio requirements and rubrics.
The portfolio authors in my cases do not provide evidence that they had thought carefully about what students learned during their lessons or considered what significant changes might be necessary to better adapt instruction to particular students' characteristics. Why weren't these preservice teachers focused on student learning? One part of the explanation lies in TEP portfolio requirements. The types of artifacts required in the portfolio are focused on teacher action--things like lesson plans, papers written for university courses, classroom management plan, evidence of subject matter knowledge, and so on. Student work and other items showing the results of instruction were all optional.
The number of student work samples included in the six portfolios ranged from three to eighteen, but authors did not engage in careful analysis of those pieces. Furthermore, the student documents chosen are not the most appropriate for prompting teacher thinking about student understanding. Nearly all the student work samples in these portfolios are full-credit or "A" papers. Samples of mediocre or poor student work might be more useful in a portfolio than assignments with no errors. Papers that reveal student misconceptions or learning difficulties would be more likely to induce preservice teachers to ponder why their instruction had been ineffective for some students. However, unless portfolio guidelines require examples of poor student work, preservice teachers are unlikely to voluntarily include or analyze it in their portfolios; quite understandably, they are not eager to advertise the inadequacy of their teaching-- especially in a high-stakes assessment.
I believe the relative lack of attention to student learning in these portfolio is partially due to the paucity of student work samples and the nature of those chosen, but the portfolio rubric also had a role in narrowing reflective thinking. The rubric for entry slips that accompany each artifact prioritizes reflective thinking about program goals and the socio-political-historical implications of teaching rather than student learning.
The entry slip rubric had significant effects on the thinking of portfolio authors. The device helped them examine what they had done during student teaching, positing multiple perspectives to interrogate those experiences for new learning. However, subtle features of the tool had what I consider to be some unfortunate constraining effects, a point I will discuss.
descriptors for a "standard" and "above standard entry slip are shown in
"above standard" entry slip
Entry slips demonstrate thoughtful, knowledgeable reflection on chosen goals and targets; connections between evidence and main points are clear. Offers multiple perspectives and/or interpretations. Occasionally considers larger social, political, or historical forces reflected in the classroom (Carney, 2001, 222).
Entry slips demonstrate thoughtful, knowledgeable
reflection on chosen goals and targets, presenting a coherent and detailed
picture of student thinking. Consistently, offers multiple perspectives
and/or interpretations and provides evidence of understanding larger social,
political, or historical forces reflected in the classroom (Carney, 2001,
Figure 1: TEP Rubrics for an Entry Slip
I found that portfolio authors proved to be consistently mindful of this rubric. Because this tool was guiding activity, authors did similar things in their entry slips. Generally they began by talking about how a given artifact demonstrated achievement of program goals. They then posited a perspective different from their own, which they use to probe their methods. Usually this meant expressing a rationale for their chosen pedagogical actions. Portfolio authors were also quite consistent about considering the larger social, political, or historical implications of their pedagogical decisions, as the rubric clearly directed them to do.
There were some things authors consistently did not do, however. They did not regularly reflect on student learning. Notice in Figure 1 that the standard for portfolio does not require these preservice teachers to consider what students may have learned from their lessons. Only when one reaches the "above standard" category is student thinking mentioned, and then in a clearly subordinate position. Reflection on program goals and the socio-political-historical implications of teaching was afforded by this rubric; at the same time it inadvertently constrained authors from reflecting on student learning. Since the artifact requirements had already prioritized items showing teacher action over student artifacts, the effect of this rubric was to channel thinking away from a serious consideration of how individual student characteristics may have impacted learning.
Institutions with portfolio systems need to make decision about what teachers' ought to be focused upon. They then must carefully design requirements and rubrics to channel portfolio authors' thinking in those area. Psychological tools such as these have affordances and constraints just as technological tools do. They had significant effects on the type and quality of reflective thinking in the portfolios I studied.
Recommendations for Focus
What should be the focus in teacher portfolios? Many portfolio programs are now focusing upon nationally-recognized standards (i.e., INTASC and NETS*T), rather than local program goals. Yet these broad descriptors are not likely to be sufficient guides to teacher thinking and portfolio authoring. My study demonstrates that program requirements and rubrics are equally important. Where shall we direct our focus in designing these structures?
I would argue that no matter what standards underlie teachers' portfolios, a primary area of focus ought to be artifacts of student learning. Ball and Cohen (1999) have recently proposed a "practice-based theory of professional education." They call for a new emphasis in teacher education (preservice and inservice) on the investigation of practice--making "systematic study and analysis of learning the core of professional education" (p. 16). Portfolios could provide a structure for the kind of practice-based teacher learning Ball and Cohen advocate--if we put concrete records of student learning at the center and use them to prompt deep reflection about one's teaching practice.
Portfolio authors will most likely be predisposed to emphasize teacher performance; our requirements and rubrics ought to prompt them to remain focused on student learning. Student work samples need to be included--and they ought not be limited to exemplary samples. Poor quality student work should be carefully analyzed for what it reveals about the inadequacies of a teacher's pedagogy.
However, this kind of reflective analysis of ones weaknesses will never be done for any but the most trusted of readers--something that will not occur unless an author has control of his or her portfolio. This leads us to one final dilemma at the core of the movement toward customized portfolio systems--the data-aggregation dilemma.
Implications of the Technological Tool for Focus
Portfolio authors who use generic tools can have some assurance that they own their portfolios--the files are portable property that can be uploaded to particular servers at their will. Particular versions of a portfolio can be burned on CD and distributed as the author chooses. A portfolio constructed within a customized system, however, is by its very nature, a rather different entity. In many cases, it is not fully portable; it exists within a database on an institutions' server and often can only be viewed within a proprietary interface. CS portfolios are "data" owned by institutions.
The Data-Aggregation Dilemma
This ownership issue has implications for what I recommended we focus upon in portfolios. It also suggests a new dilemma. Customized portfolio systems are designed for the aggregation of data and are being developed by large educational institutions with high-stakes interests in that data. For colleges of education, the data in online portfolios may determine if they achieve or retain NCATE accreditation; for school districts, the data will demonstrate whether they are obeying state mandates and employing competent teachers; for states, the data will prove they are following legislative directives and may result in additional funding.
Might there be a conflict of interest here? Are colleges of education really going to encourage novice teachers to provide evidence of their deficiencies if their program approval is based on evidence of proficiency? Are school districts eager to show state officials and local communities that not all of their teachers are competent? Will these institutions be able to convince teachers they will be safe in sharing their "wounds of practice?" Or will this technology contribute not to the emergence of a true discourse of practice, but instead more defensive posturing by teachers imbued with the idea that the problems of their teaching practice ought to be hidden.
The data-aggregationdilemma can be stated in this way: new technologies give us the opportunity to do what reformers have thought necessary to make teaching a true profession--establish standards of effective practice and devise authentic assessments that are not only valid measures of proficiency, but also inducements to continuing professional development. New customized portfolio systems have the potential for establishing extensive professional dialogue around artifacts of practice. Yet these new technologies might also worsen existing problems in the profession: our tools might inhibit teachers from dealing honestly with the difficulties they experience in the classroom, and we may inadvertently contribute to the current culture of isolation among teachers. We must make wise decisions in choosing, using, and developing these tools.
In this article, I have used findings from my own small case study research on preservice portfolios to shed light on some of the decisions and dilemmas I think we need to face if we expect portfolios to play a role in reforming the teaching profession. In framing the discussion, I have deliberately simplified the alternatives--presenting some of our decisions as clear choices between two different families of tools (the GT versus CS debate) but of course, these categories contain various software with a wide-range of characteristics. And neither GT nor CS are static--both are continuing to evolve. As we consider our options, we need to draw software designers into the discussion. With a clear understanding of our goals for portfolios, and an awareness of the dilemmas that might derail us, we educators should be able to press for generic tools that are more user-friendly and customized systems that are not simply assessment-management databases, but environments where genuine portfolios can be created.
My research convinces me that Shulman and others were correct in predicting the roles teacher portfolios can play. In the six cases I studied, preservice teachers verified that they were indeed using their portfolios to present a portrait of self as teacher, and then to compare that image with the ideal they had formulated in their philosophies of education. Their teacher education program had put into place an authentic form of assessment that was itself a powerful learning experience. And these portfolio authors had produced an artifact that could be used to credential them as qualified teachers. The device isn't working perfectly, but it's better than any other teacher assessment method at hand. As new technologies give us capabilities for representing and communicating teacher knowledge in ways never before possible, we ought to consider how we might make portfolios ever more effective in achieving our goals.
Pearl and Leon Paulson have said, "Portfolios tell a story...put in anything that helps to tell the story" (1991, p. 294). I believe web and multimedia technologies offer us campfires around which we might gather to hear the stories of teachers. Electronic portfolios could be places where practitioners communicate who they are as teachers--revealing both their triumphs and their uncertainties. This will not happen without careful thought and planning, however. Will we have the wisdom to recognize the implications of our decisions and resolve the dilemmas inherent in them? Or will our decisions result in there being no interesting stories told in the glow of the electronic campfire? This article is intended as a spark to more conversation about these issues. Let us use technology's campfire to draw teachers together--giving them a place to sing their songs, find healing for their wounds, and be illumined by the wisdom of their peers.
Ball, D. L., & Cohen, D. K. (1999). Developing practice, developing practitioners: Toward a practice-based theory of professional education. In L. Darling-Hammond & G. Sykes (Eds.), Teaching as the learning profession (pp. 3-32). San Francisco: Jossey-Bass.
Barrett, H. (2002a). Researching the Process and Outcomes of Electronic Portfolio Development in a Teacher Education Program. Paper presented at the Society for Information Technology in Teacher Education Conference, Nashville, March 17-23.
Barrett, H. (2002b). Using technology to support assessment in teacher education. Presentation at the forum on assessment and technology conducted at the National Educational Computing Conference, San Antonio, June 24-29.
Barrett, H. (2002c). Symposium on electronic portfolio development. Presentation at the PT3 Grantees Meeting, Washington, DC, July 25-28.
Carney, J.M. (1999). Teacher education portfolios on the web: New medium affordances, new needs for readers and writers. Unpublished manuscript, University of Washington.
Carney, J. M. (2001). Electronic and traditional paper portfolios as tools for teacher knowledge representation. College Park, MD: Educational Resources Information Center (ERIC Document Reproduction Service No. TM033702.)
Carney, J.M. (2002). The intimacies of electronic portfolios: Confronting preservice teachers' personal revelation dilemma. Paper presented at the Society for Information Technology in Teacher Education Conference, Nashville, March 17-23.
Diez, M. (1994). The portfolio: Sonnet, mirror and map. In Kay Burke (Ed.), Professional Portfolios. Glenview, IL: Skylight Training and Publishing.
Frederiksen, J. R., Sipusik, M., Gamoran, M., & Wolfe, E. (1992). Video portfolio assessment: A study for the National Board for Professional Teaching Standards . Princeton, NJ: Educational Testing Service.
Gibson, J. J. (1979). The ecological approach to visual perception. Boston: Houghton Mifflin.
Kerr, S. (2000). Technology and the quality of teachers' professional work: Redefining what it means to be an effective educator. Paper presented at the CCSSO Educational Technology Leadership Conference, Washington, DC., January 13-14.
Lampert, M., & Ball, D. L. (1998). Teaching multimedia and mathematics: Investigation of real practice. New York: Teachers College Press.
Paulson, P. & Paulson, F.L. (1991). Portfolios: Stories of knowing. In P.H. Dreyer (Ed.), Claremont Reading Conference 55th Yearbook 1991: Knowing the power of stories.Claremont, CA: Center for Developmental Studies of the Claremont Graduate School.
Shulman, L. (1998). Teacher portfolios: A theoretical activity. In N. Lyons (Ed.), With portfolio in hand (pp. 23-37). New York: Teachers College Press.
Vygotsky, L. S. (1981). The instrumental method in psychology. In J. V. Wertsch (Ed.), The concept of activity in Soviet psychology (pp. 134-143). Armonk, NY: Sharpe.
Wilcox, B. & Tomei, L. (1999). Professional portfolios for teachers: A guidee for learners, experts and scholars. Norwood, MA: Christopher-Gordon Publishers, Inc.
Wineburg, S. (1997). T.S. Eliot, collaboration, and the quandaries of assessment in a rapidly changing world. Phi Delta Kappan, 79(1), 59-65.