 |
 |
 |
Mathemagenic on personal productivity in knowledge-intensive environments, weblog research, knowledge management, PhD, serendipity and lack of work-life balance... |
|
|
|
|||||
|
|||||
|
|
Mathemagenic
|
 Methodology chapter: Matching quality criteria and verification strategies
[From draft version of methodology chapter for my dissertation, slightly adapted for the web] The table below presents an overview of verification strategies and their relevance to the quality criteria.
More on: methodology PhD chapters
|
03 October 2007|
Methodology chapter: Quality verification strategies
[From draft version of methodology chapter for my dissertation, slightly adapted for the web] In this section I propose several strategies to ensure quality of my research. I call them verification strategies, however similar to (Morse et al., 2002) I would like to emphasise that most of them should be used during research and not only for verifying quality of the outcomes. Exposure Prolonged exposure "in the field" ensures that a researcher had enough opportunities to encounter a variety of perspectives that would allow rich representation of the phenomenon under study. Yanow (2006) notes that prolonged exposure refers not only to the time, but to location as well. For me this means "being long enough in the right places talking to a variety of people to uncover important issues". While doing research this means taking an effort to "map the territory" (Yanow, 2006) in a way that allows representing a variety of perspectives. For example, in case of my research this means talking to bloggers with diverse practices, including those in minority. In the study at Microsoft I strive for diversity by complementing snowball interview sampling with finding people "outside the network" by searching for "deviating weblogs", e.g. those written in another language or used in "unconventional" way (see 'Those that belong to the Emperor' for an example). For KM blogger study I use social network analysis based on linking between weblogs to define communal boundaries next to my (more subjective) personal knowledge of the actors and setting. In reporting research exposure is reflected by describing the study settings, time and duration of being there; efforts made to define the field, to acquire representative data, to include multiple perspectives. Triangulation Triangulation refers to use of multiple sources and modes of evidence to make findings stronger by showing and agreement of independent measures or by exploring and explaining conflicting findings (Miles & Huberman, 1994; Schwartz-Shea, 2006). I employ several types of triangulation in my research:
Theorising Next to being a starting point or target for a research, theory could be an instrument to make it stronger. In my research I use theory:
Participants as co-researchers One of the strategies to ensure that research results represent the phenomena under study is informant feedback (Miles & Huberman, 1994; Schwartz-Shea, 2006), asking study participants to comment on the report. In my case I take it further, treating participants as co-researchers. This means not only asking for a feedback on finished reports, but also providing them opportunities to observe and to influence parts of the research process via my weblog. Reflexivity Reflexivity refers to the awareness and theorising about the role of self in all phases of the researcher process (Schwartz-Shea, 2006). I also like to think about it in terms of reflection-in-action and reflection-on-action (Schön, 1983). The first one is the source of what Yanow (2006) calls "improvisational character of interpretive research" - reflecting to address difficult to predict research circumstances (e.g. reacting to unanticipated turn in an interview). The second one refers to evaluation in retrospect, formulating the "lessons learnt" to guide actions that follow and to share in research report. In my research this strategy is used in several ways:
Transparency At its extreme, making one’s research transparent means conducting an audit, where a detailed record of research processes and decisions as documented by a researcher is examined by an independent researcher to access research quality (Halpern, 1983; Akkerman et al., 2007). I prefer a broader definition of transparency as a set of practices to document research for an inspection by others (Schwartz-Shea, 2006). In my research I do it in the following ways:
Thick description Thick description (Geertz, 1973) refers to the style of reporting the research results aimed to "transporting the reader to the field" by providing detail-rich description of life of the research participants (Klein & Myers, 1999; Brower et al., 2000; Yanow, 2006; Schwartz-Shea, 2006). In the case of my research this means providing extensive quoting from weblogs and interviews, describing history and context of a particular setting, portraying the complexity and interrelations between different aspects of blogging practices. When quoting from weblogs I preserve linking in the text and provide direct link to the post, so those who read my work digitally can literally "transport themselves to the field" with one click. Purposeful confessional writing I use the term confessional writing to address different forms of bringing personal experiences in a publication (using personal pronouns, talking about my background or beliefs, including personal examples, etc.). Taken to an extreme, this could turn a research report into autobiography (Schultze, 1999; Duncan, 2004). This strategy aims to avoid that risk by making sure that confessional writing serves a research-related purpose. I do so by:
More on: methodology PhD chapters
|
02 October 2007|
Methodology chapter: Quality criteria
[From draft version of methodology chapter for my dissertation, slightly adapted for the web] In my reading of methodological literature I often felt lost, so I was very happy to find the work of Peregrine Schwartz-Shea on quality evaluation criteria for interpretive research (Schwartz-Shea, 2006). In her discussion on quality criteria as suggested by different authors she not only discusses how multiple terms and categories are used across and within different research paradigms without making parallel terms explicit, but also draws some of the missing parallels. I used her table (Schwartz-Shea, 2006: 94) that matches terms used in classic interpretive research texts (Lincoln & Guba, 1985; Miles & Huberman, 1994) to positivist research to suggest terms that I would like to use for my own research: authenticity, trustworthiness and impact (column 5 is added by me).
I propose a simplified list of terms as a way to resolve the differences between terminology used in a variety of publications that I consulted. For example, a list of criteria suggested by Richardson (2000) to evaluate autoethnography provides an example of an alternative terminology that does not easily matches one of the classic texts, but addresses well specific issues for this type of research: 1. Substantive contribution: Does this piece contribute to our understanding of social life? Does the writer demonstrates a deeply grounded (if embedded) human world understanding and perspective? How has this perspective informed the construction of the text? In proposing my own simplified criteria I tried to integrate those from publications that discussed evaluation criteria and/or corresponding quality verification strategies in a way applicable to my work. I define proposed criteria in the previous table by describing what is judged by each of them and how this could be translated into specific questions to ask about the research.
Comparing my research approach to those done in more traditional ways I expect most challenges in defending its trustworthiness, since I report explicitly about my personal involvement and certain degrees of subjectivity in doing it. A good example of what I could expect is presented by Holt (2003), who analyses the comments to his autoethnographic paper by journal reviewers. He identifies two groups of issues related to acceptance of his work: the use of self as the only data source and the use of verification strategies in autoethnographic studies. The first is applicable fully to only one of my studies, while for the dissertation as a whole I use my own case to complement other cases and include autoethnographic elements to add transparency to the research process. I address the second concern, difficulty of using common verification strategies to judge this type of research, in the following section by describing quality verification strategies that fit my research. However, next to the efforts to ensure and to defend trustworthiness of this research, I am always prepared to defend those choices that help me to provide a better overview of my topic and to make sure the results make a difference, even if they make my work weaker in the eyes of some researchers. Fortunately, weblogs supply not only challenges of studying them, but also alternative ways to provide transparency of the research and accountability of the researcher. More on: methodology PhD chapters
|
01 October 2007|
Methodology chapter: Participation
[From draft version of methodology chapter for my dissertation, slightly adapted for the web] Although being a detached outsider is often considered to be the best position for a researcher, soon after starting my research I found myself at the opposite end, doing research through active participation (Mortensen, 2003). When I submitted my first paper on knowledge work and blogging (Efimova, 2004b which eventually became my background chapter) I realised that my insights came as much from analysing replies to the questionnaire as from my experiences of being part of the KM blogger community: writing my own weblog, interacting with other bloggers and reflecting on those experiences. I started to look for methodologies that would allow accounting for personal experiences of the researcher and soon found that ethnography addressed many of my concerns. (Yanow (2006) suggests that '"ethnography" refers both to a set of research tools and to a mode of writing'. I would position my research as "ethnographically informed" as I use some of conceptual distinctions and research instruments of ethnography, while only partially adopting ethnographic writing mode.) Ethnography, originated as a method in sociology and anthropology, is increasingly used in research of technology-mediated practices. It includes studying particular culture by learning to live a life of its members (Hammersley & Atkinson, 1994). Next to informal interviewing, participant observation is a central way to generate ethnographic data: Anthropologists are trained to use a research method known as 'participant observation', which is essentially means participating in the life and culture of the people one is studying, to gain a true insider's perspective on their customs and behaviour, while simultaneously observing them as a detached, objective scientist. Well, that's the theory. In practice it often feels rather like that children's game where you try to pat your head and rub your tummy at the same time. (Fox, 2005: 3) Although weblogs, as many other online tools, provide an opportunity to observe unobtrusively by lurking and reading, passive observation was not a choice for me, since my beginning of my PhD coincided with my first blogging experiences. In my research I played two roles: a knowledge worker who blogs about her work and a researcher who studies knowledge worker blogs. The following quote, illustrates one of my first attempts to describe the effects of combining those two roles:
Participating in the life and culture of my target group by blogging helped my research in a several ways as discussed below. Learning about blogging culture My personal blogging practices became an important source of learning about blogging, especially in respect to understanding the aspects of it that couldn’t be observed by reading weblogs. Those aspects include, for example, the effort that goes into fine-tuning a weblog tool to fit my needs, the surprises of receiving feedback on pieces that I never expected to be interesting to others, or the change of daily morning routines as a result of blogging. Next to substituting the state of the art literature study, personal blogging experiences helped me to follow weak signals of interesting issues that I might overlook otherwise. Reading weblogs, as "another blogger" and not only with the coding purposes in mind, was an important part of my personal blogging experience. It helped me to get to know people behind weblogs: "absorbing details of others' lives from their weblogs, sense of connectedness and somewhat intimate knowledge about them" turned into "interviews that could touch themes and go to the depths not possible otherwise" [quoting this post]. I also learnt a lot by comparing my own blogging practices to those of others in my own community and outside it. I has been frequently surprised that even bloggers that I considered to be very close to me in blogging attitudes would make different choices from my own. From another side, trying to describe to sceptics some practices shared in my blogging community helped to shape my research questions and to position my own work in respect to other researchers. Blogger identity, relations and access Being a blogger gave me an identity between other bloggers and helped to develop trusted relations with others. In my own blogging community I didn't need introductions and could easily contact others for information or an advice, by email, instant messaging or phone. When I travelled, I stayed in the houses of some of my blogging friends and was able to peak into their private lives and have casual conversations about blogging on topics that would likely to escape more formal interviewing. Having a weblog also served me when approaching study participants outside of my own network. Arranging for the study at Microsoft was mediated via my weblog. Also, when I emailed introductions for bloggers I didn't know and asked for interviewing opportunity, I would include link to my own weblog next to other credentials. It’s difficult to measure how much closed doors it have opened, but I also felt that it provided more equality as participants of my research could check my background as easily as I could check theirs (see Mortensen & Walker, 2002, for similar example; Beaulieu, 2004, for a methodological discussion of it). Others as co-researchers I blogged on the progress of my research and my participants could easily follow those posts, creating influences and feedback loops that researchers learn to avoid to escape "contaminating their data". I have learnt to embrace and use them in my research, taking an advice from Hammersley and Atkins: Once we abandon the idea that the social character of research can be standardized out or avoided by becoming a ‘fly on the wall’ or a ‘full participant’, the role of the researcher as active participant in the research process becomes clear. He or she is the research instrument par excellence. The fact the behaviours and attitudes are often not stable across contexts and that the researcher may influence the context becomes central to the analysis. (Hammersley & Atkinson, 1994: 19) I tried to vary the degree of closeness to my informants between and within studies. For example, as I was aware of the feedback loops in my study of KM bloggers, I had an extended email discussion with Jonathan Grudin, my supervisor at Microsoft, on possible choices in respect to blogging about the study progress while being in the company. When making choices for interviewees I made an effort to talk to people more distant from myself (for example, based on knowledge that they don't read my weblog). Many of bloggers who participated in my studies could be described as lead users, those who shape emerging technology to address their needs (von Hippel, 1986). Often their own professional interests aligned well with my research quest to discover how weblogs could support knowledge work. Blogging about progress of my research helped to involve them as co-researchers. I had multiple occasions of feedback from fellow bloggers on shaping study methods, data collection instruments, emergent interpretations or drafts of my papers. Colliding worlds As my blogging served me in both roles, a researcher and a blogger, it was not always easy to separate them and to make choices in case of role conflicts. This is an example of one of this cases as documented in my weblog (it refers to the weblog conversation study done in my own blogging community): [Extended quote from Hard choices: researcher vs. blogger?] However, role conflicts appeared also where I did not expect them. Studying blogging practices of people outside of my own circle brought me to making choices between insider participation and outsider distance as well:
[Extended quote from Studying weblogs at Microsoft: connecting the dots] In addition to the role conflicts, being a blogger made it difficult to draw a line between fieldwork and homework, participant observation and writing up, creating a risk of turning my research into an on-going endeavour (Beaulieu, 2004). In addressing this problem, publication deadlines served me well: as a deadline would approach I would not have much time to read other weblogs and to blog myself, thus creating natural withdrawal moment that would serve as a boundary between the field and home. Finally, the feedback loops resulting from blogging changed the way I would write up my research for a publication. Although sometimes study participants are expected to read final research report, they are not the intended audience for it. In my case thinking of the study participants as readers of the finished work was the choice I could not escape knowing how little effort it would take for them to access my published work. More on: methodology PhD chapters
|
© Copyright 2002-2007 Lilia Efimova 
This weblog is my learning diary. Sometimes I write about things related to my work, but the views expressed here are personal and do not necessarily reflect the views of my employer.
Last update: 15/11/2007; 01:21:07.
