In my dissertation I describe uses of weblogs as a sense-making instrument that provides a way to deal with unexpected or complex ideas by supporting articulation and organising ideas at a personal level combined with distributed collaborative thinking in “sense-making networks“. While exploring this theme as part of the content of my research was somewhat expected, I did not realise how much reflecting on practices of others and my blogging experiences in that respect would challenge my research methodology-wise.

Blogging research not only turned into participatory research and involving others as co-researchers. As I experienced the flexibility of organising my thinking with the weblog, as well as the fun and added value of the social nature of it, it became more and more difficult to use conventional data analysis methods and tools.

As it’s easy to attribute to my own personality or strange choices, I was glad to hear that I’m not alone in it. Stuart French on blogs as study tool (once you are there check University 2.0 too):

The real research is happening in NVivo, but I find that I do a lot of my thinking better in a blog than a analytic memo, so I started interchanging the two. If a memo was about a specific data or participant then in went into NVivo and was linked, but the more general thoughts about underlying cause/efect relationships, theories in the literature and in use by the participants to make sense of their environments…they end up in the blog.

More and more though, I find the challenge of writing for public consumption adds something of an edge to the process of analysis so many of the recent posts have been to this blog rather than my private one.

In my own case blogging came before I made a choice for specific data analysis methods or considered using tools for qualitative data analysis. When I tried some of them, I missed two things in comparison to blogging: flexibility of dealing with fuzzy data and emergent assumptions, and an easy way to involve of others in the process . While on the technology side including those capabilities in the research tools is probably just a matter of time, I believe that addressing them methodology-wise provides a bigger challenge.

My experiences of sense-making as a flexible, intuitive and messy process raise questions about finding a methodologically sound way to accommodate for those:

If expertise is difficult to articulate, how would you specify (for example) explicit coding criteria to pinpoint patterns? How far the need to make things explicit, to categorise beforehand would ruin the richness of what could be found? How far the decisions on what are the patterns could be logically explained? How easily the process itself could be articulated for an examination by others?

The social nature of sense-making with blogging indicates other challenges. While action research methodologies do provide a way to include others in research, the ways of doing so rely on having a shared goal and cycles of planning, action and reflection. In the case of blogging research involvement of others is unplanned, casual, and fragmented. It is those characteristics that make involvement of others especially valuable (re: exposure and unexpected connections across boundaries), they difficult to account for methodologically.

• On things that hide behind typical formats of reporting research
• Bloggers as public intellectuals and writing about them in a research report
• On attributing interviews done for my research: the dark side of transparency

Blogging PhD research and what happens next – presentation (an attempt of zen :) for the panel “New modes of scholarly communication: blogs, wikis, and web2.0 in academia” at Berlin 6 Open Access conference, November 11-13 2008, Dusseldorf, Germany.

[Video of the presentation]

In the talk I decided to talk about my experiences of blogging research to make it as relevant as possible to other researchers, so I focused primarily on connections between it and the process of growing ideas and turning them into a publication. It doesn’t give a good overview of blogging in respect to research methods and methodologies (some readable insight on it is here).

Links for more background on parts of the talk:

• Process of growing ideas: from fuzzy feelings to finished results
• Fuzzy ideas: awareness and articulation
  • Things that don’t fit
  • Blogging as creating space for important
  • Reasons for using weblog to keep information bits
• Sense-making
  • Mangrove effect: the value of making things explicit
  • PhD: experiential research and everyday grounded theory
  • Weblog and the mess of papers on my desk play similar roles in supporting my work
• Turning into an outcome
  • If most of the things I want to say in my PhD are already in my weblog, what’s the added value of the dissertation?
• Issues related to doing networked research
  • Blogging research: attribution and ownership of ideas
  • Bloggers as public intellectuals and writing about them in a research report
  • Paper: Blending blogging into an academic text (it discusses blogging from a methodological perspective; relevant in this case for the discussion about research quality issues and ethical challenges of studying bloggers)

I know it bugs for being turned into a readable paper. Will work on that after finishing the dissertation (soon, submitting first draft as a whole in three weeks!), but any comments on where it makes sense to publish is are very welcome.

• Time in blogging: catching a moment to write
• Pull vs. push strategies for sharing knowledge
• Time to reflect: my uses of weblog

My recent silence (and being stuck with the last study for my dissertation as the main reason for it) is a results of an attempt to make my research more transparent and inclusive by doing it “in public”.

My original intentions are outlined in the study description I used to invite bloggers to participate,  Networking practices of KM bloggers:

In general I prefer using real names of the participants and links to their weblogs to give credits similar to how it’s done in the blogging world. [...]

I would like to post summary of the interview on my weblog (after all interviews are done). I will email it to you before posting, so you can correct anything wrong or decide if it should not be published.

There is a chance that I will blog about work-in-progress while analysing the interview data or working on a publication. In this case I will only quote from publicly available sources (e.g. from your weblog or summary of the interview after you give permission to publish it online).

The main motivation behind this approach was to give credits to the participants and create a possibility of a dialogue around their contributions. I also didn’t expect that the things I wanted to ask would be extremely sensitive, so thought that bloggers wouldn’t mind (or even would appreciate) sharing them in public (in fact, a couple of people I interviewed said that I could just post interview summaries without checking with them first). An additional motivation for doing so was “methodological”: adding transparency to the research process as a way to improve the research quality.

Now what’s happened:

I’ve got stuck with writing interview summaries as those had to satisfy both, putting them online and analysing them for my research. For the analysis the ideal way would be to have “extended summaries”, those with as many direct transcripts of actual interviews as possible, however those would be too long and too fragmented to post in public. I could also make shorter summaries to post online, but this would limit what I could use while discussing the results since I promised to “only quote from publicly available sources”.

So I thought of analysing the data, deciding what had to be quoted and revising the summaries accordingly. But then I’ve got stuck with the analysis. For me discussing emergent interpretation with others is the best way to get “unstuck” and for this study doing that in the weblog would be really the best option. However, I couldn’t blog about anything untill posting the interview summaries online. And I couldn’t write those summaries either…

I eventually got “unstuck” with finding a way to discuss the interviews before making public summaries of them. I made anonymous summaries and used them to have a discussion on emergent themes with two colleagues who are far from blogging. With that input I could get a better picture of how to discuss the study results and which parts of the interviews are really important to include. I’m currently making blog-friendly versions of the anonymous summaries, so I can finally email those to the participants to be checked, post them online and get into blogging the results.

In the process I also figured out a few other issues with making the research data publicly available and attributed to the participants: it made more challenging including background data on the participants (e.g. age) or discussing “difficult” issues around their practices. So, my idealistic attempt for an “extreme” transparency didn’t really work – I guess I’ll be more moderate next times :)

• Methodology chapter: blogging practices
• DIRN workshop
• Research methodology: everything is relative

Just finished and submitted :)

Efimova, L. (2008). Blending blogging into an academic text. Paper submitted IN THE GAME: Ethnographic relationships, mediation and knowledge, workshop at Internet Research 9.0, Copenhagen, Denmark, 15-18 October 2008.

Abstract. For my research blogging has been a blessing and a curse. While it has turned into a set of research practices that brought rich results, it also resulted in a search for methodologically sound ways to justify those practices, put me into a struggle of being a researcher and a blogger at the same time, and challenged everything I knew about academic writing. As I work on the chapters of my PhD dissertation, blending blogging into an academic text to bringing together the blogger and the researcher in me, this paper provides an opportunity to reflect on this process. I start from introducing my research and the roles blogging played in it, and then discuss bringing my own weblog in the dissertation through autoethnography and confessional writing as well as the challenges of representing other bloggers in the text of it.

Looking forward to the workshop – the list of the workshop participants and their papers is intriguing…

• On the role of theory
• Weblog-mediated relationship: a co-constructed narrative
• Soul searching

Working on a paper on how I bring blogging in the text of my dissertation, I finally get to write a bit more on When They Read What We Write: The Politics of Ethnography promised long time ago. Although the book is well worth reading as a whole for anyone interested in relations between a researcher and those participating in the research, one of the papers is a must read for those studying bloggers:

• Sheehan, Elizabeth A. (1993). The student of culture and the ethnography of Irish intellectuals. In C.B.Brettell (Ed.), When they read what we write: the politics of ethnography (pp. 75-89). Westport, CT: Bergin & Garvey.

In the paper the author tells about the challenges of representing in a research report academics she studied: public intellectuals, “who earn their living in large part through their ideas” (p. 81).

It is a cliché to say that knowledge is power, but in the case of informants who are intellectuals, knowledge is also capital, symbolic and otherwise. Here too the boundaries between public and private forms of information become confused, merge, and cross over to opposite sides in the exchange between anthropologist and informant. As a results, ethnographic writing about academics and intellectuals raises serious issues of intellectual attribution. [...] As intellectuals, many academics create their lives through their work, and their work through their lives. Interviews with such information can provide exhilarating insight for the ethnographer (Yes! Yes! This is what I mean!), brought to a sudden halt by the realization that the ideas you are now thinking – and thinking of writing about – are not entirely your own at all but the product of mutual intellectual exchange. How to you correctly ascribe ideas that are offered within the context of an interview but which may also be the basis of new works, new publications? How do you separate the public thinker from the private, honour his confidentiality and intellectual property, and still offer a meaningful analysis? (Sheehan,1993, p.81)

This one has direct connections to my early questions on weblog research ethics in respect to he choices between protecting privacy of the participants and recognising their authorship. Browsing through the referrals to my post on attribution and ownership of ideas when blogging research I came across a nice summary of the issue from a research participant side in a post by Frank Carver (bold is mine):

One of my current concerns is the tension between perceived needs one the one hand for attribution, academic traceability and ownership of ones own words; and on the other hand for privacy. This is seen in sharpest relief in solicitations for academic surveys. Routinely such instruments come with a disclaimer pointing out that all answers will be anonymous. Well-structured surveys and questionnaires, though, often also contain a section for general comments and feedback. In most cases I do not want this to be anonymous – indeed I would rather it formed part of a dialogue between the researcher and subjects, allowing both to benefit, learn and develop.

I am considering taking up a habit of always adding my contact details to academic survey submissions to deliberately challenge the assumption that I wish to be an anonymous donor of information, and to encourage researchers to participate in a community of interest.

The stress on mutual benefits is important: often it’s not only the researcher who learns new things, but also people who participate in the research, when their thinking on a subject is triggered as a result of an interaction. Elizabeth Sheehan gives a nice example that the challenges of attributing the ideas in a case like this one may also exist on the participant’s side:

I might add that this process can work both ways, but with less ethical difficulty for the informant. I was both flattered and dismayed to see some insights of mine appear in the Irish Times, unattributed, under the byline of an academic I had interviewed a few days earlier. He had no need, as had I, to sort out his ides from my own in a setting which was, for him, just and interesting discussion with another academic. (Sheehan,1993, p.81)

Another issue that the paper touches is the one I had to deal myself: the need to represent research participants in a way that multiple parts of their input could not be attributed to the same person (in When they read what we write: respondent identification). An example from the paper:

…his identity had to be fragmented. In the dissertation he becomes several people, not my the questionable device of pretending he was really a number of different individuals, but simply by my failing to inform the reader that “one professor,” “another commentator,” and so forth who appear throughout the dissertation are actually one person. Consequently, this single individual is discessed as the unnamber center of the appointment controversy, as an anonymous example of the links between scholarship and party politics, as an attributed commentator on his research discipline, and as a published sources on his research specialty. (Sheehan,1993, pp.83-84)

• On research again…
• Studying weblogs at Microsoft: connecting the dots
• On blogging and book writing

Some quotes that are not likely to be included in the Methodology chapter of my PhD, but pretty much explain how I think about methodological choices:

…the validity of scientific claims is always relative to the paradigm within which they are judged; they are never simply a reflection of some independent domain of reality (Hammersley & Atkinson, 1994, p. 12)

…methods rest on philosophical presuppositions. These remain embedded in them, even if they are not taught or discussed or attended to explicitly. (Yanow & Schwartz-Shea, 2006, p. 370)

No context is value-free. Academic disciplines promote particular ways of observing, dissecting, measuring, interpreting, and otherwise making sense of the phenomena under investigation. One’s decisions may emerge within or resistant to these disciplinary structures. One’s decisions also derive from one’s research goals, which are seldom acknowledged in research reports but which meaningfully affect the design, process, and outcome of a study. (Markham, 2006)

…all research is a practical activity requiring the exercise of judgement in context; it is not a matter of simply following methodological rules (Hammersley & Atkinson, 1994, p. 23)

References

• Hammersley, M. & Atkinson, P. (1994). Ethnography: Principles in Practice. (2nd ed.) Routledge.
• Markham, A. N. (2006). Ethics as method, methods as ethics: A case for reflexivity in qualitative ICT research. Journal of Information Ethics, 15(2), 37-54.
• Yanow, D. & Schwartz-Shea, P. (2006). Interpretation and Method: Empirical Research Methods and the Interpretive Turn. M.E. Sharpe.

• Bloggers as public intellectuals and writing about them in a research report
• On research again…
• On the role of theory

Referrer logs bring me to the post on high-stakes reflection (mirrors, maps and masks) by Jen:

One of the things I found really fascinating in the e-portfolio literature was Barrett and Carney’s idea of ‘conflicting’ or ‘competing’ paradigms: ‘positivist’ (product-driven, performative, externally assessed, based on externally defined outcomes), vs ‘constructivist’ (process-driven, reflective, learner constructed outcomes) (2005, p7-8). These are also sometimes described as ‘map’ and ‘mirror’ portfolios. [...]

Then I became interested in the extent to which the tension between these ‘conflicting’ paradigms might in fact be an intrinsic part of professional reflective practices. [...]

To describe this, along with ‘map’ and ‘mirror’, I have added a third category: portfolio as ‘mask’. I’ve been working on this metaphor a bit over the past few months and am delighted by its richness – so far I’ve identified at least 6 (overlapping) genres of mask: protection, disguise, performance, memory, transformation, punishment.

This post, together with the one detailing the six mask genres, provides metaphors to think on some of the comments I’ve got on the PhD chapter that looks at blogging PhD ideas. Part of the struggle I had while working on it was drawing the boundaries between the different perspectives I use to look at blogging ideas, (knowledge base / process / context). Although the metaphors do not easily fit onto what I have written (they are also more appropriate for someone looking at blogging from the outside), but they do provide an input for reflecting on it.

The mask metaphor (read the post on six genres) is an interesting one to look at the blogging in the context of my PhD research. Here a quick look on the genres in respect to my weblog research-wise (reordered):

• Memory (trace in the second post) – literally, to keep traces of my thinking.
• Performance / disguise – presenting myself through writing, intentionally and not.
• Punishment – being shaped by the mask, the traces I leave via blogging and the image that others construct of me.
• Transformation – what happens with the ideas as they have been blogged and with my own identity as I go through the process (re: Kamler&Thomson, 2005).
• Protection – the choices I made in bringing blogging back into the dissertation as an instrument to address methodological challenges (a bit here, but more in the paper I’m supposed to write instead of this post). [Update: finished paper - Blending blogging into an academic text]

• ‘Those that belong to the Emperor’ (on weblog types)
• Am I killing publication opportunities with blogging PhD results?
• How blogging makes my life difficult

[This post was in drafts while I was moving weblogs]

Here comes another turn on my researcher vs. blogger dilemma. I’m working on a study that aims to describe how my own blogging practices contributed to the development of ideas for my dissertation (I’m extremely happy to be able to do exactly what I envisioned long time ago) and I use my weblog as a data source to do so. A quote from my current draft:

As a starting point for reconstructing blogging practices I use my weblog artefacts (weblog entries, link, tags, etc.). In particular, I look at the content of meta-blogging entries and at the patterns of weblog uses.

Meta-blogging entries provide an unstructured documentation of my experiences of using the weblog to develop ideas. I use this in-situ view of my blogging practices as a starting point for the analysis and restructuring aimed to produce a systematic description of them. For an additional insight and an illustration of practices I look at patterns of weblog uses, especially focusing on categorisation of ideas, their development over time and their transformation that resulted in this dissertation.

To support the analysis of weblog entries as well as the access to them for the readers of this work, they are categorised using emergent and retrospective codes. Emergent codes are tags that I used for topical organisation of my weblog entries when I wrote them. Retrospective codes are added for specific purposes at the moment of doing this study (March-April 2008); those include references to the specific chapters of the dissertation and marking all meta-blogging entries as such. All coding is done using the functionality of my weblog software, so the pages that aggregate the results on specific codes are visible in public and could be used as a reference throughout the dissertation.

So, now the practical problem. As a researcher I want the codes to be “frozen” at the moment of doing the study, as a blogger I want to be able to edit and change them, since the “codes” are tags and categories that provide views on my weblog. The good thing is that I’m doing the study at the moment of moving from Radio to WordPress, so I can just “freeze” Radio archives as they are. The bad thing is that I can’t update Radio public pages anymore, so they include only emergent codes (liveTopics tags that I used so far), while it could make sense to see them together with the retrospective codes I added specifically for this study (categories “Chapter 1-7″ and “Meta-blogging”).

My way out:

• Researcher’s hat on:
  • I use the archived version of Radio weblog as a snapshot of what was before the analysis. All posts are there with corresponding emergent codes – liveTopics tags (they are called “topics” – confusing, is not it?). The index of those tags is at http://blog.mathemagenic.com/allTopics.html
• Blogger’s hat on:
  • I use the WordPress version of the weblog as a living blog, which means editing categories and tags in a way that makes sense for me as a blogger.
• Researcher’s hat on:
  • For those other researchers who might want to have all codes frozen in one place I’ll put an RSS archive of my weblog with all codes used for this study online. It’s not going to be easy to access via browser, but at least it will be somewhere in public.

As my new weblog should provide an easier way for browsing I use links to it as a reference in the dissertation, while making sure that all posts that have an “archived” Radio version are linked from their new copies.

• Good research…
• When they read what we write: respondent identification
• Notes on my PhD methodology: weblog studies

[From draft version of methodology chapter for my dissertation, slightly adapted for the web]

The table below presents an overview of verification strategies and their relevance to the quality criteria.

Archived version of this entry is available at http://blog.mathemagenic.com/2007/10/25.html#a1950; comments are here.

• OneNote for recording and coding qualitative data (+ more on tagging metaphor for flags)
• Methodology chapter: Participation
• Weblog as a research notebook (3): my own experiences

[From draft version of methodology chapter for my dissertation, slightly adapted for the web]

In this section I propose several strategies to ensure quality of my research. I call them verification strategies, however similar to (Morse et al., 2002) I would like to emphasise that most of them should be used during research and not only for verifying quality of the outcomes.

Exposure

Prolonged exposure “in the field” ensures that a researcher had enough opportunities to encounter a variety of perspectives that would allow rich representation of the phenomenon under study. Yanow (2006) notes that prolonged exposure refers not only to the time, but to location as well. For me this means “being long enough in the right places talking to a variety of people to uncover important issues”.

While doing research this means taking an effort to “map the territory” (Yanow, 2006) in a way that allows representing a variety of perspectives. For example, in case of my research this means talking to bloggers with diverse practices, including those in minority. In the study at Microsoft I strive for diversity by complementing snowball interview sampling with finding people “outside the network” by searching for “deviating weblogs”, e.g. those written in another language or used in “unconventional” way (see ‘Those that belong to the Emperor’ for an example). For KM blogger study I use social network analysis based on linking between weblogs to define communal boundaries next to my (more subjective) personal knowledge of the actors and setting.

In reporting research exposure is reflected by describing the study settings, time and duration of being there; efforts made to define the field, to acquire representative data, to include multiple perspectives.

Triangulation

Triangulation refers to use of multiple sources and modes of evidence to make findings stronger by showing and agreement of independent measures or by exploring and explaining conflicting findings (Miles & Huberman, 1994; Schwartz-Shea, 2006). I employ several types of triangulation in my research:

• Triangulating by study – studying blogging practices from three perspectives using a variety of methods.
• Data triangulation – including in the analysis different types of data (e.g. text and statistics), data sources and data collection methods. In my research that means, for example, including non-elicited data (Pargman, 2000) from public sources (e.g. weblog text) next to the recorded interviews.
• Involvement of multiple researchers in data collection and analysis in all studies except one.

Theorising

Next to being a starting point or target for a research, theory could be an instrument to make it stronger. In my research I use theory:

• as a “sensitising device” (Klein & Myers, 1999) to inform research questions and conceptual categories, to tease out implicit nuances;
• to explain and to position findings;
• to “normalise the atypical” (Brower et al., 2000) by drawing parallels between my cases and conditions more familiar to the readers.

Participants as co-researchers

One of the strategies to ensure that research results represent the phenomena under study is informant feedback (Miles & Huberman, 1994; Schwartz-Shea, 2006), asking study participants to comment on the report. In my case I take it further, treating participants as co-researchers. This means not only asking for a feedback on finished reports, but also providing them opportunities to observe and to influence parts of the research process via my weblog.

Reflexivity

Reflexivity refers to the awareness and theorising about the role of self in all phases of the researcher process (Schwartz-Shea, 2006). I also like to think about it in terms of reflection-in-action and reflection-on-action (Schön, 1983). The first one is the source of what Yanow (2006) calls “improvisational character of interpretive research” – reflecting to address difficult to predict research circumstances (e.g. reacting to unanticipated turn in an interview). The second one refers to evaluation in retrospect, formulating the “lessons learnt” to guide actions that follow and to share in research report. In my research this strategy is used in several ways:

• My weblog serves as a reflexive journal (Lincoln & Guba, 1985), which documents many of my reflexive moments, doubting and thinking aloud while doing research or evaluating it in retrospect. I also reference and quote those entries where relevant in the text.
• I try to convey uncertainties, choices and lessons learnt while doing research in reporting the results by choosing not to present the research as clear path, but instead incorporating stories about reflexive moments in the text. I also discuss my own role, influences and mistakes while doing research. I write (in my academic publications, but also in my weblog) to think with the readers, not for them, as I believe that “the skill of writing is to create a context in which other people can think” (Edwin Schlossberg).

Transparency

At its extreme, making one’s research transparent means conducting an audit, where a detailed record of research processes and decisions as documented by a researcher is examined by an independent researcher to access research quality (Halpern, 1983; Akkerman et al., 2007). I prefer a broader definition of transparency as a set of practices to document research for an inspection by others (Schwartz-Shea, 2006). In my research I do it in the following ways:

• I use my weblog not only to document my research, but also to carry out part of the research process. In this way it is available not only for auditing in retrospect, but also for a real-time feedback, which is more useful for a correction.
• Since some of the data I use is publicly accessible, I provide readers of my research reports with references to it, so they have an opportunity to check my interpretations by examining it for themselves to the public data.
• I provide descriptions of research processes and decisions made in research reports.

Thick description

Thick description (Geertz, 1973) refers to the style of reporting the research results aimed to “transporting the reader to the field” by providing detail-rich description of life of the research participants (Klein & Myers, 1999; Brower et al., 2000; Yanow, 2006; Schwartz-Shea, 2006). In the case of my research this means providing extensive quoting from weblogs and interviews, describing history and context of a particular setting, portraying the complexity and interrelations between different aspects of blogging practices. When quoting from weblogs I preserve linking in the text and provide direct link to the post, so those who read my work digitally can literally “transport themselves to the field” with one click.

Purposeful confessional writing

I use the term confessional writing to address different forms of bringing personal experiences in a publication (using personal pronouns, talking about my background or beliefs, including personal examples, etc.). Taken to an extreme, this could turn a research report into autobiography (Schultze, 1999; Duncan, 2004). This strategy aims to avoid that risk by making sure that confessional writing serves a research-related purpose. I do so by:

• articulating the purpose of including confessional material (e.g. as an additional data source, to provide transparency of the research process, as a way to engage the readers);
• drawing parallels between my personal experience and those of other bloggers (in my studies or as reported in a literature);
• interlacing self-revealing writing with more traditional forms of academic writing (e.g. as a layered account Ronai, 1995);
• separating between personal data available for others to examine (e.g. my weblog posts) and personal interpretations of it.

References

Archived version of this entry is available at http://blog.mathemagenic.com/2007/10/03.html#a1949; comments are here.

• Kate Fox on participant observation and hidden rules of English behaviour
• On the role of theory
• Paper: Blending blogging into an academic text