not all literature ‘reviews’ are the same

I was trying to explain to a doctoral researcher the other day that the literature work that you do at the beginning of the doctorate is not the same as the literature work for the actual, final thesis that is handed up. I was doing a Not Very Good Job of this explanation when I remembered the book Doing your literature review. Traditional and systematic techniques, by Jesson, Matheson and Lacey (Sage 2011).

Jesson and her colleagues suggest that there are two basic types of literature review – they use the term review so I will too – the traditional and the systematic. I’ve adapted what they have to say in the following account and the argument I’m making is mine, not theirs. It’s the point I was trying unsuccessfully to make with the doctoral researcher.

The systematic review is what is also sometimes called an evidence-based review. It applies measures of the ’quality’ of research as a way of filtering and evaluating what texts are included and excluded. According to Jesson, Matheson and Lacey, the systematic review’s hierarchy of research usually has randomized controlled trials and meta-analysis at the top, followed by other quantitative studies such as cohort studies and surveys. Then follows various forms of qualitative research; these are generally, but not always, omitted from systematic reviews. The decision about what comes on top of the quality hierarchy is of course highly contentious, and there are lots of published debates about it. I don’t intend to canvass those here, just to understand this as a type of review.

The traditional review by contrast is usually based on a critical assessment of a personal selection of material and has different purposes. Jesson and colleagues offer five different variants of the traditional review. None of these are completely separate, in fact they overlap quite considerably. Nevertheless, there are some discernable differences in purpose and process.

The five types of traditional literature review are :

(1) a conceptual review. This synthesises and critically assesses literature to see the way in which a particular issue is understood. The conceptual review might also examine how the issue is researched, how those understandings are produced. The purpose of the conceptual review is to produce a greater understanding of the issue.

(2) a state-of the-art review. This examines the most recent contributions to a field or area of study in the light of its history of research. It particularly looks for trends, agreements, and debates. This is the kind of review that editors of journals write at periodic intervals in order to position their journal and its future directions.

(3) an expert review. Rather like the state-of-the-art review, but undertaken by a senior figure in the field and heavily inflected with their own particular interests and contributions. This is the kind of review that presidents of learned societies give to the assembled masses at a conference.

Now the next variation is the one that is undertaken at the start of the doctorate or in a research bid in order to position the new research project:
(4) the scoping review. This review sets out to create an agenda for future research. It documents what is already known about a topic, and then focuses on the gaps, niches, disputes, blank and blind spots. It delineates key concepts, questions and theories in order to refine the research question(s) and justify an approach to be taken.

The final variation is the literature review as it appears in the final humanities/social science thesis or in a journal article or book, after the research has been completed.
(5) the traditional review. This is somewhat like a scoping review, but its argument is not to create the space for a research project. It is to position a piece of research that has already been undertaken. In essence the reader gets what’s-already-known, plus the newly conducted piece – this research as the contribution. The literature is used to locate the contribution, the what-we-now-know-that-we-didn’t-before-and-why-this-is-important. Some texts and themes that were in the initial scoping review are omitted, and other things are now emphasized in order to make clear the connections and continuities, similarities and differences of the new research to what’s gone before.

So, (4) scoping and (5) traditional just don’t do the same job. They are different in purposes. One justifies the research to be done, the other locates the contribution in the field of completed research.

The obvious implication of this difference is that you can’t just dump the literature work you did for the proposal in the thesis text. You have to do more work on the literature when you are writing the thesis. It has to be modified in some way from what you did at the outset. Not just because you’ve read more since you started, but because the argument is somewhat different.

Well you can, and some do, just cut and paste the proposal literatures into the final dissertation text – and there’s nothing wrong with using the proposal as a kind of ‘holding text’ while you’re working on the results. However, there are consequences for not changing it before submission. The lack of awareness of the difference between beginning=scoping and end=traditional literature work is often why examiners ask people to write more/write again.

And we don’t want that – neither doctoral researcher, nor their supervisor!

Jill K Jesson with Lydia Matheson and Fiona M Lacey (2011) Doing your literature review. Traditional and systematic techniques. Los Angeles: Sage (pp 76, 117)

seven reasons why journals reject papers

I’ve written about rejections several times, and most of this is scattered throughout the blog, so I thought it might be helpful to amalgamate the most important points together. All in one place.

There are some very common reasons why journal papers get rejected:

(1) They are overcrowded with ideas. They lack focus. Most journal papers have one point to make, they work with one idea, one angle.

(2) They don’t reassure the reader that the research is trustworthy, in other words, that it has been thorough and that it fits within a recognizable tradition of work. Different disciplines require different levels of detail about how the research was conducted, with whom or what, where, how often, how many … The vast majority of journals require something that is methodological and/or about methods.

(3) They don’t fit the journal. It’s very important to check out the specific journal for which you re writing and tailor the paper to fit it. Journals can be thought of as conversations, and each paper as an entry into an ongoing conversation about a particular topic. That’s why it’s important to always see what other papers there have been on the same topic in the journal you are aiming for. If there’s nothing, there may well be a reason, namely, the journal isn’t interested in the topic. It’s also important to check out the way in which other authors in the journal write their abstract, headings, introductions and conclusions because that’s what the referees and readers will be expecting.

(4) There’s no sense that the paper is adding anything new. The writer hasn’t been able to summarise what’s already known about the topic, and what this paper adds. They might just report a piece of research without being able to say why it’s important, and why people need to know about it or what should happen now that they do. In other words, there’s no So What and no Now What.

(5) The writing sounds inexperienced. This usually means that the paper is front loaded with too much literature and lacks a strong conclusion that deals with the So What, Now What questions. But it can be because there is too much time spent on method, or the paper is weighted too heavily to results, or there isn’t enough grounding for the study, or enough analysis.

(6) The paper is poorly structured. There isn’t enough signposting to help the writer find their way through the argument. The headings are meaningless or there’s not enough of them, or there’s too many. The argument doesn’t flow. The order of chunks in the paper doesn’t follow.

(7) It’s just too local, too small, too insignificant. Not every piece of research can become a paper, although most can. However, sometimes people slice the research too thin, don’t do enough analysis, don’t make enough connections with other research, or are just too theory-light for the reviewers to judge the piece worthy of publication.

It’s possible NOT to make these basic mistakes. Making sure that you avoid these things leaves the referees able to engage with the actual ideas and the argument, which is after all, why you are writing…

(Is this where I put a shameless plug for our journal writing book? Gulp.)

early onset satisfaction – a bad thing for writing and writers

(health warning – this post is a tiny rant)

early onset satisfaction – this is a notion that I once heard Mem Fox talking about. She put EOS as the enemy of all writers. Feeling too happy with a piece of writing meant that you didn’t rewrite and rewrite as often and as hard as you ought to.

I had cause to think about early onset satisfaction again last week as I was reviewing an article which had clearly been sent in to the journal far too soon.

It was one of those articles about a single aspect of a very small study about something not very important… and I confess I wondered if the paper was so slight because the writer was trying to get more than one paper out of it. There was probably one passable paper in the whole study I suspected, but certainly not enough in the one I had to review.

I doubted very much if the author had given the paper to anyone else to read. And if they had, I’d have taken bets that they hadn’t asked anyone to ask them the hard questions – like – So what, and Why should I care?

When I remembered Mem’s phrase, I wondered whether perhaps that was what had happened. The writer had been struck down with EOS. They were so pleased to have written the thing, so darn satisfied, that they’d just sent it off. (No, my referee comments weren’t cruel. Yes, I didn’t feel nice, but I was as nice as I could manage.)

Now Mem is not the only one to suggest that it is dangerous for writers to be satisfied too soon with their work. I recently noticed that Rose Tremain had expressed a similar sentiment in her 10 rules for writers. She offers Never be satisfied with a first draft. In fact, never be satisfied with your own stuff at all, until you’re certain it’s as good as your finite powers can enable it to be.

Will Self says You know that sickening feeling of inadequacy and over-exposure you feel when you look upon your own empurpled prose? Relax into the awareness that this ghastly sensation will never, ever leave you, no matter how successful and publicly lauded you become. It is intrinsic to the real business of writing and should be cherished.

Zadie Smith says Leave a decent space of time between writing something and editing it.

Roddy Doyle advises Do be kind to yourself. Fill pages as quickly as possible; double space, or write on every second line. Regard every new page as a small triumph ­– until you get to Page 50. Then calm down, and start worrying about the quality. Do feel anxiety – it’s the job.

We get the picture. Real writers don’t get too satisfied too soon. They don’t suffer from EOS.

These same writer’s maxims are also true for academic writing. Even though academics are in a publication churn, it is still really helpful not to rush papers off as soon as the printer ink is dry. It’s good to let a paper sit and percolate on its own for a while, after you’ve had the first couple of goes at it. Have a break, come back to it later and see how it stacks up.

It’s also really useful to pass a paper around among some trusted friends to get some feedback. Ask the readers not to be nice, but to be helpful. You don’t have to take their advice, and they might just come up with things that are really important. Tell them to ask you the So what question.

No matter how tempting it is to just send the paper off, it’s MUCH better to take that extra time because it may well be the difference between rejections and small revisions. Premature publication is risky and bad for the mental health. Let “cant get no satisfaction” be the mantra.

should you, could you, would you… co-write with your supervisor?

It’s not hard to find a horror story or two about the PhD researcher who wrote something with their supervisor only to find when it was published that they weren’t given credit for the work. The supervisor put their name first, they made false claims about how much of the research was theirs. The worst case scenario is of course that the doctoral work was just taken and published under the supervisor’s name.

Co-writing is common in lab teams. Most science researchers working in research groups expect that they will co-write with others. Indeed, it also seems to be mainly lab teams that produce the horror stories of plagiarism and not giving due credit. Well, there is just much more co-writing going on, and some of it is pretty high-stakes, we mustn’t assume any in-built disciplinary evil. Co-writing is less common in the humanities and social sciences. It is, however, becoming more usual – not only because of the PhD by publication, but also because supervisors are taking up co-writing as part of their supervision practice.

I’m sure that the horror stories are true and I’m sure that there are some unscrupulous people out there. But the majority of supervisors aren’t out to rip off the doctoral researchers they work with. So, if we don’t take corruption and unethical behaviour as the starting point, then what IS the answer to the question of whether doctoral researchers should co-write with their supervisors?

The usual reasons against co-writing – I found these by doing the inevitable bit of google searching – were:
(1) the doctoral researcher is the expert in the research not the supervisor and they should therefore just write by themselves
(2) the supervisor is getting a free publication on the doctoral researcher’s back
(3) hiring committees look askance at articles co written with supervisors and discount them.

There’s actually not a lot of knowledge about the last point. When I found mention of this, I googled around a bit more to see if there was any discussion about hiring committees and I did find a tiny bit. One suggestion was that if the hiring committee saw a co-written article, then they assumed that this was because the supervisor thought highly enough of the doctoral researcher and their work to want to write with them. Another comment was that if there was ONLY one co-written article then this could be seen as an indicator of poor productivity. I don’t know the answer to this, and it’d be interesting to do a bit of systematic re-searching to find out.

BUT, what if we reverse the question. Why should supervisors co-write with the researchers they work with?

The answer now seems to be either that supervisors are in it for a free publication – that’s the mean and nasty version – or that they co-write in order to help doctoral researchers learn the genre of the journal article and to help them get the work out there as soon as possible. If the latter helping is more generally true than the former self-interest, then we can see that while the doctoral researcher does absolutely contribute their research expertise, the supervisor also contributes by way of knowledge about academic writing.

So to go back to the original question then, why co –write with supervisors?

My answer is that doctoral researchers might want to co-write with their supervisors in order to learn the publication game. There’s also some safety in writing with a more experienced writing partner. The doctoral researcher ought to be able to rely on their supervisor to avoid the most obvious writing ‘mistakes – no focus, lack of signposting, too much literature, not enough methods, no conclusion, too much jargon. They also might learn how to get the right authoritative stance.

Of course there is always the vexed question of author order, but that’s a topic for another post.

For now, what do you think about co-writing? If a doctoral researcher, do you write with your supervisor? Would you? Or would you avoid it like the plague? If a supervisor, do you do this, why and how?

self packaging – when is enough already?

I’ve realized recently that I’m pretty half hearted about the idea of self packaging/self promotion.

It’s not that I don’t do it. I do.

Well you have to now in HE. CVs and bio notes and university home pages are the bottom line for all of us. But there are of course multiple additional platforms across which you can re/present an academic self – facebook, tumblr, twitter, Wikipedia, pinterest etc. A new possibility nearly every month.

I do seem to rather relentlessly plug blog posts on twitter, probably much too much. On the other hand a friend who keeps an eye on such things was recently surprised to hear that my book with Barbara Kamler on publishing in academic journals had been out for some months ( read a review here). It had clearly passed her by and I obviously did a pretty crap job of promoting it. If your friends don’t know when you’ve published something, you’re in trouble! (Ive tried to compensate with these links!)

I confess that there are aspects of this self-packaging stuff which make me uncomfortable. It all seems a bit blowing your own trumpet very loudly and often… a lot like spin… more like self advertising for the sake of it…

But hang on… How culturally, generationally, and gender loaded is my reticence, I ask myself.
– generationally – I’ve not had to sell my soul or my self to get a job
- culturally – Australians, despite the reputation for being brash, tend to look disparagingly on self promoting tall poppies and England appears to be even more so
- gender – well I’m pretty ancient, and it was a case of girls being brought up not to put themselves forward. ‘Clever girls’ in particular were always regarded with some suspicion.
So there is probably a good sociological explanation for why I feel the way that I do about the self packaging game.

All well and good. Aside from considerations about the source of my discomfort about self promotion, I do also wonder about where you draw the line. On what basis do you decide something is OK, and something is too much?

I couldn’t name anything after myself. And I ‘m pretty sure I couldn’t write a Wikipedia entry about myself, although I know that some colleagues who have done both of these things. I might get round to doing a Wikipedia entry on something that I’ve researched – perhaps. Same for facebook – I could just about manage a research focused facebook page I think, but really I use FB just to connect with friends I don’t see in person very often.

However, I do get pretty miffed when I discover a book/Wikipedia/blog about something that I’ve been heavily involved with that doesn’t mention me – although I don’t make the effort to provide a correction. So there’s a contradiction for you. I want the credit, but I don’t want to have to do it myself.

Clearly it’s almost impossible to separate out the scholar from the scholarly work, and that’s what makes this whole topic of self packaging so interesting – and fraught. Promote the work and you promote the scholar – but is it always true vice-versa? Are there academic star cultures/academic cults of personality – and how does self packaging feed into them and are they a Bad Thing?

I’m not at all sure where I stand on all this and where to draw any new lines about what to do and what not to do… What do you think?

oh no, someone did the research before me…

I very frequently meet doctoral researchers who are worried about not being original enough. They are afraid that what they are doing has been ‘done before’ and they won’t therefore be making an ‘original contribution’ to knowledge. They also are terrified that just as they are finishing, they will find the definitive article which says everything they have to say, perhaps better and more convincingly.

Now in the arts and social sciences this really isn’t a big problem. The doctorate is not seen as the place for enormous breakthroughs. It is more a case that the PhD is intended to demonstrate that the researcher can think their way through a problem, design a defensible and interesting research project, carry it out thoroughly, and write it into a coherent form. Even if the topic is one which is widely studied, the researcher brings their own particular readings of the literatures – and if they are doing empirical work they also bring the particular site, sample and method – to the study. These are what makes another look at a very well trodden area ‘the contribution’.

But even if the magic says-it-all article does appear it is still possible to incorporate it into the thesis, with a note to say that it wasn’t available at the time the research was designed. Sometimes this even appears as a post script. PhD examiners understand that this happens, and they are really not expecting to see the equivalent of a Nobel prize winning piece of work.

In reality there are very few of us who don’t work in fields where there are lots of other people slogging away just like we are, and the chances of someone else doing something pretty similar are relatively high. Very few of us find the unknown manuscript, or develop the killer app. Rather than see this as a problem, finding someone in or around the same question ought to be an opportunity for pleasure – there is someone we don’t know who we can potentially talk to, maybe even work up a collaboration.

The problem of someone else doing the same work changes for post doctorate researchers. While arts and social science scholars do not generally wake up in a cold sweat thinking that someone else has found the key to the universe or the cure for cancer before we have, there are still a couple of potentially nightmarish issues.

Doing the same work as someone else is an issue when it comes to publication. I recently encountered someone who was just about to write an article about a neglected form of poetry. But when she got the contents list of a newly released book in her field she saw there was a chapter addressing the very set of literatures she was also working on. What’s more the chapter appeared to be arguing the same thing she had intended – that this was an interesting set of texts which offered new insights into understandings of the period. Catastrophe!

Now this really was depressing – but not for long, as it turned out. The researcher was able to take this chapter as already written and it became a building block to which she could add. Despite her initial alarm, she was quite quickly able to find another angle on the texts. This allowed her to write a different article to the one she had first intended – but it was one which did make a further contribution to the emergent area.

Not everyone is so lucky as to find a new way to write about the shared topic, of course. People who are working in densely populated scholarly areas often have to work pretty hard to get something new to say. It can be quite tricky to turn a perfectly acceptable research project, which ends up coming to a fairly standard conclusion, into something interesting. Such projects could have a real impact on policy and/or practice, but require more imaginative work to get taken up by a journal.

The other being-in-the-same-place nighttime terror comes from the funding game. Some research councils believe that having different approaches to the same topic is potentially useful. Others however do not like funding more than one project on the same topic. It is a dreadful experience to spend ages on a bid for funding to then discover that someone down the road has put in something terribly similar on the same topic. The research council probably won’t fund both and will make a decision based on some kind of fine discriminator.

It is always important for those engaged in the bidding game to search through funded project archives to see what has already been done and reported, and what is underway, so at least avoiding the been-there-done-that funder response. It also helps to find out from someone in the know the council’s ‘hidden’ rules about double-funding. And of course it’s crucial to be well networked enough to know who is planning to do what – joint proposals are always possible.

Do you have any more angles on the researching–what’s-already-been-done issue? Do you agree that it’s not necessarily a problem, particularly at doctoral level? Got any stories to share?

the big book thesis has some advantages

This final post in the series on publication in the PhD and as the PhD comes from Dr
Greg Thompson, an Australian Research Council funded early career fellow at Murdoch University. Greg also blogs at Effects of Naplan and tweets as @effectsofNAPLAN

As an academic who just squeezes into the definition of “early career” in a School of Education, the series of posts by Pat Thomson and Katie Wheat about the merits of theses by publication against the “big book thesis” caused me to reflect on my experience of being both a candidate and a supervisor, how much I have learnt and how much more I have to learn.

The point I would like to make in this post is that there is an upside to the big book thesis.

I wrote the traditional monograph. Subsequent to it being passed, it yielded a couple of conference papers, a couple of journal articles and was published as a book. It also landed me a job as an academic in a university.

Now, I do not claim to be an expert on either thesis modes, nor am I attempting to argue that one mode of thesis is better than another. They both have advantages and disadvantages.

I have found that the preparation of doing the big book thesis has served me well in my early career in academia. Of course, this is offset by the acknowledgement that the majority of PhD candidates no longer find tenured jobs in the academy. Whether people feel well-served by the big book thesis if they do not enter academia is a moot point.

However, in an environment where there is an increased interest in thesis by publication, particularly as universities become subject to various research metrics and rankings measuring outputs and their quality, I am aware of more examples of candidates undertaking thesis by publication where emphasis is placed on the outputs.

This leads me to wonder whether this movement to outputs pays enough attention to, or values enough, that moment when a candidate realises that they can equal, or have surpassed, the expertise of their supervisor in the area of their study.

It is a rich and powerful moment, and one to be celebrated. I still remember that moment in my candidature when, whilst discussing a complex theoretical point, I realised that I was engaging as an equal, rather than as a subordinate. That I ‘knew my stuff’ and could argue beyond the limits of the project on which I was engaged. That I was ready to engage in academic debate with others, and had something that was worthwhile and important to contribute.

Now, when I supervise PhD candidates, I explain this as a critical part of their journey, and a moment that I am both trying to cultivate and will welcome when it happens. It happens at different times, rarely early, but hopefully before the end of the thesis process.

I wonder if this is made more or less likely by the mode of thesis that a candidate undertakes? In other words, is the big book thesis better at facilitating that process? I doubt that if I had been focused on outputs this moment would have been as important or significant, or maybe as likely. As somebody who has seen and experienced the best and the worst of the peer review process, I also doubt that we can rely on reviewers who cannot know they are giving feedback to a candidate or a full professor to fulfil this function.

I have no answers to these questions – the limits of my experience prevent a more developed picture. But I would value your perspectives and experiences. We talk a lot about imposter syndrome – is it more or less likely in the big book or publication thesis?

I think these are important considerations that have practical consequences.

At the end of 2011 I was awarded an ARC Discovery Early Career Research Award. As part of its co-contribution, my university offered a PhD scholarship to work on the project. This placed me in a dilemma – in some ways it would have been easier to offer it by publication. In the ARC model, Investigators commit to outputs including the number of articles they will publish throughout the grant. Having a PhD student producing their thesis by publication, with the supervisor as a co-author, would count as outputs.

But I thought it would be worthwhile to canvas some more experienced colleagues and talk to some candidates about their perceptions. Should the scholarship be for a thesis by publication or by monograph?

From the anecdotal perspectives of supervisors and candidates, there are merits and problems with each of the modes of thesis. Monographs can be formulaic, require a structure that is stultifying, require an academic artifice that hides practical applications and can make it seem like there is no light at the end of the tunnel.

Thesis by publication can lack coherence as a stand-alone project, rely on the vagaries of peer review, journal publication schedules and time frames and candidates compete for journal space with more experience and celebrated academics. However, one comment caused me to sit back and reflect – the candidate who only co-wrote with their supervisor always remained subordinate. They never experienced becoming a peer.

In the end I offered the PhD scholarship by monograph.

Thompson, Greg Who is the good high school student? Cambria Press