How to do research

Every research project should aim to be written up as a paper eventually (Paper: article in a scientific journal). Therefore, planning a paper and planning a project are intimately connected.

These are the steps that I usually go through for each research project:

(1) I have an idea, or intriguing observation, or puzzling data in another study. I develop this into a clear question with interesting potential answers: the hypotheses. Feasibility and (potential) impact are the top criteria that determine whether a project is a good one. Neither is completely knowable beforehand, but important to have a good guess. This is where the experience of your mentor(s) (including peers, senior students or technicians in the lab, or colleagues) is most important. I recommend to accept a lot of guidance from your current or future mentors on what is a good project. Actively seek that guidance if it is not given right away.

To determine feasibility, you'll have to decide on principal methods, and define exactly what your hypotheses would predict in terms of outcomes. This can be trivial in some cases, but not in others: it's important to really rack your brain *beforehand* about what other hypotheses are possible and what outcomes they predict, as well as what other outcomes are possible, and what hypotheses they would imply. Every project in which 'the data is not consistent with either hypothesis' or 'the data are not easy to interpret' usually was preceded by not spending enough time on this step. 

To determine impact, you need to find out if other people would care about this answer. Surprisingly, this is really hard to accept: impact is *about other people*. See if you can 'sell' your project to peers, or visiting scientists, or even your mentor. If they react with 'meh', either re-think your sales pitch or change your project. This is the part of research that most benefits from a good knowledge of the literature, but also from conversations with other scientists in your field, going to seminars slightly outside your area, and going to conferences. Lab meetings are a great first sounding board. The same project can be made to sound interesting or boring depending on whether you manage to connect it to issues and key questions that other researchers think about. Yes, this all requires practice, but if you are a grad student or more senior, this is really a skill you need, and you really should have at least 1-2 projects that do sound exciting to other people in the field you imagine getting hired in later.

  • Undergrads: assume that you cannot come up with a project completely by yourself. Choosing the right question is something that is very hard without a lot of experience. Do suggest ideas you have to your mentor(s), however. Model your ideas on previous types of questions that your lab has studied. Remember each project answers only a very specific question, but that you need to choose that question so that its answer will have wide implications.
  • Grad students: assume that you are at the beginning of learning to do this. Get a lot of feedback. Do not assume it is easy, or that you can ‘try out’ a lot of different projects. This step, coming up with the right question and the right hypotheses, can make your research easy and productive or a hard slog. My personal philosophy is that to some degree you just have to take a plunge: you cannot know what the perfect project is in advance, and you can spend way too much time deliberating what to do instead of doing it. Your adviser should prevent you from too foolish a choice if you listen to them. IMHO, spending 1-3 months of deliberate effort on reading literature, discussing specific project ideas with adviser and others, and writing brainstorm documents should be enough to define an outline of a thesis and start on your first project - you can always change directions and details later. 
  • Postdocs: your mentor will likely assume that you are willing to be a creative driver, that you will take leadership in your project and come up with specific questions with little input from your mentor. However, it is always a good idea to get feedback. To capitalize on your mentor's experience, both conceptual and technical, you should deliberately think about where the most interesting overlaps between their and your expertise and interests are. Too much overlap and it can seem like you are not sufficiently owning your project; too little overlap and you and your mentor will have little to talk about and you will not get that much out of the collaboration (also see my point on authorship below). 

(2) I find who my immediate collaborators will be. Who can teach me the methods, who has the equipment, who has the time to actually collect the data, etc. Make the roles, expectations, and reward for each person participating clear; define who is ‘leading’ the project. The project leader checks that everyone does their job on schedule, that the parts fit together and the project works out, and writes the first draft of the manuscript and thus becomes first author on the paper in the end. Generally you should be proactive and not shy about contacting possible collaborators. Be clear and honest with them, and assume that if someone is senior to you they have less time and will expect you to do more of the work. What I mean by this is: you can pretty much email any scientist in the world to ask if they want to collaborate with you on a project; noone will think this odd. You should clearly define though what it is that you want them to contribute, in what timeframe, and what resources will be required. E.g., do you just want to skype with them to discuss a project? Or do you want to visit their lab to learn a technique? If so, do you expect them to cover your travel costs or any costs of supplies and equipment? Or do you want them to visit you or your field site? Or do you want to send them samples of something and they do an analysis? Again, who is paying the costs of supplies? Will a student or other staff member of their lab be involved? If the person is at your institution, generally I would schedule an in-person meeting and first give your sales pitch about the project or idea, and then brainstorm the practicalities with them. Do not underestimate how much easier your life will be if you explicitly discuss responsibilities, costs, and authorship order beforehand.

(3) Develop the specific experimental plan, including sample sizes, methods of measurement, precision, planned statistical tests. Make sure you have and understand all equipment, experimental subjects, space in the lab, software needed. Sign up for equipment use if it is shared equipment. Browse through methods used in our lab and other labs (from the literature). It is also worth thinking about your publication plan early on; in fact I would recommend starting a file that will become your paper draft, i.e. starting to write an outline for your paper.

(4) Do it. After the first round of tests/measurements, assess if the methods is working. If not, revise and start again. Do not waste time: once your experiment has started, it is generally best to crank through, do the highest sample size feasible (in our field there is rarely a prior estimate of effect size, and rarely a case where the highest feasible sample size is unnecessarily large). Observe good rules of data management and record-keeping.

(5) If you did steps 1-4, the analysis part should be pure joy. You have a data file in the computer that tabulates the data you need for your analysis, and you know what analysis you are going to do. Now you really test your hypotheses, which, ideally, will lead to a fascinating outcome no matter the results of the stats test. Often you have to do a couple of intermediate data formatting/summarizing/analysis steps before you get to the final p-value (see advice on practicalities of data analysis and statistics here). But: do not be distracted from your well-planned goal. If there was a test that will distinguish your hypotheses, and you checked that the methods worked, then the outcome of the test tells you what your conclusion is. Do not second-guess it or spin it. However, there may be new observations or interesting patterns you didn’t plan on but that also answer interesting side questions. Sometimes the side questions end up more interesting than the initial question. Think hard about whether you need additional experiments to draw appropriate conclusions here though, since you didn’t plan for this and thus presumably your experiment and controls are not ideal for answering these additional questions.

(6) Write your paper, then submit it for publication.


Other people's advice:

Basic experimental design by AR Goldsmith

Find the right literature and read it regularly - tipps from Brian Enquist