Today I met my PhD supervisor face-to-face for the first time. Obviously, most of what was said primarily relates to my specific research topic, but we did discuss a few general things which may be of use to others (and myself in the future!)

Learn only what you need

Firstly, a PhD should not be treated as an extension of undergraduate studies. The purpose of a PhD is to do novel research. That’s it. Anything you learn should be learnt with that aim in mind.

For most students, this is a radical change of approach. The point of undergraduate study is to pass exams, usually in a broad range of topics under the umbrella of a specific subject. This is not the purpose of a PhD programme. My supervisor advised me to have a concrete problem in mind and read papers and book chapters on that topic only when absolutely necessary.

He told me that he has seen many, many students work very, very hard for the first couple of years of the programme, only to have no publishable work.

By the time you get to graduate school, you should have a solid enough foundation in the essentials of your field to work top-down, from problem to theory - building up an ever large theoretical base would be a waste.

What counts as research

Once that point was sufficiently hammered home, we discussed what does and does not count as novel, and therefore publishable, research.

Existing techniques applied to new datasets are not novel enough to merit publication in top-tier conferences and journals.

New systems built from existing techniques layered together also don’t count, even though they might be a good foundation for building a startup.

The only thing that really counts is a new technique. A new thought. A new way of doing things.

How to be done

To get a PhD in machine learning, you have to have at least 3 publications in top-tier conferences. The reason conferences are preferred to journals is a subtle one that I was unaware of until today.

When you submit to a journal, if you’re idea is at least novel, any other short-comings can be fixed. Poor writing can be improved, a more thorough overview of the state of the art can be added, and charts can be simplified. This means that your paper will eventually get accepted in most cases.

On the other hand, conferences have strict deadlines and outright rejections. No opportunities to edit this or add that. The bar is higher.

Of course, this means that to get 3 papers accepted in top-tier conferences you have to write more than 3. Any paper that has made the rounds and been universally rejected can be put forward to lower-tier conferences, or expanded and made into a journal article, so it’s not completely useless. But to have a real shot at a research position, you do need those 3 in top-tier conferences.

It was my understanding that research students were most productive at the tail end of their studies. That they spent the first 18 months to 2 years developing a knowledge base and collecting ideas to later put into publication. My supervisor told me that this was a dangerous way to think. He said it’s much better to aim for ~1 paper per year (in a 3 year programme).

Go narrow

The best outcome of a PhD programme is that you be known as the person who researches X. When a fellow researcher thinks of a particular topic or problem, your name comes to mind. Occasionally, this is done through one major result or innovation, think Chris Hawkins and Q-Learning or Ian Goodfellow with GANs. But more often than not this kind of reputation gets developed over time with incremental results in a specific area.

For that reason, it’s important that your first paper is a kind of introduction to the questions you’ll be considering. It’s your first attempt at coming to grips with some particular open problem.

A research snowball

Having writing a paper in your first year as a target focusses the mind on the important parts of the research process. After all, it’s very easy to get distracted reading papers on Arxiv and watching conference videos on YouTube. Actually writing a publishable paper in your first year, on the other hand, has a lot of downstream benefits.

  1. It allows you to quickly get an idea of whether your research is novel enough to merit entry into a top conference. Allowing you to recalibrate before going any further.
  2. It means that you’ll have the basic framework of a publishable article in your toolkit to use as you discover new things in your research, allowing you to get those 3 publications quicker.
  3. Your first results serve as a jumping-off point, allowing you to delve deeper into that particular topic, focussing you ever more finely into a specific research area.

This snowball effect ultimately means that everything from knowing what to research, to getting published, to compiling your thesis will be quicker and less painful than it might otherwise.

With all this advice fresh in my mind, I’m going to go and get started!