I found myself signed PhD (Permanent head Damage) program for deep learning topic. It was a conscious decision to put myself into new rhythm, to explore current trends in AI technology, do research and perhaps contribute little things to the body of knowledge. Not a bias decision, as I wanted to get “in rhythm of research” by putting additional external pressures. Decision had been made!. For me, the key is research itself, however, I attend two classes for refreshment – Research Methodology and Philosophy of Science as starting point. Very excite to find myself as student, instead of lecturer. Even I found myself lost in philosophy class, I think it is also entertaining to see how can I fool myself with philosophy.
I read lot of PhD guides before I took the PhD program. Many cover formal research methods but only few cover basic skills for doing it on artificial intelligence area. What I mean basic is concrete, real and must have skills to enjoy the whole research process without losing interests. Even for myself, hard to define it formally, but I believe we as researcher already “know” it, just have to follow our heart and intuition consistently. For me is about curiosity, reading, writing, thinking and coding.
It is actually positive behavior or emotion for being curious, in regards to explore, investigate and learn something. It must be persist by time, not just an instance interest, but something continuously burning inside our brain. Curiosity will help to differentiate between knowing the name of something and understand something. In more emotional statement, curiosity is related to fall in love with some intellectual activity to explore something. But what? Is the deep neural networks for example are really interesting to go deeply enough? Or nearly everything should be interesting if we go deep like Feynman said? I prefer to work as hard as much I can on things I like to do the best. Hard means – in the most undisciplined, irrelevant and original manner. Forget what we as students want to be (like graduate timely by following common methods) and focus on what we want to do (research). The plan is simple, do it hard and find beauty on it to pay off. It is not something normal for some people, but like business, there are only two options in research, fun (intellectual curiosity) or profit (what the hell, papers and patents?).
It is not easy to explain something like curiosity in formal way. But usually, my best reality check for the existence of curiosity in student is by asking, how long time you have interest in that? And tell me stories how hard have you learnt about it. Period! If you tell beautiful stories, then lets the roller coaster begin and enjoy!.
“Physics is like sex: sure, it may give some practical results, but that’s not why we do it.” Richard Feynman.
It is impossible to do research without reading. It is a must have skill, no option!. Body of knowledge has been built for decades, few are well explained on books and rest are mostly scattered on papers or other technical documents. I was in debate with my friend regarding books. We concluded there are hundred books for neural network topic, but only few worth read, that really explaining key concepts or show teaching style. It is impossible to find one book that explains all we need for research, but also it is impossible for us to read all books. We have to find books that explain the key concepts well but it is hard to find. Great if our Professor can recommend few books to finish, however, I think he also faces same problem.
In case of deep neural networks, I found Michael Nielsen online book was entertaining. I like the way he explain deep learning, reminded me they way Feynman explain Physics. His book explained key concepts for me to read more formal or academic style of books like Bishop on Neural Networks and Machine Learning. Other classic machine learning texts may also help – like Murphy MLPP and Hastie’s ELS, but I think we need to have right purposes on reading books. The best purpose that works for me is to understand key concepts first. The topic is already too big and hard to follow state of arts without understanding key concepts. Some people prefer to learn key concepts from online learning like Coursera and Udacity. However, I found videos are less engaging compare to read right books. It might bias to me personally, but I think I have a point. Texts are still the best way to explain complex things as it can set our brain freely to imagine the visuals if the author can pick right words. Not the other way around.
Reading papers is another skill that takes time to practice. You can pick any journals and search your topic and find hundreds “nearly related” papers. Of course we can’t read all, impossible. My physics Prof. Rosari Saleh taught me 20 years ago to categorize papers based on its root Professors or research labs. In case of deep neural networks, it should be Prof. Geoff. Hinton (Google), Prof. Yan Le Cunn (Facebook), Prof. Yoshua Bengio (Montreal), etc. I did the same simple approach and found it very interesting to share. First, I collected all related papers to see if there is anything interest in it. The way I do usually by scan read its abstracts that supposed to be brief. If I found it is very closely related to my topic, I will put some notes on my notebook mind-map by copy paste the abstracts. The purpose is to tell myself to go back on it later, if I already have questions. Reading papers without initial questions potentially will waste time. I like to read with question like:
- Why it is relates to my topic?
- How can I potentially use it later?
- What is the key engineering approach or heuristics tricks?
- Is there future research areas suggested by authors?
I found that have questions before reading is very important discipline to structure my research notes. The questions can be changed time-to-time depend on my understanding level. I designed the questions and put the answers from papers I read into it, usually, by copy-paste. During the process, it is actually record something into my memory and helps me to relate it with my other ideas from books or other technical document. It is also helping to write later. My friend told me that Quora can help to trigger interesting questions. I will look on it sometime with a hope it will not distract me.
Writing is another essential skill to do research and it is closely related to reading… I will cover it in part 2 of this post.
TSMRA – Jakarta, March, 2016.