Part 3: I am not doing PhD, I am doing Research.

[Thinking]

Anyone who does research required to think hard. To solve problem. Thinking usually happen to everyone, however being a better thinker and keeping our mind sharp, creatively and critically, is not an easy task. The thinking process requires mental effort in which physically, our nervous system will consume more glucose to execute. The effortful mental activity appears to be especially expensive in the currency of glucose. Also, it is not intrinsically pleasurable to do hence most people avoid thinking when possible. However, as computer science researchers, we should flow naturally with our thinking. The flow means we always have a state of effortless concentration to deeply engage on readingwriting, and coding activities which all require thinking. Otherwise, without the flow, we’ll find research is a very unpleasant experience that will trigger negative emotion.

Thinking is a valuable mental ability that our ancestors have developed over three thousand years. However, not too many people know how to think even though this topic has been discussed over centuries, from philosophical to scientific aspects. Most of techniques that cover thinking methods are actually scientific. Even historically science was a result of an evolution from philosophy, I am biased that the thinking should be pure scientific.  Take a look on Coursera for example, we can easily find courses like core thinkingmathematicalalgorithmicmodel thinking, etc. If I continue to search (which I will not do), maybe I can find more philosophical thinking approach, but I believe the material will be outdated in this postmodern scientific era.  There a lot of thinking techniques, huge amount of knowledge that have been published.  I will not confuse you this time. I want to share my own experiences.

Yes, I want to write about how to think, or more precisely to share proven practices to keep me in thinking flow.  The problem is, when thinking about something that I currently explore or not fully understand (or partially), I always have uncomfortable feeling called confusion. I am unhappy and feel stupid as my understanding is shallow or biased sometime. Some book authors wildly use terms/jargons that I dont understand, or jumping around with mathematics, visualisation or analogy to fool me. It is absolutely the worst distraction while reading.  In return, I can easily leave the problem, procrastinate and do something else.

[1] To think creatively is to know many concepts and get ideas. When I am flowing with algorithm contents, I will think algorithmically. The same when I read math, physics, my thinking will biased to what is currently active in my brain’s memory region. The main question will be – what we have to have to be able to think comprehensively? Ideas. Yes, having as many ideas as we can. We can’t be creative in Artificial Intelligence research for example by only learn current knowledge in machine learning. We need inspirations from all related disciplines, statistics, probability, physics, or neuroscience for example. Learning how to think really means a continuous learning process from many disciplines. Creative thinking happens when our brain can relate many concepts, recognize patterns, build new ideas, propose new theory/model, and finally simplify. In fact, many beautiful concepts in science are very simple. But the simplicity comes from complex thinking process with many inputs – inductively or deductively.

[2] To think critically is to recognize and remove biases. One of most influential book I have read is the Thinking Fast and Slow – written by a Nobel Prize winner Daniel Kahneman. He proposed a beautiful model to understand how our brain works – System 1 (Fast, automatic, frequent, emotional, stereotypic, subconscious) and System 2 (Slow, effortful, infrequent, logical, calculating, conscious). He explained how those two systems works, its characteristics and described common biases we can have as consequences of those two systems.  Prof. Kahneman introduced methods for me to recognise biases in thinking and that is important to learn how to think. If we can recognize our own biases, we can use that to probe and debug other people thoughts or opinions. Lot of cognitive biases are listed here., specific for Thinking Fast and Slow, you can find here. I will not be able to cover all of them, but I do suggest all AI researchers to read that book.

[3] The creative idea is generated randomly from the most active brain. I don’t have scientific proof to say that, but I experience it many times. The myth is everyone can have creative ideas. I agreed with one condition. We have to deeply engage and flow within the problem to get ideas. Some engineers who are working deeply on graph problem can get creative ideas on how to solve the problem, but not others who are not. The same thing with business. The motivators will never be able to give precise ideas without doing the business. Thats is bullshit. When we do something our brain is active and that is the first rule to have ideas. However, the ideas are not coming on time. It is random and probabilistic process in nature. Sometime while I am drunk or running out after long hours of reading or writing. Last nite I discussed this with my best friend, Ari, we want to propose an additional System 3 model to Prof. Kahneman as random ideas generator connected to System 1 and 2. Sounds crazy but we do believe it is exists.

Thanks for reading my two cents!.

TSMRA, Jakarta, March 2016.

Why I love to teach even I am not really good at

One thing that I do enjoy in my job is teaching. No, I don’t believe I can really do my job without teaching ! My personal reason is simple, when I do teach, I contribute to next generation scientists and engineers in good way. It is not just about delivering engineering contents like machine learning, C++ or algorithm, but to me, to inspire. Sounds like a big leadership word – inspire. Yes, it is. When I was student long time back, I could see what happened to me once inspired by Professors. I will do hard, more than it normally takes to understand more. Learning happen when students got really inspired to learn.  Well-thought words coming from inspiring Professors will go directly to our brain – set it permanently for long time.

When I was in University lab – back in 1995-2000, we have a learning culture to teach each other. We learn something together in the lab and each one of us can become teacher and student. It was interested. Sometime, we think we do understand something, but we can’t teach to other people. It may because our understanding is too shallow, or we don’t really understand it. Take for example, all of us understand banana. We can explain banana very well to our kids without any preparation even without slides. But can we explain concept of energy as good as banana? Maybe not. So, for me the best way to test my understanding is to teach it to others and ask if they understand it well. To test whether we don’t know, partially know or know a concept. You’ll be surprised with question they ask or maybe being intimidated if they don’t. The fact is, profound question from inspired students is always valuable. It is often source of ideas or new research for me.

Creativity and ideas need triggers. In any thinking process there are moments when everything is not smooth, blank and I don’t have any ideas. Teaching is a good relaxation for me to switch back into the thinking rhythm. Relax when I have nothing more critical to do. Yes, at least I do something while I have no more important ideas to do. Or at least I can say, I am teaching new hires and contributing to prepare them. Beyond all of it, I am thinking when I am teaching, as I want the students to understand what I explain from their current state. I was explained concept of object and pointer in C++ yesterday and I had to choose the most precise words instead of useless analogies. I do it carefully as I know the impact of a shallow understanding. The bonus is, usually I can think something new or find new way to look/explain same thing.

I think I am not really good in teaching. I never ask the students how they think of me, as I’m not really care or maybe afraid to hear the answer. Somehow I still think I am not a good teacher, but I enjoy teaching. In fact, I have accepted (or even created) jobs in IT industry that allows me to keep teaching. With that I hope I can say, I love to teach even I am not really good at.

TSMRA, Jakarta, March 2016.

Part 2: I am not doing PhD, I am doing Research.

[Writing]

Basically we can put like this – reading affects writing and writing affects reading. Good writers are also good readers. Activities while reading and writing share many of the same working brain’s memory region, and potentially draw upon the same as well as unique cognitive systems. Neuroscientists perhaps have more evidences or maybe can praise it more to creativity, but personally I can sense the strong correlation between my writing and reading activities. I hardly write if I don’t read. When writing this blog, or reading classic machine learning book that are well-written, I can feel a super computer inside my brain working in background with harmony. Writing and reading are coupled, can’t be separated when we deeply involve in research.

Why we have to write? That was an interesting question from a student in my Research Methodology class. I believe, as researcher, we have more than enough reason to write. However putting it in discipline way is quite hard. Not all of students have discipline to write even they know it is mandatory requirement to write dissertation.  Not only for communication and publication of research results, writing is the best way to debug and validate our own thinking. Maybe the best question is why writing is hard? And my last answer was potentially because we don’t read enough. Yes, writing is sometime painful. Even in our R&D organization, engineers who must write simply don’t do. For researcher – it is not an acceptable excuse. It is a fastest way to fail.

I am not going to teach how to write – many books done it and we just need to practice. I prefer to suggest reading a lot – reports, books and papers related to our research. The first law of writing is to have “initial” contents and contexts in order to know what we want to write. It doesn’t have to be complete or perfect when started. In fact – perfectionism will end up as never ending polishing. The practice I did for so long is to write letter to someone I respect or love. I will do it in the most structured way – to make him/her understand correctly. Pick right words carefully, meaningfully and set an interesting story line or contexts. However, think of it as a conversation with them – not everything goes perfectly; some lost words or maybe mistakes, but it should be flowing well. It is similar to technical blog post or paper. It must be flowing like a conversation to someone you respect or love.

Writing for research is not a trivial task without practices. I can’t cover the details now, need more time until I experiences publishing new paper or patent (as I am new PhD student as well). However, based on my master degree experiences on theoretical physics, writing technical paper to about telling stories of what we have done in the lab. We have to make it easy to reader (usually also physicists) to understand what we have done, how we did it and what insights we have. We have common language that maybe hard to understand for non-physicists and we also trimmed some concepts as the pages of publication are limited. However, it not a sales brochure to sell crappy things with big claims. It is plain text – to explain our research idea, process, and results.

Writing academic / scientific research paper requires guidance and practices. A lot of practices and usually take sometime to be familiar. You can easily find guidance in internet, however, to be more organized, pick book that is well-written. I am not going to cover here, but based on my personal experiences, reading lot of papers also can help us to be more familiar with writing.

TSMRA – Jakarta, March 2016.

Part 1: I am not doing PhD, I am doing Research.

I found myself signed PhD (Permanent head Damage) program for deep learning topic. It was a conscious decision to put myself into new rhythm, to explore current trends in AI technology, do research and perhaps contribute little things to the body of knowledge. Not a bias decision, as I wanted to get “in rhythm of research” by putting additional external pressures. Decision had been made!. For me, the key is research itself, however, I attend two classes for refreshment – Research Methodology and Philosophy of Science as starting point. Very excite to find myself as student, instead of lecturer. Even I found myself lost in philosophy class, I think it is also entertaining to see how can I fool myself with philosophy.

I read lot of PhD guides before I took the PhD program. Many cover formal research methods but only few cover basic skills for doing it on artificial intelligence area. What I mean basic is concrete, real and must have skills to enjoy the whole research process without losing interests. Even for myself, hard to define it formally, but I believe we as researcher already “know” it, just have to follow our heart and intuition consistently. For me is about curiosity, reading, writing, thinking and coding.

[Curiosity]

It is actually positive behavior or emotion for being curious, in regards to explore, investigate and learn something. It must be persist by time, not just an instance interest, but something continuously burning inside our brain. Curiosity will help to differentiate between knowing the name of something and understand something.  In more emotional statement, curiosity is related to fall in love with some intellectual activity to explore something. But what? Is the deep neural networks for example are really interesting to go deeply enough? Or nearly everything should be interesting if we go deep like Feynman said? I prefer to work as hard as much I can on things I like to do the best. Hard means – in the most undisciplined, irrelevant and original manner. Forget what we as students want to be (like graduate timely by following common methods) and focus on what we want to do (research).  The plan is simple, do it hard and find beauty on it to pay off. It is not something normal for some people, but like business, there are only two options in research, fun (intellectual curiosity) or profit (what the hell, papers and patents?).

It is not easy to explain something like curiosity in formal way. But usually, my best reality check for the existence of curiosity in student is by asking, how long time you have interest in that? And tell me stories how hard have you learnt about it. Period! If you tell beautiful stories, then lets the roller coaster begin and enjoy!.

“Physics is like sex: sure, it may give some practical results, but that’s not why we do it.” Richard Feynman. 

[Reading]

It is impossible to do research without reading. It is a must have skill, no option!. Body of knowledge has been built for decades, few are well explained on books and rest are mostly scattered on papers or other technical documents. I was in debate with my friend regarding books. We concluded there are hundred books for neural network topic, but only few worth read, that really explaining key concepts or show teaching style. It is impossible to find one book that explains all we need for research, but also it is impossible for us to read all books. We have to find books that explain the key concepts well but it is hard to find. Great if our Professor can recommend few books to finish, however, I think he also faces same problem.

In case of deep neural networks, I found Michael Nielsen online book was entertaining. I like the way he explain deep learning, reminded me they way Feynman explain Physics. His book explained key concepts for me to read more formal or academic style of books like Bishop on Neural Networks and Machine Learning. Other classic machine learning texts may also help – like Murphy MLPP and Hastie’s ELS, but I think we need to have right purposes on reading books. The best purpose that works for me is to understand key concepts first.  The topic is already too big and hard to follow state of arts without understanding key concepts. Some people prefer to learn key concepts from online learning like Coursera and Udacity. However, I found videos are less engaging compare to read right books. It might bias to me personally, but I think I have a point. Texts are still the best way to explain complex things as it can set our brain freely to imagine the visuals if the author can pick right words. Not the other way around.

Reading papers is another skill that takes time to practice. You can pick any journals and search your topic and find hundreds “nearly related” papers. Of course we can’t read all, impossible. My physics Prof. Rosari Saleh taught me 20 years ago to categorize papers based on its root Professors or research labs. In case of deep neural networks, it should be Prof. Geoff. Hinton (Google), Prof. Yan Le Cunn (Facebook), Prof. Yoshua Bengio (Montreal), etc. I did the same simple approach and found it very interesting to share. First, I collected all related papers to see if there is anything interest in it. The way I do usually by scan read its abstracts that supposed to be brief. If I found it is very closely related to my topic, I will put some notes on my notebook mind-map by copy paste the abstracts. The purpose is to tell myself to go back on it later, if I already have questions. Reading papers without initial questions potentially will waste time. I like to read with question like:

  • Why it is relates to my topic?
  • How can I potentially use it later?
  • What is the key engineering approach or heuristics tricks?
  • Is there future research areas suggested by authors?

I found that have questions before reading is very important discipline to structure my research notes. The questions can be changed time-to-time depend on my understanding level. I designed the questions and put the answers from papers I read into it, usually, by copy-paste. During the process, it is actually record something into my memory and helps me to relate it with my other ideas from books or other technical document. It is also helping to write later. My friend told me that Quora can help to trigger interesting questions. I will look on it sometime with a hope it will not distract me.

Writing is another essential skill to do research and it is closely related to reading… I will cover it in part 2 of this post.

TSMRA – Jakarta, March, 2016.