What’s the Fuss about KPI?

Some years ago, my buddy Tuck-Cheong Tang forwarded an interesting article by Faria (2003) on the classification of economists, which I would like to share here (click here for Prof. Faria’s profile). Briefly, when more emphasis is placed on the quantity of their publications, economists are classified as r-strategists. On the other hand, K-strategists are those who target top journals as the outlets for their research papers. In his simple model, the author shows that neither r– nor K-strategies can maximize the influence of an economist in the profession. Instead, economists are more likely to be successful if they are able to strike a balance between quantity and quality.

Of course, this paper assumes that academics often strive to maximize the success of their careers (either to be prolific or influential). From my general observation, this is a very strong assumption in Malaysian academia as a whole. For instance, when many Malaysian public universities set publication in quality journals as a key performance indicator (KPI), many academics felt that their comfort zones have been challenged (see their grumblings here and here). It is worth highlighting that quality here is defined as citation-indexed journals such as Scopus or ISI/WoS, but not those top-tier journals targeted by K-strategists.

Even with such a realistic KPI, many are still feeling the pressure. I assume that they have no problem with KPI per se as performance evaluation is common in every public or private organization. Before joining academia, I was told that publishing academic papers is part and parcel of academic life (see here), and professors are those who have achieved intellectual excellence and are widely regarded as the experts in their respective fields of study. Just like the advice given by Sanjiv Das to those prospective PhD students and academics, “If you “know” that you enjoy research and will love teaching, don’t mind working all the time, anytime, for small money but great personal satisfaction, then sign up now. But don’t do it because you think the lifestyle is great, or you believe some rosy picture of an academic in an ivory tower” (click here). So what is the big fuss now? (I am supportive of the stand stated here and here).

I can only assume that they have been misinformed about academic life. As this author suggested,  “To avoid having “dead wood” in academia, universities should also stringently screen potential lecturers at the interview emphasising their expected role once they are in the system. Those who think they cannot comply with the university requirements can seek their fortune elsewhere” (click here).

 

Competence Level Assessment (PTK)

Cuepacs, the umbrella union for civil servants, has been urging the government to abolish the Competence Level Assessment (PTK) for many years (for the latest, see here).

In academia, regardless of whether it is a public or private university, the competence of a lecturer should be evaluated based on his/her teaching performance and research achievements, with the latter getting a larger weight. In Malaysian public universities, PTK is a compulsory requirement for promotion exercises. For instance, PTK 3 is required for promotion to Senior Lecturer, whereas applicant for Associate Professor must have passed PTK 5. I always hold the view that academics should be promoted primarily on the basis of their research achievements. If pure scholarly merit and intellectual excellence are not the basis of promotion, then one would be left with the question of what do professors profess (see the article by Orr, 2000).

I am glad that our Prime Minister Datuk Seri Najib Tun Razak, in his 2011 Budget speech on 15 October 2010, announced that the PTK for civil servants has been abolished and will be replaced with a more suitable evaluation system by June 2011 (see here). According to a news report, when the Prime Minister highlighted this at the national teachers’ assembly at Stadium Putra, it drew thunderous applause from the audience (see here).

In Loving Memory of Prof. Melvin J. Hinich: I

I am deeply saddened by the news that Prof. Melvin Hinich has passed away on Sept 6, 2010 (click here). It is really touching to read through the sharing of those he has influenced either as a researcher, mentor, teacher, or friend (see here). At this moment, I find it difficult to use the right words to describe my sadness.

Mel is a world class scholar whose publications traverse four fields: the spatial theory of voting in political science, time series analysis in statistics, political economy and regulatory issues in economics, statistical theory and methods in engineering (you may read his Wikipedia entry here).

Mel was a great mentor and teacher. When I worked on my master thesis in 2002, I saw the potential of his bispectrum test in addressing the research question. However, I did not have the skills to write the computer code.  I was then trying my luck to see if Mel could share his code. Unexpectedly, given his busy schedule, he responded my email and attached his FORTRAN code. He even invited me to meet up during his visiting trip to Singapore the following month. I was in a state of disbelief that a world class professor would meet up with a master student from a Malaysian university. Without much hesitation, I grabbed this rare opportunity and took a train from Kuala Lumpur to Singapore. We didn’t talk much in that short meeting, as I was in awe of his reputation. But he was patience enough to demonstrate the execution his bispectrum test.

After coming back from Singapore, I wrapped up my master thesis within the next 2 weeks. During the viva voce, the examiner has very few comments, and the thesis was granted a pass with minor corrections. I subsequently extracted a paper from the thesis, and invited Mel to be the third author (my supervisor M. Azali was the second author). He promptly responded with a ‘yes’, and told me that he liked the paper very much. Unfortunately, the three referee reports from Macroeconomic Dynamics suggested major revisions, and eventually we did not resubmit. Mel still kept a copy of this working paper in his website (click here). Despite the setback, this first collaboration has given me the much needed confidence to kick start my academic career.

I have known Mel for 8 years, and it is impossible to share those wonderful moments in this short posting. As I walk down the memory lane, I will compose some thoughts in his loving memory. He has been an inspiration and role model to me, and no word can describe my gratitude to him.

By Kian Ping

LSIBF in ISI/WoS

Given the emphasis on publications in ISI/WoS, it is good to check the performance of LSIBF. To do an institutional search in WoS (click here), we have to search for “Labuan Sch Int Business & Finance”  in “Address”. This will return 13 records (click here).

How about the School of Business and Economics (SBE) in the main campus? To do that, we have to search for “Sch Business & Econ” in “Address” AND “Univ Malaysia Sabah” in “Address”. This will return 5 records (click here). If we exclude proceeding papers, SBE will only have 3 records.

Despite the lack of research facilities and the small number of academic staffs (no full professor in LSIBF), LSIBF is publishing more ISI publications than their sister school in the main campus.

Well done and keep up the good work.


Academic Advice from Sanjiv Das

I came across the blog of Sanjiv Das after reading his 2007 paper published in Management Science (click here).

I find that his posting on “Need PhD?” provides some useful thoughts for early career researchers, prospective PhD students, and academics in general (click here).

I like the concluding remark: “If you “know” that you enjoy research and will love teaching, don’t mind working all the time, anytime, for small money but great personal satisfaction, then sign up now. But don’t do it because you think the lifestyle is great, or you believe some rosy picture of an academic in an ivory tower“.

Enjoy your reading!

UM III: Clear Promotion Criteria

I am a strong advocate of a clear promotion criteria for academics. As pointed out by Kim et al. (2009: 354), the higher education sector is one of those knowledge-based industries in which individual productivity is recognizable and individual reputation is important.  Academics are always promoted based on their research output in recognition for their contribution to academia

What I mean by clear promotion criteria is that they should be objective and measurable. I would like to use University Malaya as a case study (click here for the full audit report).

UM has outlined her target of being ranked in the World’s Top 200 within 2 years and the World’s Top 100 within 5 years. To achieve that, the Vice Chancellor has outlined his strategic plans in the above report, and also the website of the Office of VC (click here). In terms of academic promotion criteria, it is clear that research productivity is rewarded by the university (see slides 12-14). More importantly, the criteria are objective and measurable, for instance 1) the number of citations; 2) h-index of researcher; 3) publications in Tier 1 and Tier journals.


UM II: A Model of Reward Structure for ISI/WoS Publications

In my previous posting on National Academic Award (click here), I noted there is still much room for improvement in terms of the reward structure offered by my home university.

When designing a reward structure based on ISI/WoS (Web of Science) publications, it is important to first understand the underlying mechanisms. Several important points are highlighted here:

  1. Sciences, social sciences, and arts & humanities are three different fields that cannot be put together (for details, click here). It just does not make sense to compare the impact factor of a science journal vs a social science journal.
  2. Even in the same field (sciences, social sciences, arts & humanities), the impact factor should be discipline-specific (for details, click here).
  3. Is the reward structure based on publications indexed in WoS or impact factor reported by Journal Citation Reports (JCR)? UM encourages her staffs to publish in journals indexed in ISI-WoS, which consists of 3 important components- Science Citation Index, Social Sciences Citation Index, and Arts & Humanities Citation Index (click here for notice dated 10/8/2010, and here for WoS fact sheet). However, if the reward structure is based on impact factor, one has to bear in mind that JCR only provide impact factors for journals indexed in SCI and SSCI, but not AHCI (click here and here). How do you then reward those staffs who publish in AHCI if the incentive is structured around impact factor?
  4. How do we define high impact journals? Well, not all journals indexed by ISI-WoS are considered high impact. UM defines high impact research as any research that could produce publications in Tier 1 or Tier 2 ISI/WoS journals (click here).  I suppose it is Q1 and Q2 in the JCR.
  5. The next related question is whether the university rewards publications indexed in ISI-WoS or just high impact journals. UM has structured her reward system according to ISI Journal Tiering, that is Q1- RM6000, Q2- RM4000, Q3- RM2000 and Q4- RM1000 (click here). Apart from that, the university has also announced a cash incentive of RM50,000 for each publication in Science or Nature (click here). This is a good model, not only in monetary terms but also in ensuring fairness across disciplines.


UM I: RM50,000 for Publication in Nature or Science

When I was preparing a module on “Publishing in High Impact Journal” , my google search directed me to the website of UM Vice Chancellor Office (click here). I am really impressed by their focus on research excellence, in particularly their strong drive for ISI/WoS publications.

In a notice posted on 16/4/2010 (click here), UM VC has announced a new reward for publication in Science or Nature, i.e. a monetary reward of RM50,000 per paper. This is similar to the incentive given by Badminton Association of Malaysia for World Championships gold (click here or here). The most attractive part is that there is no limit to the number of paper one can claim, provided that UM academic staff is the first author. This incentive will surely motivate their staffs to publish in high quality journals.

As a Malaysian, I am glad to learn that UM is moving in the right direction, and hope that the university will one day be ranked among the top 100 leading universities in international rankings.


The Irish Economy Blog

As I was browsing the website of Prof. Philip R. Lane, I came to know about  The Irish Economy blog which he founded (click here). Perhaps, Malaysian economists can follow their approach, and connect with the public via blog.

Area of Expertise

“Area of Expertise”  refers to the area where one is a specialist in knowledge of and an authority on information in that particular area. In academia, professors are generally regarded as the qualified experts, because they are supposed to be promoted primarily on the basis of their research achievements.

When I was the managing editor of Labuan Bulletin of International Business and Finance, I often need to seek the expertise opinion of  a referee to evaluate the contribution of the submitted manuscript. To find the suitable referee, I would use Google Scholar, Scopus or ISI/WoS. Sometimes, I found that some professors in my discipline did not have many publications in the area they were supposed to profess. At the other extreme, there were some academics who have published extensively in a particular area yet did not even have a PhD. When such situation arose, I often opted for the latter. In my view, a solid publication record is a necessary (or even a sufficient) condition for an academic to be labeled as “expert”.

After returning to Malaysia, I often heard academics with PhD claimed that their “area of expertise” is XXX.  Again, a PhD does not qualify one to be an expert. It is just a recognition that the successful candidate has proven his/her ability to conduct independent research, and that the PhD thesis materials are publishable (sometimes, due to varying reasons, the potential of publishable never fulfilled).  A 3-4 year of rigorous PhD training helps the candidate  to develop an area of specialization, but he/she still needs to further research and publish before that area becomes his/her “area of expertise”. Even someone without PhD can become an expert  in his field, as long as he has solid publication records.

I always look up to John Creedy, the Truby Williams Professor of Economics at the University of Melbourne (check his bio here). In terms of academic qualifications, Prof. John Creedy has a B.Sc. (Economics with Statistics) from the University of Bristol, and a B.Phil. (Economics) from Oxford University. A quick check of his CV reveals that he was promoted to Professor of Economics at Durham University in 1978, 6 years after he obtained his B.Phil. Even without master and PhD degrees, Prof. John Creedy is widely regarded as an expert in economics. His excellent publication records include  32 books, 17 edited books,  54 book chapters, and 234 journal articles. He is often ranked as the top economist in Australia (see, for example, Macri and Sinha, 2006, and references cited therein).