Intangible Capital

• Home
• About
• Login
• Archives
• Submissions
• Publication fee
• Indexing & Statistics

Home > Vol 13, No 4 (2017) > Somensi

Knowledge construction about port performance evaluation: An international literature analysis

1Universidade Federal de Santa Catarina (Brazil)

2Universidade do Sul de Santa Catarina (Brazil)

3Universidad de Valencia (Spain)

Received November, 2016

Accepted March, 2017

 

Abstract

Purpose: This study aims to identify and analyze the characteristics of international scientific research that addresses the segment of literature referring to port performance evaluation, in order to identify the existence of a theoretical alignment between the concept of performance evaluation, as an area of knowledge, and the practical application of port performance evaluation.

Design/methodology: For the approach to the problem, this paper makes use of qualitative research, analyzing bibliographical portfolio characteristics related to port performance evaluation. An action research strategy was adopted, according to which the authors selected the bibliographical portfolio based on analysis and interpretation.

Findings: From the segment of literature analyzed, it was possible to identify a certain degree of misalignment among what has been pointed out in the literature regarding the management practices in the port sector. Specifically, this discrepancy refers to management practices that are ignored by port managers, which lead to lost opportunities and may even come to jeopardize the organization's performance.

Research limitations/implications: The literature search was restricted to articles written in English, published in indexed scientific journals in the selected databases, according to the following criteria: (i) articles published after the year 2000; (ii) the generation of knowledge based on characteristics selected by the researchers and (iii) the analysis of BP articles based on the judgment and interpretation of the research authors. For future work, it is suggested to expand this research to other databases, in other languages and according to other features, and to continue this investigation by carrying outthe “systemic analysis” and “identifying research opportunities” stages using ProKnow-C.

Originality/value: Although two similar works have been carried out in the same area of research in 2015, the results achieved have contributed to the advancement of research in port performance evaluation, as they have extended the time horizon, the databases used for research and the variables.

Keywords: Performance evaluation, Literature review, Port

Jel Codes: Z

1. Introduction

The port sector is considered to be the main infrastructure for foreign trade, occupying a strategic position in the loading transport system, handling approximately 90% of world trade carried out via international shipping (Puig, Wooldridge & Darbra, 2014). Accordingly, the progress of various countries depends on the implementation of efficient logistical systems that have the ability to adapt to constant changes (Langenus & Dooms, 2015).

This changing environment is marked by the introduction of new organization and storage techniques, because the evolution of the technologies introduced in the systems not only generates the need for new standards, it also expands port system functions (Al-Eraqi, Mustafa & Khader, 2010). Economic instabilities have enhanced the need to reduce costs and increase the level of service quality, which has led to greater complexity of the relevant supply chains (Ursavas, 2014).

This complexity is related to the fact that many stakeholders are involved, each with different power relations, different values and visions, and many interests are affected, such as the different federal government agencies involved, port agents, port operators, shipping companies and the port community as a whole. According to Brooks and Schellinck (2013), in order to deal with this complexity, it is necessary to identify and prioritize strategic actions to seize opportunities for future growth, taking into account the needs of organizations and the characteristics of the environment in which they operate.

In this context, the performance evaluation knowledge area has been used to contribute to different economic sectors (L. Ensslin, Giffhorn, Ensslin, Petri & Vianna, 2010). The literature has highlighted the importance of performance evaluation (PE) in ports, and several works seek to propose tools, performance measures or indicators for use in this context (Chou & Liang, 2001; Chou, 2007; Li & Jiang, 2014; Simões & Marques, 2010; Talley, Ng & Marcillac, 2015). However, these publications are spread throughout various media, which has hindered the reuse of these sources of knowledge and the construction of a solid theoretical framework. The authors of this study thus argue for the need to further consolidate and reflect on the findings in this segment of literature investigating port performance evaluation. This argument is strengthened when we consider the concept/notion of the performance evaluation knowledge area and its practical use, i.e., there is a need to verify the existence of a theoretical alignment with the practical scenario, in order to identify new practical and theoretical opportunities.

Given the centrality of the port sector and the emergence of the identification of ways to manage and leverage the performance of ports, there is a need to know what the scientific community has studied so far on this subject, since the starting point for robust and sound knowledge involves identifying what already exists about the subject in the literature. Therefore, a recompilation is believed necessary of the studies that focus on port performance evaluation. Thus, the following research question emerges: What is the stage of “maturity” of the theory-practice interaction in terms of port performance evaluation? This study thus aims to identify and analyze the characteristics of international scientific research that addresses the segment of literature referring to port performance evaluation, in order to identify the existence of a theoretical alignment between the notion of performance evaluation, as an area of knowledge, and the practical scenario of port performance evaluation. To reach the research objective, the Knowledge Development Process-Constructivist (ProKnow-C) tool was used, a structured process for literature selection and analysis from a constructivist perspective, which served as a guide for researchers in conducting the research.

This study is justified by its importance and viability. Its importance lies in the fact that it reviews the scientific literature on a sector that drives the world economy from the aspect of performance evaluation, as an activity promoting the achievement of the objectives of each port. In this context, the present study highlights the characteristics of this segment of the literature to identify practical and theoretical opportunities arising from the analysis of the theoretical alignment, if any, of the notion of Performance Evaluation as a knowledge area with the practical area of port performance evaluation. It is interesting to note that two studies considering the same body of literature were identified (Dutra, Ripoll-Feliu, Ensslin & Ensslin, 2015) and (S.R. Ensslin, Ensslin, Dutra, Valmorbida, & Cardoso, 2015). Therefore, it should be noted that due to the constructivist philosophy of the selected instrument, and hence the boundaries established by the authors, the results can be viewed as a contribution to the area, since it extends the time horizon, the databases investigated and the variables analyzed. Viability is achieved through access to the data, given that the articles used for the analysis have been searched for and identified on the CAPES journal portal, and is further ensured by the availability of the researchers to collect, review and analyze the publications.

In addition to this introductory chapter, this study consists of four sections:

• •the second section presents the theoretical framework;

• •the third introduces the research methodology;

• •the fourth presents the results; and

• •the fifth addresses the final considerations.

Finally, we present the references that were the basis for this work.

 

2. Theoretical background

This section presents the theoretical basis of this research: the concept of performance evaluation and how the literature has addressed port performance evaluation.

 

2.1. Performance evaluation

The first performance measures were proposed and used in the field of accounting at the end of the thirteenth century (Johnson, 1981). Because they originate from this area, performance measures were limited to financial ones, arising from both cost and management control systems (Nudurupati, Bititci, Kumar & Chan, 2011). This situation remained the same until the industrial revolution (Bititci, Garengo, Dörfler & Nudurupati, 2012; Neely, 1999; Otley, 2001). After World War II, the focus turned to other aspects, such as productivity, quality, time, flexibility and customer satisfaction (Bititci et al., 2012; Neely, 1999; Nudurupati et al.., 2011). Consequently, the process of organizational performance evaluation had to take into account multiple financial and non-financial performance measures.

In this evolution, researchers now point to the importance of the alignment of the performance evaluation process with organizational strategy (Neely, 1999). In this context, the organization’s strategy and objective (or strategies and objectives) begin to guide the selection and definition of performance measures (Neely, Gregory & Platts, 1995). Thus, proposals such as “balanced scorecard(s)” aim to promote this alignment (Otley, 2001). At this time, the area of performance evaluation faces at least two more perspectives that need to be considered:

• •the presence of and need to incorporate “stakeholder” objectives; and

• •the need for the management of the process and the results of the organizational performance evaluation.  

Accordingly, Melnyk, Bititci, Platts and Tobias (2014, pp. 175) argue that a Performance Evaluation system must be made up by two components: the “performance measurement” and the “performance management” systems. The steps consist of identifying performance measures, setting goals, identifying the importance of measures to achieve the organizational objective(s), identifying the organizational status quo and identifying the overall performance that is part of the performance measurement system. These are then followed by the steps of analyzing the differences between the actual and desired results, identifying “discrepancies” in critical aspects, suggesting/implementing and monitoring improvement actions for these situations, and monitoring the organization system and evolution in order to verify the adjustments needed in the organizational performance evaluation system and to promote these adjustments as part of the performance management system. Thus, in general terms, the measurement alone is not enough to run a business. There is a need to complement it with a performance management system.

From this perspective, it is argued that the concept formulated by Dutra et al. (2015, pp. 246) can encapsulate measurement and management components, adding the need to incorporate stakeholder objectives into the constructivist philosophy. Namely, this would consist of the conception of performance evaluation as “the process to build knowledge in decision-making in a specific context that should be analyzed through activities that identify, organize and ordinally and cardinally measure the objectives that allow the decision maker to identify the consequences of actions to improve performance”.

From these notions, two central features are evident in the development and use of a performance evaluation system:

• •it is a communication process that ensures the understanding of the proposed indicators among all stakeholders; and

• •it provides organizational learning (Choong, 2014; Franco-Santos et al., 2007).

Based on the above, we turn our attention towards the literature that addresses port performance evaluation in order to identify its current stage of development and thus its alignment with the stage of the source concept.

 

2.2. Port performance evaluation

Most of the existing research on the port sector referring to performance evaluation focuses on the evaluation of locations and the productivity analysis of container terminals; other works seek to identify criteria and measures to evaluate the overall efficiency of ports (Bergantino, Musso & Porcelli, 2013; Bichou, 2007; Brooks, 2006; Li & Jiang, 2014).

In addition to these perspectives, the literature on performance evaluation also includes relevant studies dealing with efficiency, using the data envelopment analysis (DEA) technique and attempting to identify relationships among multiple inputs and outputs (Langenus & Dooms, 2015). Examples of these studies include works by Al-Eraqi, Mustafa, Khader and Barros (2008), Al-Eraqi et al. (2010), Bergantino et al. (2013), Birgu and Akten (2005), Lin and Tseng (2007), Simões and Marques (2010), Talley et al. (2015), Wu, Liang and Song (2010) and Wu, Yan and Liu (2010).

The selected studies reveal acceptance on the part of port managers to use measurement techniques in order to achieve certain organizational goals. However, their tendencies may restrict the evaluation and stakeholder spectrum (Garcia-Morales, Baquerizo & Angel Losada, 2015). At the same time, by doing so, they also limit the actions of the performance evaluation system.

Accordingly, Madeira Junior, Cardoso Junior, Belderain, Correia and Schwanz (2012) argue that in order to evaluate the performance of a port, considering its criteria/objectives/relevant aspects so that its management is made possible, it is necessary to create an evaluation system that includes as many performance measures as necessary to reflect and manage the port performance, regardless of whether these arise from objective aspects or subjective ones. Brooks (2006) calls attention to the careful selection and implementation of measures or performance indicators, in the sense that they should:

• •be linked to the organization’s strategy;

• •focus on critical measures of performance; and

• •emphasize the performance evaluation (PE) process.

The creation of this type of PE system was configured as a complex task, since the stakeholders (decision makers/managers and interveners) have to overcome numerous challenges and barriers that go beyond merely technical issues, investments and service quality. The port structure consists of different levels of administration, in which different stakeholders work. Thus, the intensity of relations between the parties may lead to conflicts, making the management and implementation of performance measures more difficult (Bergantino et al., 2013; Langenus & Dooms, 2015). In addition, the magnitude of the system is considered in terms of the hierarchical levels present at a port, on an operational, tactical or strategic level or on all three (global), where the relationships between these levels and the various dimensions must also be considered (Langenus & Dooms, 2015).

Based on the selected articles, a “theoretical” alignment seems evident with the performance evaluation literature. It is now necessary to check for the presence of this alignment in practical studies.

 

3. Research methodology

This section presents the methodological framework and the description of the instrument selected to achieve the research objective.

 

3.1. Methodological framework

With regard to the approach to the problem, this study is construed as qualitative research, since the analysis and discussion of the characteristics of the studied segment of literature on port performance evaluation were carried out according to the authors' own judgment and interpretation of this research (Creswell, 2010).

According to Creswell (2010), research with a qualitative approach needs to make use of some sort of validity strategies. In this study, to verify the accuracy of the results, in the sub-steps of the bibliographical portfolio (BP) selection and bibliometric analysis stages, “member checking” (pp. 226) and “external auditor” (pp. 227) techniques were used. The authors used Excel and PowerPoint tools to document the research results in each step of the process.

As for the strategy adopted, this study makes use of the action research method (Creswell, 2010), since the search for articles that generated the bibliographical portfolio (BP) and then allowed a better understanding of the port performance evaluation research segment emerged from the researchers' own interpretation and analysis.

 

3.2. The knowledge development process–The constructivist (ProKnow-C) intervention instrument

Developed by researchers at the Federal University of Santa Catarina (UFSC) LabMCDA (Laboratory of Multicriteria Methodologies for Decision-making Assistance), ProKnow-C is configured as an important and structured process for literature selection and analysis by researchers from a constructivist perspective. In recent years, several studies have been published in different areas of knowledge that have used ProKnow-Cto:

• •identify a segment of the relevant literature on the topic of interest to the researcher (bibliographical portfolio);

• •determine the peculiarities of the area of study;

• •carry out a critical analysis of this BP based on different theoretical perspectives selected by the researcher; and

• •suggest gaps in the literature that support the design of future works (Dutra et al., 2015; L. Ensslin, Dutra, Ensslin, Chaves & Dezem, 2015; Tasca, Ensslin, Ensslin & Bernardete, 2010; Valmorbida & Ensslin, 2016). Figure 1 shows the four steps of the ProKnow-C process.

 

Figure 1. ProKnow-C Stages (Extracted from Valmorbida and Ensslin (2015, pp. 7). Adapted from Ensslin et al., 2010)

 

To achieve the objective of this research, steps 1 and 2, the selection of the bibliographic portfolio and bibliometric analysis, respectively, will be operationalized.

 

3.2.1. Data collection procedures

To achieve the objective of this research, a bibliographical portfolio selection is necessary, which is the first stage of Proknow-C. The survey was conducted in March 2016, consulting seven different databases: Science Direct, Isi Web of Knowledge, Scopus, Wiley, ProQuest, Engineering Village and EBSCO. The keywords used for this research were: Improve Performance, Decision Making, Measurement Development, Performance Appraisal, Performance Measurement, Performance Management, Performance Evaluation, Performance Assessment, Performance Indicator, Port, Harbor and Seaport.

The survey was conducted respecting the structure of each of the databases. Certain delimitations were defined:

• •scientific papers published from 2000 to 2016;

• •research in specific fields: title (article title), abstract and keywords;

• •type of publication (journal article);

• •articles written in English; and

• •at least one scientific paper in the search results.

Figure 2 shows the operationalization of the data collection stage.

From the 3,112 raw articles obtained from the seven databases, Scopus provided the most results, with 1,028 articles, which represented 33% of the sample. Verification of these publications was conducted on April 1, 2016.

 

Figure 2. Selection Process of the Bibliographical Portfolio (Research Authors, 2016)

 

3.2.2. Data analysis procedures

After BP selection, the second stage of Proknow-C (Bibliometry) was implemented, consisting of the analysis and interpretation of the data identified in 37 articles. The bibliometric analysis carried outi n this work aims to generate knowledge for researchers regarding the characteristics of the publications contained in the BP and its references, as well as determining how to find information relevant to the topic (Dutra et al., 2015; S.R. Ensslin et al., 2015; Valmorbida & Ensslin, 2016).

The characteristics studied by Proknow-C are:

• •who was/were the author(s) with the most prolific career(s) in this area of knowledge;

• •which journals had the most publications on the topic;

• •what was the most commonly used methodology/performance evaluation tool mentioned in the articles;

• •in which countries was the model/study implemented;

• •for which areas were the performance measures in practical studies built; and

• •the presence of components of a performance evaluation system.

 

4. Results

The first characteristic examined was related to the authors found in the bibliographical portfolio (BP), as shown in Figure 3.

 

Figure 3. Featured Authors in the BP (Research Data, 2016)

 

Based on the cross-checking of the authors in the BP (37 articles), of the authors with aligned studies reported in the references of the BP 37 articles, it was found that the author Chien-Chang Chou stands out, with six publications in the BP, and seven in the bibliographical portfolio references. Chou, who holds a Ph.D. in Transport and Navigation Science, is a professor in the Department of Transport and Navigation Science at the National Oceanic University of Taiwan (China). His specializations are in the areas of international trade, shipping management, operations and business expedition and management. All BP articles written by him are works published in mathematics journals.

Mary R. Brooks can also be considered a featured author, since she participates with four publications in the BP. She is an international researcher in the field of transport management, international policy and marketing. She is associated with Dalhousie University in Scotland. In addition to Chou and Brooks, Ji-Feng Ding deserves mention, mainly for the research area in which he operates. He is a professor (Ph.D.) in the Department of Aviation and Maritime Transport Management at the University of Taiwan, and has three articles in the BP. Ding operates in the areas of diffuse MCDM, port management, navigation and logistics management. Based on the information presented, it can be noted that of the three main authors, Ji-Feng Ding deserves special mention, since this author operates in the area of port management and works according to a multicriteria approach for decision making (MCDM), i.e., he is more closely aligned to the theoretical positions presented.

Wayne K. Talley stands out in the BP references with three publications. He is a professor (Ph.D.) in the area of Business and Public Administration at Old Dominion University, USA. Talley is the Logistics Revision and Transportation magazine's editor-in-chief and he operates in the areas of transport, marine and port economy research.

As for the variable of the journals most receptive to this topic of research, it was found that BP publications and BP references are found in a total of 34 journals. Figure 4 shows the most receptive journal.

 

Figure 4. Cross-checking of Journals (Research Data, 2016)

 

Based on the cross-checking of the journals in which articles and references were published, it is clear that Maritime Policy & Management is featured with six publications in the BP and five in the BP references. Another journal that deserves attention, although not featured in the BP references, is the International Journal of Maritime Economics, which has three publications in the BP.

After analyzing the leading journals, different methodologies and approaches for performance evaluation were identified in the articles that make up the portfolio, as shown in Figure 5.

 

Figure 5. Tools/Approaches Used for Performance Evaluation (Research Data, 2016)

 

Various tools/approaches were used to evaluate the performance of organizations. The analysis of the methodologies presented in the articles evidence that DEA is the most commonly used tool and evaluation. From the 37 articles in the BP, nine use this methodology. DEA uses linear programming to measure the efficiency of multiple units of decision that are characterized as an input and output production structure. Although this result has already been indicated in the theoretical framework, it is important to mention the need for studies on ports in connection with the performance evaluation system.

In addition to DEA, the “fuzzy” approach known as Multicriteria Method for Decision-Making (MCDM) was another evaluation tool highlighted, appearing in seven publications. This methodology is characterized as a model for multicriteria performance evaluation used for decision-making. It should be noted that although the fuzzy method contemplates multiple objectives, as does DEA, it does not take into account:

• •the presence and need to incorporate the stakeholders' objectives and

• •the need for process management and results from the organizational performance evaluation, as well as the two central functions of the development and implementation of a performance evaluation system (communication and organizational learning process), as indicated by Choong (2014) and Franco-Santos et al. (2007).    

The fourth variable analyzed the countries in which the model/study was developed. The results are presented in Figure 6.

 

Figure 6. Countries in which the Model/Study was implemented (Research Data, 2016)

 

The country that had the largest number of studies and developed models was China, which stood out with 13 occurrences. It is worth pointing out that the same publication presented several subjects of study, i.e., multiple ports indifferent countries, which is why the same country appears a large number of times. Another important fact is that the author highlighted in the BP cross-checking and in the BP references, as well as the author with the largest number of publications in the bibliographical portfolio, is Chinese. In recent years, the Chinese economy has undergone significant development, which has further boosted the growth of its regional ports (Li & Jiang, 2014). This explains why China is the country with the largest number of applied works.

Another important consideration is the fact that, of the 10 largest ports in the world, seven are Chinese, including the top four, which are the ports of Shanghai, Singapore, Guangzhou and Hong Kong. These ports were those most often used as study subjects, as can be seen in some studies (Chou, Gou, Tsai, Tsou, Wong & Yu, 2010; Chou, 2010; Huang, Teng, Huang & Kou, 2003; Lin & Tseng, 2007; Murty, Liu, Wan & Linn, 2005; Pak, Thai & Yeo, 2015; Teng, Huang & Huang, 2004; Wu, Yan & Liu, 2009).

A fifth variable was analyzed to identify the area for which performance measures were constructed in practical studies, i.e., which areas receive the most attention from both port managers and researchers. Table 1 shows the survey of this variable.

 

Table 1. Areas for which performance measurements were constructed in practical studies

As can be seen in Table 1, the performance measures show that most practical studies were focused on logistical, strategic and financial areas, especially on the former, which accounted for 85% of the publications. Logistical variables mainly focused on studies that make use of the DEA tool.

Analyzing the areas, the absence of performance measures to subsidize the port strategic level becomes evident (they are only used in 29% of the analyzed articles). This circumstance is contrary to theoretical positions. The studies focused on the strategic area made use of the following performance measurements: the existence of a future development plan (Chou, 2007; Chou et al., 2010; Chou, 2010; Chou & Ding, 2013), planning ability, market analysis (Chou & Liang, 2001), investment plans, political/social and economic stability (Teng, Huang & Huang, 2004), the ability to develop services tailored to meet the needs of the different market segments (Schellinck & Brooks, 2014), and the continuous improvement of the management and political process, in order to reduce bureaucracy (Pak et al., 2015).

Finally, to verify the existence of a theoretical alignment of the concept of performance evaluation, as a field of knowledge, with the practical area of port performance evaluation, we turned to the notion proposed by Melnyk et al. (2014) regarding the composition of a performance evaluation system in order to verify the presence of its two components. Of the 27 articles analyzed (practical studies that included a model/practice tool), 26 were identified that had only the measurement component, subsequently classified as a performance measurement system; the only study that incorporated the management component was authored by Madeira Junior et al. (2012), and is thus the only study in the bibliographic portfolio that uses a performance evaluation system aligned with the theoretical concept. Thus, 96% of the analyzed articles are not concerned with management, and are limited to measuring port performance through a set of performance measures.

 

5. Final considerations

Given the centrality of the port sector and the emerging identification of ways to manage and leverage the performance of ports, the authors of this research, in order to contribute to the advancement of this area of knowledge, argue that the starting point lies in the knowledge gained from the identification of what is available in the literature on the subject. The aim of this study was to identify and analyze the characteristics of international scientific research that forms part of the literature segment referring to port performance evaluation in order to identify the existence of a theoretical alignment between the concept of performance evaluation as a knowledge area and current practice in terms of port performance evaluation. To meet this objective, the Knowledge Development Process-Constructivist (ProKnow-C) instrument was used to determine the selection of the bibliographical portfolio (BP) and the bibliometric analysis, thus aligning a constructivist view with the objective of this research.

The selection of the bibliographical portfolio found in the Science Direct, Isi Web of Knowledge, Scopus, Wiley, ProQuest, EBSCO and Engineering Village databases, with time delimitation to between the years of 2000 and 2016, resulted in 37 articles. The analysis of these articles showed that:

• •The researchers Chang Chien-Chou, Mary R. Brooks and Jai-Feng Ding stand out. Although Ji-Feng Ding does not have the largest number of articles in the BP, the authors consider him to be prominent in the field and indicate that his publications in other databases are worthy of analysis in the area of port management, as well as his works using the multicriteria approach for decision making (MCDM), which is more closely aligned with the presented theoretical positions.

• •The journal Maritime Policy & Management stands out, with six publications in the BP and five in the BP references, and especially for having published articles on various topics of concern to the maritime world, addressing the latest findings and analysis in terms of port management, the community, transportation companies and board levels.  

• •Of the 37 articles in the BP, nine use DEA methodology. Although this result was expected, due to the presentation of the theoretical framework on port performance evaluation, the authors of this study indicate the existence of distinct stages of maturity between the theoretical concept of performance evaluation and port performance evaluation itself, observing that by using DEA or other tools supported by realistic or prescriptive approaches (Roy, 1993), the studies investigating the ports do not use performance evaluation systems, as proposed by Melnyk et al. (2014).

• •China was the country with the largest number of studies and developed models, with 13 occurrences in the BP. Perhaps this is due to the fact that, of the 10 largest ports in the world, seven are Chinese.

• •The area of logistics was the most commonly studied and within it, performance measurements were most often identified and selected in practical studies. This circumstance is contrary to the theoretical positions of performance evaluation, which indicate that the strategic area is worthy of attention in the contemporary context.

• •A single study made use of a “performance evaluation system”, as proposed by Melnyk et al. (2014), by incorporating the management component, thus aligning it with the theoretical concept.

• •In general, based on these conclusions and findings, ignoring the practices of performance management activity seems like a missed opportunity to explore their contributions. It may even compromise the performance of the organization, as port managers fail to analyze the differences between the actual and desired results, identify “discrepancies” in critical aspects, suggest, implement and monitor improvement actions for these situations, and monitor the system and the progress of the port in order to verify the need for adjustments in the organizational performance evaluation system and then promote these adjustments. Additionally, the use of the concept formulated by Dutra et al. (2015) was not seen in any of the studies, nor was the process and the result of using performance evaluation in their core functions: communication and the generation of organizational learning (Choong, 2014; Franco-Santos et al., 2007). In another aspect, the opportunity arises for theorists and researchers to direct their efforts towards investigating the benefits of using a performance evaluation system that addresses measurement and management components. Thus, the knowledge area of port performance evaluation appears to be a fertile field for exploration.

As for the selected methodological instrument (Proknow-C), the authors argue that its vision and structured process led them to deeper reflection during the course of the two stages carried out, generating knowledge on the segment of literature pertaining to port performance evaluation.

This work has certain limitations:

• •the literature search was restricted to articles written in English, published in indexed scientific journals in the selected databases and available free-of-charge over the Internet;

• •the time limit restriction on the articles required them to be published after the year 2000;

• •the generation of knowledge based on the characteristics selected by the researchers: authors, journals, the performance evaluation tool used, the subject of study (limited to countries through PE systems) and study areas where indicators were used; and

• •the analysis of BP articles regarding the investigated variables was determined by the judgment and interpretation of the study authors.

 For future works, it is suggested to:

• •expand this research to other databases, other languages and other features, and to extend the time horizon; and

• •to further this investigation by conducting the “systemic analysis” and “identification of research opportunities” stages in ProKnow-C, based on the establishment of the theoretical affiliation of researchers.

 

References

Al-Eraqi, A.S., Mustafa, A., & Khader, A.T. (2010).  An extended DEA windows analysis: Middle East and East African seaports. Journal of Economic Studies, 37(2), 208-218. https://doi.org/10.1108/01443581011043591

Al-Eraqi, A.S., Mustafa, A., Khader, A.T., & Barros, C.P. (2008). Efficiency of Middle Eastern and East African seaports: Application of DEA using window analysis. European journal of scientific research, 23(4), 597-612.

Bergantino, A.S., Musso, E., & Porcelli, F. (2013). Port management performance and contextual variables: Which relationship? Methodological and empirical issues. Research in Transportation Business & Management, 8, 39-49. https://doi.org/10.1016/j.rtbm.2013.07.002

Bichou, K. (2007). Chapter 24 Review of Port Performance Approaches and a Supply Chain Framework to Port Performance Benchmarking. Research in Transportation Economics, 17, 567-598. https://doi.org/10.1016/S0739-8859(06)17024-9

Birgun, S., & Akten, N. (2005). Relative efficiencies of seaport container terminals: A DEA perspective. International Journal of Integrated Supply Management, 1(4), 442-456. https://doi.org/10.1504/IJISM.2005.006305

Bititci, U., Garengo P., Dörfler, V., & Nudurupati, S. (2012). Performance measurement: Challenges for tomorrow. International journal of management reviews,14, 305-327. https://doi.org/10.1111/j.1468-2370.2011.00318.x

Brooks, M.R. (2006). Chapter 25 Issues in Measuring Port Devolution Program Performance: A Managerial Perspective. Research in Transportation Economics, 17, 599-629. https://doi.org/10.1016/S0739-8859(06)17025-0

Brooks, M.R., & Schellinck, T. (2013).  Measuring port effectiveness in user service delivery: What really determines users' evaluations of port service delivery? Research in Transportation Business & Management, 8, 87-96.  https://doi.org/10.1016/j.rtbm.2013.04.001

Choong, K.K. (2014). Has this large number of performance measurement publications contributed to its better understanding? A systematic review for research and applications. International Journal of Production Research, 52(14), 4174-4197. https://doi.org/10.1080/00207543.2013.866285

Chou, C.C. (2007). A fuzzy MCDM method for solving marine transshipment container port selection problems. Applied Mathematics and Computation, 186(1), 435-444. https://doi.org/10.1016/j.amc.2006.07.125

Chou, C.C. (2010). Application of FMCDM model to selecting the hub location in the marine transportation: A case study in southeastern Asia. Mathematical and Computer Modelling, 51(5-6), 791‑801. https://doi.org/10.1016/j.mcm.2009.10.015

Chou, C.C., & Ding, J.F.  (2013). Application of an integrated model with MCDM and IPA to evaluate the service quality of transshipment port. Mathematical Problems in Engineering, 13(Special issue), 1-7. https://doi.org/10.1155/2013/656757

Chou, C.C., Gou, R-H., Tsai, C-L., Tsou, M-Ch., Wong, C-P., & Yu, H-L. (2010). Application of a mixed fuzzy decision making and optimization programming model to the empty container allocation. Applied Soft Computing, 10(4), 1071-1079. https://doi.org/10.1016/j.asoc.2010.05.008

Chou, T.Y., & Liang, G.S. (2001). Application of a fuzzy multi-criteria decision-making model for shipping company performance evaluation. Maritime Policy & Management, 28(4), 375-392. https://doi.org/10.1080/03088830110049951

Creswell, J.W. (2010). Projeto de Pesquisa: Métodos Qualitativos, Quantitativos e Mistos (pp. 296). Tradução: Magda Lopes. Porto Alegre: Artmed.

Ding, J.F., & Chou. C.C (2011). A fuzzy MCDM model of service performance for container ports. Scientific Research and Essays, 6(3), 559-566.

Ding, J.F., & Chou, C.C. (2013). An evaluation model of quantitative and qualitative fuzzy multi-criteria decision-making approach for location selection of transshipment ports. Mathematical Problems in Engineering, 13(Special issue), 1-12. https://doi.org/10.1155/2013/783105

Dutra, A., Ripoll-Feliu, V.M., Fillol, A.G., Ensslin, S.R., & Ensslin, L. (2015). The construction of knowledge from the scientific literature about the theme seaport performance evaluation. International Journal of Productivity and Performance Management, 64(2), 243-269. https://doi.org/10.1108/IJPPM-01-2014-0015

Ensslin, L., Dutra, A, Ensslin, S. R., Chaves, L. C., & Dezem, V. (2015). Research Process for Selecting a Theoretical Framework and Bibliometric Analysis of a Theme: Illustration for the Management of Customer Service in a Bank. Modern Economy, 6(6), 782. https://doi.org/10.4236/me.2015.66074

Ensslin, L., Giffhorn, E., Ensslin, S.R., Petri, S.M., & Vianna, W.B. (2010). Avaliação do Desempenho de Empresas Terceirizadas com o Uso da Metodologia Multicritério de Apoio à Decisão-Construtivista. Revista Pesquisa Operacional, 30(1), 125-152. https://doi.org/10.1590/s0101-74382010000100007

Ensslin, S.R., Ensslin, L., Dutra, A., Valmorbida, S.I., & Cardoso, T.L. (2015). Avaliação Multicritério de Desempenho de Portos Marítimos Brasileiros: Estudo de Caso para Apoiar à Gestão do Porto de São Francisco do Sul. In: Congresso Internacional de Desempenho Portuário (CIDESPORT) Florianópolis/SC.

Franco-Santos, M., Kennerley, M., Micheli, P., Martinez, V., Mason, S., Marr, B., et al. (2007). Towards a Definition of a Business Performance Measurement System. International Journal of Operations and Production Management, 27(8), 784-801. https://doi.org/10.1108/01443570710763778

Garcia-Morales, R.M., Baquerizo, A., & Angel Losada, M. (2015). Port management and multiple-criteria decision making under uncertainty. Ocean Engineering, 104, 31-39. https://doi.org/10.1016/j.oceaneng.2015.05.007

Huang, W-C., Teng, J-Y., Huang, M-J., & Kou, M-S. (2003). Port competitiveness evaluation by fuzzy multicriteria grade classification model. Journal of Marine Science and Technology, 11(1), 53-60.

John, A., Yang, Z., Riahi, R., & Wang, J. (2014). Application of a collaborative modelling and strategic fuzzy decision support system for selecting appropriate resilience strategies for seaport operations. Journal of Traffic and Transportation Engineering, 1(3), 159-179. https://doi.org/10.1016/s2095-7564(15)30101-x

Johnson, H.T. (1981). Toward a New Understanding of Nineteenth-Century Cost Accounting. The Accounting Review, 56(3), 510-518.

Langenus, M., & Dooms. M. (2015). Port industry performance management: A meso-level gap in literature and practice?. International Journal of Logistics-Research and Applications, 18(3), 251-275. https://doi.org/10.1080/13675567.2015.1026885

Li, J., & Jiang, B. (2014). Cooperation Performance Evaluation between Seaport and Dry Port; Case of Qingdao Port and Xi’an Port*. International Journal of e-Navigation and Maritime Economy, 1, 99-109. https://doi.org/10.1016/j.enavi.2014.12.009

Lin, L.C., & Tseng C.C. (2007). Operational performance evaluation of major container ports in the Asia-Pacific region. Maritime Policy & Management, 34(6), 535-551. https://doi.org/10.1080/03088830701695248

Madeira Junior, A.G., Cardoso Junior, M.M., Belderain, M.C.N., Correia, A.R., & Schwanz, S.H. (2012). Multicriteria and multivariate analysis for port performance evaluation. International Journal of Production Economics, 140(1), 450-456. https://doi.org/10.1016/j.ijpe.2012.06.028

Melnyk, S.A., Bititci, U., Platts, K., Tobias, J., & Anderson, B. (2014). Is performance measurement and management fit for the future?. Management Accouting Research, 25(2), 173-186. https://doi.org/10.1016/j.mar.2013.07.007

Murty, K.G., Liu, J.Y., Wan, Y.W., & Linn, R. (2005). A decision support system for operations in a container terminal. Decision Support Systems, 39(3), 309-332. https://doi.org/10.1016/j.dss.2003.11.002

Neely, A. (1999). The Performance Measurement Revolution: Why Now and What Next?. International Journal of Operations and Production Management, 19, 205-228. https://doi.org/10.1108/01443579910247437

Neely, A., Gregory, M., & Platts, K. (1995). Performance Measurement System Design – A Literature Review and Research Agenda. International Journal of Operations and Production Management, 15, 80-116. https://doi.org/10.1108/01443579510083622

Nudurupati, S.S., Bititci, U.S., Kumar, V., & Chan, F.T.S. (2011). State of the art literature review on performance measurement. Computers & Industrial Engineering, 60(2):  279-290. https://doi.org/10.1016/j.cie.2010.11.010

Otley, D. (2001). Extending the boundaries of management accounting research: Developing systems for performance management. British Accounting Review, 33, 243-261. https://doi.org/10.1006/bare.2001.0168

Pak, J.Y., Thai, V.V., & Yeo, G.T. (2015). Fuzzy MCDM Approach for Evaluating Intangible Resources Affecting Port Service Quality. The Asian Journal of Shipping and Logistics, 31(4), 459-468. https://doi.org/10.1016/j.ajsl.2016.01.004

Puig, M., Wooldridge, C., & Darbra, R.M. (2014). Identification and selection of Environmental Performance Indicators for sustainable port development. Marine Pollution Bulletin, 81(1), 124-130. https://doi.org/10.1016/j.marpolbul.2014.02.006

Radonjić, A., Pjevčević, D., Hrle., Z & Čolić, V. (2011). Application of DEA method to intermodal container transport. Transport, 26(3), 233-239. https://doi.org/10.3846/16484142.2011.622127

Roy, B. (1993). Decision science or decision-aid science. European Journal of Operational Research, 66(2), 184-203. https://doi.org/10.1016/0377-2217(93)90312-B

Schellinck, T., & Brooks, M.R. (2014). Improving port effectiveness through determinance/performance gap analysis. Maritime Policy and Management, 41(4), 328-345. https://doi.org/10.1080/03088839.2013.809632

Simões, P., & Marques, R.C. (2010). Seaport performance analysis using robust non-parametric efficiency estimators. Transportation Planning and Technology, 33(5), 435-451. https://doi.org/10.1080/03081060.2010.502375

Talley, W.K., Ng, M., & Marsillac, E. (2015). Port service chains and port performance evaluation. Transportation Research Part E: Logistics and Transportation Review, 69, 236-247. https://doi.org/10.1016/j.tre.2014.05.008

Tasca, J., Ensslin, L., Ensslin, S.R., & Bernardete, M.M.A. (2010). Na approach for selecting a theoretical framework for the evaluation of training programs. Journal of European Industrial Training, 34(7), 631-655. https://doi.org/10.1108/03090591011070761

Teng, J-Y., Huang, W-C., & Huang, M-J. (2004). Multicriteria evaluation for port competitiveness of eight East Asian container ports. Journal of Marine Science and Technology, 12(4), 256-264.

Ursavas, E. (2014). A decision support system for quayside operations in a container terminal. Decision Support Systems, 59, 312-324. https://doi.org/10.1016/j.dss.2014.01.003

Valmorbida, S., & Ensslin, L. (2016). Construção de conhecimento sobre avaliação de desempenho para gestão organizacional: Uma investigação nas pesquisas científicas internacionais. Revista Contemporânea de Contabilidade, 13(28), 123-148.

Valmorbida, S.M.I., & Ensslin, S.R. (2015). Avaliação de Desempenho de Rankings Universitários: Revisão da Literatura e diretrizes para futuras investigações. In: Anais do XXXIX Encontro da ANPAD (EnANPAD 2015) Belo Horizonte MG.

Wu, J., Liang, L., & Song, M. (2010). Performance based clustering for benchmarking of container ports: An application of DEA and cluster analysis technique. International Journal of Computational Intelligence Systems, 3(6), 709-722. https://doi.org/10.1080/18756891.2010.9727734

Wu, J, Yan, H., & Liu, J. (2009). Groups in DEA based cross-evaluation: An application to Asian container ports. Maritime Policy & Management, 36(6), 545-558. https://doi.org/10.1080/03088830903346095

Wu, J, Yan, H., & Liu, J. (2010). DEA models for identifying sensitive performance measures in container port evaluation. Maritime Economics & Logistics, 12(3), 215-236. https://doi.org/10.1057/mel.2010.6

---