Miloš Milosavljević[1] - Nemanja Milanović[2] - Slađana Benković[3]: Public Procurement Management and Contract Efficiency: Empirical Evidence from Serbian Local Administration (Annales, 2017., 123-140. o.)

https://doi.org/10.56749/annales.elteajk.2017.lvi.8.123

Abstract

Public procurements in Serbia account for one tenth of the country's Gross Domestic Product. Accordingly, control, monitoring and regulation of public procurements are crucial factors in and constituents of efficient governance. Ever since the introduction of the Law on Public Procurements in 2002, Serbia has been reforming the system of public procurements. The establishment of an independent regulatory institution was aimed at facilitating governance, educating clerks and managers, supporting transparency and competitiveness, and decreasing corruptive behaviour in public procurements. Hitherto, however, the adversarial goals of public procurement management and contract management have been out of the scope of both scholars and practitioners. This study aims to examine the influence of public procurement management on contract management practices. Study results indicate that procurement planning and solicitation have the most important influence on the efficiency of contract management. The paper draws attention to the importance of public procurement and contract management for the efficiency of local administration in Serbia.

Keywords: public procurement, contractual management, local administration, Serbia

I. Introduction

Public procurement relates to all purchases made by public authorities. It is centred around the issue of how authorities spend taxpayers' money on goods, services and

- 123/124 -

works.[1] As such, it is one of the key economic activities of governments[2] and an important tool for the effective redistribution of national wealth. The procurement function in contracting authorities expanded from a simple acquisition of office supplies to contracting a broad range of public functions a long time ago.[3] For instance, municipalities and cities nowadays contract out a wide range of services, ranging from education and healthcare to transportation and domestic and commercial waste collection.[4] The shift to market delivery of goods, services and works was conceived as a means of promoting efficiency and managerialism in the public sector.[5]

Proponents of contracting out goods, services and works to private delivery argue that competition incentives yield efficiency and bypass costly labour and supply. The most important reasons for private delivery are related to fiscal stress and cost reduction, whereas political pressure and ideological considerations of policy-makers are not found to play an important role in the field.[6] Contrary to this, Boyne[7] states that the empirical arguments for cost-reduction and the efficiency of contracting out services are bureaucratic myths rather than reality, as a result of numerous methodological flaws in the extant studies. Using a meta-regression analysis Bel, Fageda and Warnerd[8] infer that contracting out services to private providers does not lead to any efficiency improvements; moreover, when the efficiency of service delivery has already been enhanced, the opposition to contracting out is more likely.[9]

Given their high importance for a myriad of different economic, societal, political and technological dimensions, public procurements have been heavily regulated. In an economic sense, they account from 10 to 25 percent of total GDP

- 124/125 -

across Europe. In addition, the public purchasing system administers public money. Therefore, all countries face the problem of unrelenting budget constraints and concerns about efficiency.[10]

Public procurements not only uphold the efficiency, but also a broader set of strategic goals. To mention a few, public procurements are used to spur innovation and development[11]), implement different national, regional and local sustainability policies,[12] influence political and fiscal decentralisation and foster local economies[13] and to improve the rational usage of natural resources and achievement of social outcomes.[14]

On the other side, the goal of contract management and administration is to 'ensure that proper mechanisms are in place to monitor and evaluate contractors, suppliers and service providers' performance in the fulfilment of their contractual obligations'.[15] Effective contract management requires mitigating the risks and problems that could plague contractual processes, and requires contract-management capabilities for dealing with possible problems.[16] It is argued that highly specialised expertise is needed for the optimal monitoring arrangements that allow the quality and quantity of goods and services delivered to be compared against the contract specifications.[17] The main theoretical propositions in the current body of knowledge suggests that public

- 125/126 -

contractors are poor-quality agents in procurement contracting game, which leads to high inefficiency on the public sector side.[18]

Although the distinctions between the main goals of public procurements and contract management are obvious, the relationship between these two driving forces of efficient public administration have received a paucity of attention in recent scholarly studies. For instance, Decarolis[19] examined how ex post renegotiations severely affect the lowest price bid and distorts the performance of public procurements, while, Davison and Sebastian[20] reported on the most common issues in contract administration as affected by contract type. These, and other similar studies only tangentially address the relationship between public procurement efficiency and contract administration and management. Accordingly, this study aims to fill the gap in the present body of knowledge by examining the influence of public procurement management on contract management practices.

In the context of this study, public procurements are viewed as a strategic function in procuring organizations, rather than a clerical function used for gatekeeping purposes.[21] As such, public procurement processes include not only the purchasing of goods, services and works, but planning these activities, solicitation, implementation and monitoring as well.

The remainder of this paper is organised as follows: Section 2 depicts the geographical context of the study by delineating the Serbian setting of public procurements. The same section outlines the theoretical framework for public procurement and its importance, reviews the literature on the relationship between public procurement management and contract management efficiency and develops the main research questions. Section 3 thoroughly explains the methodology used in the study with particular emphasis on the development of measures, indicators and scales used for the analysis. Section 4 presents the results of the study. Section 5 discusses the results and provides an insight into the strengths, limitations, and implications of the study for various stakeholders.

- 126/127 -

II. Literature review

1. A case for public procurements in Serbia

The regulated practice of public procurement in Serbia was established in 1875, when the Law on Military Procurement and Auctions was passed to ensure more effective spending of public funds. Certain features of this law are incorporated in the current public procurement system, such as setting limits above which a public auction must be organized. Below these limits, direct agreement is allowed.

History aside, a modern legal framework of the Serbian public procurement system was introduced with the Law on Public Procurement (LPP) from 2002.[22] The LPP incorporated exemplary EU contracting directives in procedures for the award of public service contracts,[23] public supply contracts[24] and public works contracts,[25] and the procurement procedures for entities operating in the water, energy, transport and telecommunications sectors.[26] The LPP underwent the most significant change in 2004, when the Republic Commission for Protection of Rights in Public Procurement Procedures was established within the Public Procurement Office.

For a decade and a half, Serbia has been undergoing reforms aiming to achieve good governance in public procurements. Persistent modifications of the law and harmonisation with EU legislation are tackling public procurements, which currently account for one tenth of Serbia's Gross Domestic product. A set of legal and institutional prerequisites was set in 2002, and the system has been gradually improving ever since. In 2004, European Union bodies adopted two new directives on public procurement: Directive 2004/18/EC on procedures for the award of public works contracts, public supply contracts and public service contracts, and Directive 2004/17/EC on procurement procedures of entities operating in the water, energy, transport and postal services sectors. This change strongly influenced the harmonisation of the legislative framework for the public procurement system in Serbia with EU directives. The new LPP was therefore adopted[27] in December 2008, being in effect until April 2013, when it was replaced by the current LPP.[28]

The current Law represents a substantial shift in harmonising Serbian with EU public procurement legislation, particularly considering the amendments adopted in

- 127/128 -

EU Directive 2014/24/EU and Directive 2014/25/EU. The negotiation position in the inter-governmental conference on Serbia's EU accession for Chapter 5 - Public procurement was established at the beginning of 2016. The Republic of Serbia opened Chapter 5 on December 13, 2016.[29]

Bearing this in mind, public procurement attracted attention from both practitioners and scholars in Europe, and similarly in Serbia, as well. From a practical point of view, the central authority - the Public Procurement Office - frequently reports on the status of and dynamics in public procurements with occasional reporting on particular projects related to various improvements.[30] The system is also monitored by a few nongovernmental institutions.

From an academic point of view, the body of evidence is still developing. Nevertheless, only a paucity of research has filled the knowledge-base on public procurements in Serbia. Specific issues, such as the benefits of centralising public procurements,[31] transparency in public procurements,[32] distinct features of e-procurements,[33] and methodologies for bidder selection process improvements[34] have been thoroughly examined in the extant literature.

The influence of public procurement management on contractual efficiency has been out of the scope of research radars. As displayed in Table 1, the number of procurement contracts has been gradually decreasing in last fifteen years. However, the value of procurements per contract has increased approximately 7.5 times (from 426,000 in 2003 to 3,212,000 RSD in 2016). Terefore, the importance of managing individual contracts increased immensely throughout the observed period.

- 128/129 -

2. Public procurement contract efficiency

Erridge and Nondi[35] elaborate on the types of public procurements according to the EU regulation and point out three ways in which a contract can be granted to a supplier: (1) open procedure - any supplier may tender; (2) restricted procedure - any supplier may apply to be considered, and the purchaser then selects suppliers to tender; and (3) negotiated procedure - the purchaser conducts direct discussions with one or more suppliers of the purchasers' choice. On the other side, different contracts require different contract management approaches by the contracting authority. Criteria that define the rigour of the contracting authority's contract management are explained by Kraljic's supply matrix.[36]

Goods and services of great importance for the contracting authority are either easily available on the market or offered by a limited number of providers. In the first case, contracting authorities can easily choose another bidder in the event of contractual obligations being violated and use the benefits derived from great volume and value of their public procurements. Even so, there is a risk that the contracting authority cannot purchase a sufficient amount of goods and services during the process of choosing another supplier. In that case contract management should be founded on inventory management and a contract monitoring process.

The significance of contract management is even higher when the contracting authority considers goods and services as exceptionally important, but there are only a few providers. In contrary, the contracting authority should accumulate inventory, which leads to higher inventory costs, and so contract management and market research should contribute to a more efficient public procurement process. Even though procurement contracts differ in terms of their complexity, all legal arrangements require distinctive, holistic and multidimensional contractual skills, knowledge, expertise and experience.[37]

3. Public procurement management and contract efficiency

Public procurement management refers to all activities needed for efficient planning, implementation and monitoring of procurement processes within a contracting authority. More or less all jurisdictions worldwide have similar managerial objectives

- 129/130 -

related to public procurement.[38] Schapper, Veiga Malta and Gilbert[39] pinpoint efficiency and effectiveness as important goals of public procurement policies and emphasise public procurement management as an important tool for achieving these goals.

Appropriate management of procurements requires appropriate tools and techniques. The main procurement management tools and techniques identified in the extant literature encompass (1) procurement planning, (2) solicitation planning, (3) solicitation, (4) source selection, (5) contract administration, and (6) contract closeout.[40] It should be stressed that these key procurement areas (particularly tools and techniques needed for these areas) do not differ from those of private sector contractors. Murray[41] argues that application of any methodology developed for the private sector is not necessarily valid for the public sector party. This is particularly due to the fact that public procurement regulation puts constraints on the contracts and the award mechanisms that private procuring agencies use.[42] However, the tools and techniques previously mentioned are rather generic by nature, and could be implemented for all procurement projects in both the public and private sector.

Following the previous literature review, the study hypothesises that public procurement management processes affect the contractual efficiency of local administrations. The specific hypotheses researched in this study are:

RQ 1: Procurement planning positively affects overall contract management efficiency;

RQ 2: Solicitation planning positively affects overall contract management eiciency;

RQ 3: Solicitation positively affects overall contract management efficiency;

- 130/131 -

RQ 4: Source planning positively affects overall contract management efficiency;

RQ 5: Source selection positively affects overall contract management efficiency;

RQ 6: Contract administration positively affects overall contract management efficiency;

RQ 7: Contract closeout positively affects overall contract management efficiency.

III. Methodology

1. Development of the research instrument

The study used questionnaire as the main research tool. The questionnaire was developed for the purposes of this study, and encompassed three sections. The first section dealt with the demographic features of respondents - the size of the local administration where respondents worked, their age and gender and their current working position.

The second section dealt with the dependent variable. The examinees were asked about their perception of the overall efficiency of contract management in their local administration with a single item inquiry (on a seven-grade Likert-type scale, ranging from completely inefficient to completely efficient).

The third part provided the dependent variables - the efficiency of procurement management tools and techniques. Similar to the dependent variable, a Likert-type scale was used for the assessment of respondents' perception of the efficiency of the tools and techniques used in local administrations. An initial set of questions was created on the basis of tools and techniques described in PMI.[43] After piloting the questionnaire with seven respondents (three with an academic and four with a practical background), the questions were refined in order to ensure their comprehensibility. Public procurement planning was addressed with three items: (1) Efficiency of needs analysis, (2) Compliance of procurement plans with financial and strategic plans, and (3) Efficiency of contract type selection. Performance of solicitation planning was measured with four items: (1) Efficiency of the standard forms for procurement, (2) Efficiency of expert judgments for the assessment of needs and inputs in the process, (3) The quality of internal procedures for public procurements, and (4) The quality of selection and evaluation criteria. Solicitation was covered with three items: (1) Efficiency of advertising procurements, (2) Efficiency of pre-bid conferences, and (3) Efficiency of proposal collections. Source selection was reviewed with four items: (1) Efficiency of contract negotiation, (2) Efficiency of weighting system, (3) Efficiency of

- 131/132 -

screening system, and (4) Efficiency of independent estimates. Contract administration was reviewed with three items: (1) Efficiency of contract change control system, (2) Efficiency of performance reporting, and (3) Efficiency of payment system. Finally, contract closeout was reviewed with two items: (1) Efficiency of verification system, and (2) Efficiency of procurement audits.

2. Sampling procedure

The study used a paper-and-pencil approach. The questionnaire was distributed to civil servants and political appointees (clerks, specialists and managers) in Serbian local administrations (cities and municipalities) in written form. Since the list of all public procurement clerks is unknown (to the best of authors' knowledge there is no compiled list of public procurement administrators), the study was based on a 'snowball' sampling technique.[44] This sampling relies on peer-to-peer recruitment of study participants and formation of a referral chain.[45] The initial group of examinees were graduate students on the Public Procurement Study Programme at the Faculty of Organizational Sciences in Belgrade. The initial group accordingly created a referral chain.

Although it can be a subject of various biases,[46] the referral chain was actively controlled - particularly its initiation, progress and termination. Using the coded questionnaires, the number of referrals was controlled to limit clustering within local administrations. None of the local administrations received a quota higher than 10% of a total sample size. In total, 158 examinees responded to the questionnaire.

3. Data collection and analysis

Data was collected in the period January-April 2017 by a group of trained assistants. Afterwards, the data was entered in SPSS (Statistical Package for Social Sciences). Quantitative data was analysed using descriptive statistics: percentages, means and standard deviations. Interdependence of determinants (independent variables) and contract management efficiency (dependent variable) were determined by correlation (Pearson moments two tailed correlation coefficient analysis) and multiple regression.

- 132/133 -

IV. Results

1. Pre-analysis

The study was conducted among 158 public procurement officers and clerks in 38 municipalities in Serbia. The respondents were evenly distributed when it came to the size of municipality they have been working for.[47] Regarding the gender profile, almost two thirds of replies were from women. Finally, as for the working position of examinees, more than a half of them were operating staff-clerks.

The study further analysed the descriptive statistics. Contract management efficiency in Serbian local administrations is perceived as medium to high, which indicates the relative maturity of processes. As for the public procurement management areas, the highest grade was given to source selection (=5.28/7, n=158), but all the average scores on examinees' perceptions were in the medium to high range. The details of the descriptive statistics are given in Table 3.

Table 3 shows that composite measures for independent variables have high values of internal consistency. All variables excluding Solicitation scored between 0.7 and 0.95 regarding Cronbach's Alpha, which indicates an acceptable unidimensionality of the measured variables.[48]

As displayed in Table 4, the study identified a strong positive relation between contract management efficiency and all examined variables. The highest correlation coefficients were calculated for source selection (r=.685, p<.01) and procurement planning (r=.652, p<.01). Nevertheless, a significant correlation with overall efficiency was found for other variables.

2. Main analysis

Since the study found a strong positive correlation between public procurement management variables and contract management efficiency, the next step was to examine the influence and intensity of variables seen as independent of contract management efficiency (dependent variable). The results of the multiple regression analysis indicated that the research model predicted 49.2% (R[2]=.492) of the variability of contract management efficiency, which is shown in Table 5. As Durbin-Watson was d=1.849 (between the two critical values 1.5<d<2.5), it could be assumed that there is

- 133/134 -

no first-order linear autocorrelation in the multiple linear regression data. Collinearity was further examined with the variance inflation factor, and high VIF was found for the determinant of procurement planning. However, the construct of this variable was kept as such.

The high significance of the F-test (p<.01), indicates the existence of linear interdependence. In this way, the study results indicate that there was a linear relationship between the variables in the model. Beta expresses the relative importance of each independent variable in standardized terms. Only two determinants were found to be significant predictors of contract management efficiency. Accordingly, the study results clearly indicate that appropriate contract management depends on the quality of procurement planning and source selection.

V. Discussion and conclusions

1. Summary of key findings

The aim of this study was to explore the relationship between two conflicting managerial processes - public procurement management and contract management. For this purpose, a specially developed questionnaire was distributed to 158 public procurement clerks, specialists, and managers in local administrations in Serbia. After analysing the study results, it was found that public procurement management activities strongly affect contract management efficiency. In total, nearly a half of the variability of contract management efficiency depends on the efficiency practices of public procurement clerks, specialists, and managers. In particular, some specific public procurement determinants were found to have an important role in predicting the efficiency of contract management.

First, procurement planning was found to be a significant predictor of efficient contract management. Planning as such is advocated as an important element of management in municipalities.[49] It should, however, be emphasised that the concept of procurement planning is based on the perception of the respondents. The results would be more robust if objective descriptions were used for procurement planning processes and practices.[50]

- 134/135 -

Second, the results of the study indicate that source selection plays an important role in the efficient contract management. This result can be explained in two conflicting ways. On one side, careful source planning may lead to partnerships between contracting authorities and bidders. Partnerships are particularly important when public procurement manifests some salient features, such as a recognition that the goal of procurement cannot be reached in traditional ways.[51] Careful planning and closer relations with bidders can sometimes meet the requirements of value for money, accountability and flexible competition[52] and reduce the risk of non-selection.[53] Close relations with bidders and tight purchaser-supplier models are generally more related to private than public sector procurers,[54] although public authorities tend to create winning patterns for suppliers who once won the bid as well.[55] On the other side, it could be speculated that public procurement officers in a contracting authority might incline towards collusive tendering and bid rigging. Such behaviour is hard to detect, as bidders might create quasi-cartel firms for the procurement of goods, services and works to contracting authorities.[56] Given the level of corruption in Serbia,[57] this speculation should be thoroughly considered and analysed in-depth.

2. Contributions and implications

Public procurement is high on the agenda of policy holders, decisions makers, scholars and other interested parties. The findings of this study make several contributions to the body of knowledge related to public procurement. Any thorough discussion of a myriad of public procurement governance issues contributes to better understanding of the ongoing Serbian reforms and harmonisation with EU procedures. The existing literature reports that newcomers to the EU perform worse than their counterparts regarding the

- 135/136 -

efficiency of public procurements,[58] probably due to the fact that anti-corruption efforts in Europe's post-communist states have been less successful than expected.[59] The results of this study depict a self-perceived progress of local administrations in establishing state-of-the-art contractual procedures, plans and managerial mechanisms related to public procurement.

Some lessons are also drawn for public procurement improvements in candidate countries. As there was no real public procurement system in Serbia prior to 2002, most of the legal and institutional infrastructure was 'imported' from the EU legislation as part of the process of harmonisation and accession. The latter values speed and efficiency,[60] and leaves a small space for debates and arguments on real capacities for and capabilities of developing any particular systems change. The findings of this paper confirm that the development of coordinated administration and efficient and uniform goals are time consuming activities, and even the legacy of accession is insufficient when it comes to public administration reforms.

The study also provides several implications for scholars and practitioners' contract management in public administration. First, the study provides empirical evidence for the multiplicity of local government' goals. The main aim of public procurement management is to improve efficiency. However, administrative goals related to contractual administration[61] only partially support fulfilling public procurement goals. The results raise some questions related to the main focus (bureaucratic or managerial efficiency) of local administrations. Another important implication of the study is the development of a new research instrument for the collection of data related to contract administration. Research instruments of this kind are sparse in the current body of knowledge.[62]

- 136/137 -

3. Limitations and further recommendations

One of the main strengths of this paper is its geographical context, in that the study was conducted among local administrations in Serbia. The country was one of the last to establish the system of public procurement and it has been gradually improving and evolving over the last decade and a half. Public procurement studies of this kind are in rather short supply. Even so, this creates certain limitations. As with other studies with a strong national background,[63] the results obtained from this cannot be generalised to other countries and their local administrations. Policy holders, decision makers and researchers should refrain from any generic interpretation of results in other geographical contexts. This, however, offers an avenue for further research. Using the same (or an improved) methodology, the study could be replicated in other regions. Not only would that provide an insight from other countries, but it would create an opportunity for comparative analyses.

The other important strength is the focus on local administrations. Bearing in mind the poor oversight of the system, the study adds to the body of knowledge and harvests the momentum of administrative reforms based on harmonisation and integration with the EU. It should, however, be noted that local administrations account for only a small fraction of total public procurements in Serbia. In 2016, they accounted for around 16% of total public procurements.[64] This questions whether the study findings can even be generalised to public procurement in Serbia. At the same time, it clears the path for additional research. Other public sectors - central administration, justice, health and social protection, education and science, public and municipal enterprises, foundations and charities - should also be examined and explored in follow-up studies.

Finally, the study offers an empirical insight and provides primary data collected via a specially developed questionnaire. Nevertheless, future, more comprehensive study should focus on (1) the inclusion of additional variables and (2) capturing the development of the observed variables. As for the first item, further instruments for data collection on public procurement management should at least include the assessment of inputs and outputs, rather than solely examining managerial tools and techniques. The additional set of variables and more sophisticated research instrument would contribute to the better understanding of the effect of public procurement management to contract

- 137/138 -

efficiency. As for the second item, this study is cross-sectoral and captures only a static dimension of public procurement management. For more prolific results, a new study using time-series analysis would be needed. Only then would the evolutionary characteristics of the research phenomena be captured.

Tables

Table 1. Number of public procurement contracts, total value and mean value per contract

Source: Public Procurement Office, Report on public procurement for 2016 and authors' calculation

- 138/139 -

Table 2. Some demographic features of respondents

Table 3. Descriptive statistics for the observed variables

Table 4. Correlation matrix for the observed variables

- 139/140 -

Table 5. Regression model for contract management efficiency

a. Dependent Variable: Contract mngt efficiency

R=.715 R[2]=.511 SE=1.094 F=26.292 Sig=.000 Durbin Watson=1.849 ■

NOTES

[1] H. Walker and S. Brammer, Sustainable procurement in the United Kingdom public sector, (2009) 14 (2) Supply Chain Management: An International Journal, 128-137, https://doi.org/10.1108/13598540910941993

[2] S. Brammer and H. Walker, Sustainable procurement in the public sector: an international comparative study, (2011) 31 (4) International Journal of Operations & Production Management, 452-476, https://doi.org/10.1108/01443571m119551

[3] T. L. Brown and M. Potoski, Contract-management capacity in municipal and county governments, (2003) 63 (2) Public Administration Review, 153-164, https://doi.org/10.1111/1540-6210.00276

[4] J. Levin and S. Tadelis, Contracting for government services: Theory and evidence from U.S. cities, (2010) 58 (3) The Journal of Industrial Economics, 507-541, https://doi.org/10.1111/j.1467-6451.2010.00430.x

[5] A. Hefetz and M. E. Warner, Contracting or public delivery? The Importance of service, market, and management characteristics, (2011) 22 (2) Journal of Public Administration Research and Theory, 289317, https://doi.org/10.1093/jopart/mur006

[6] G. Bel and X. Fageda, Why do local governments privatise public services? A survey of empirical studies, (2007) 33 (4) Local Government Studies, 517-534, https://doi.org/10.1080/03003930701417528

[7] G. A. Boyne, Bureaucratic theory meets reality: public choice and service contracting in U. S. local government, (1998) 58 (6) Public Administration Review, 474-484, https://doi.org/10.2307/977575

[8] G. Bel, X. Fageda and M. E. Warner, Is private production of public services cheaper than public production? A meta-regression analysis of solid waste and water services, (2010) 29 (3) Journal of Policy Analysis and Management, 553-577, https://doi.org/10.1002/pam.20509

[9] P. Garrone and R. Marzano, Why do local governments resist contracting out?, (2015) 51 (5) Urban Affairs Review, 616-648, https://doi.org/10.1177/1078087414549548

[10] K. V. Thai, International public procurement: innovation and knowledge sharing, (2015) International Public Procurement, 1-10, https://doi.org/10.1007/978-3-319-13434-5_1

[11] V. Lember, R. Kattel and T. Kalvet (eds), Public procurement, innovation and policy (Springer, 2014) https://doi.org/10.1007/978-3-642-40258-6; S. Appelt and F. Galindo-Rueda, Measuring the link between public procurement and innovation, (2016) (3) OECD Science, Technology and Industry Working Papers, https://doi.org/10.1787/5jlvc7sl1w7h-en

[12] C. Bratt, S. Hallstedt, K.-H. Robert, G. Broman and J. Oldmark, Assessment of criteria development for public procurement from a strategic sustainability perspective, (2013) 52Journal of Cleaner Production, 309-316, https://doi.org/10.1016/Uclepro.2013.02.007; M. K. Amann, J. Roehrich, M. Eßig and C. Harland, Driving sustainable supply chain management in the public sector, (2014) 19 (3) Supply Chain Management: An International Journal, 351-366, https://doi.org/10.1108/scm-12-2013-0447; S. Witjes and R. Lozano, Towards a more Circular Economy: Proposing a framework linking sustainable public procurement and sustainable business models, (2016) 112 Resources, Conservation and Recycling, 37-44, https://doi.org/10.1016/j.resconrec.2016.04.015

[13] S. Vagstad, Centralized vs. decentralized procurement: Does dispersed information call for decentralized decision-making?, (2000) 18 (6) International Journal of Industrial Organization, 949963, https://doi.org/10.1016/s0167-7187(98)00044-7; B. Brezovnik, Ž. J. Oplotnik and B. Vojinović, (De)Centralization of public procurement at the local level in the EU, (2015) 11 (46) Transylvanian Review of Administrative Sciences, 37-52.

[14] C. McCrudden, Using public procurement to achieve social outcomes, (2004) 28 (4) Natural Resources Forum, 257-267, https://doi.org/10.1111/j.1477-8947.2004.00099.x

[15] J. A. Lynch T., Public procurement and contract administration: A brief introduction (Lean pub, 2013) 8.

[16] Brown and Potoski, Contract-management capacity in municipal and county governments, 153-164, https://doi.org/10.1111/1540-6210.00276

[17] Prager, J. Contracting out government services: Lessons from the private sector, (1994) 54 (2) Public Administration Review, 176-184, https://doi.org/10.2307/976527

[18] Y. Minchuk and S. Mizrahi, The (in) effectiveness of procurement auctions in the public sector, (2016) 24 (4) Applied Economic Letters, 247-249, https://doi.org/10.1080/13504851.2016.1181704

[19] F. Decarolis, Awarding price, contract performance, and bids screening: Evidence from procurement auctions, (2014) 6 (1) American Economic Journal: Applied Economics, 108-132, https://doi.org/10.1257/app.6.1.108

[20] B. Davison and R. J. Sebastian, An analysis of the consequences of contract administration problems for contract types, (2009) 1 (2) Journal of Management Research, 1-32, https://doi.org/10.5296/jmr.vli2.44

[21] D. Matthews, Strategic procurement in the public sector: A mask for financial and administrative policy, (2005) 5 (3) Journal of Public Procurement, 388-399, https://doi.org/10.1108/JOPP-05-03-2005-B005

[22] Official Gazette RS 39/2002.

[23] Council Directive 92/50/EEC.

[24] Council Directive 93/36/EEC.

[25] Council Directive 93/37/EEC.

[26] Council Directive 93/38/EEC.

[27] Official Gazette RS, 116/2008.

[28] Official Gazette RS, 124/2012, 14/2015 and 68/2015.

[29] The Government of the Republic of Serbia, Pregovaračka pozicija Republike Srbije za međuvladinu konferenciju o pristupanju Republike Srbije Evropskoj Uniji ua pregovaračko poglavlje 5 - Javne nabavke (The Government of the Republic of Serbia, text in Serbian, Belgrade, 2016) http://eupregovori.bos.rs/progovori-o-pregovorima/uploaded/pg_pozicija_pg_5(1).pdf (Last accessed: 3 July 2017).

[30] E.g. Public Procurement Office, Strengthening public procurement in Serbia (Public Procurement Office, Belgrade, 2010) http://www.ujn.gov.rs/en/obavestenja/story/119/STRENGHTENING+PUBLIC+PROCUREMENT+IN+SERBIA.html (Last accessed: 6 June 2017).

[31] P. Jovanović, N. Žarkić Joksimović and M. Milosavljevic, The efficiency of public procurement centralization: Empirical evidence from Serbian local self-governments, (2013) 11 (4) Lex Localis -Journal of Local Self-Government, 883-899, https://doi.org/10.4335/11.4.883-899(2013)

[32] M. Milosavljevic, N. Milanovic and S. Benkovic, Waiting for Godot: Testing transparency, responsiveness and interactivity of Serbian local governments, (2017) 15 (3) Lex Localis - Journal of Local Self-Government, 513-528, https://doi.org/10.4335/15.3.513-528(2017)

[33] M. Milovanovic, M. Bogicevic, M. Lazovic, D. Simic and D. Starcevic, Choosing authentication techniques in e-procurement system in Serbia, in 2010 International Conference on Availability, Reliability and Security, IEEE, https://doi.org/10.1109/ares.2010.82

[34] V. Bobar, K. Mandic and M. Suknovic, Bidder selection in public procurement using a fuzzy decision support system, (2015) 7 (1) International Journal of Decision Support System Technology, 31-49, https://doi.org/10.4018/ijdsst.2015010103; V. Bobar, K. Mandić, B. Delibašić and M. Suknović, An integrated fuzzy approach to bidder selection in public procurement: Serbian Government case study, (2015) 12 (2) Acta Polytechnica Hungarica, 193-211, https://doi.org/10.12700/aph.12.2.2015.2.12

[35] A. Erridge and R. Nondi, Public procurement, competition and partnership, (1994) 1 (3) European Journal of Purchasing & Supply Management, 169-179, https://doi.org/10.1016/0969-7012(94)90006-x

[36] P. Kraljic, Purchasing must become supply management, (1983) 61 (5) Harvard Business Review, 109-117.

[37] K. Lavery, Smart contracting for local government services: Processes and experience (CT: Greenwood Publishing Group, Westport, 1999).

[38] O. Jaško, P. Jovanović and M. Čudanov, Cost efficiency of public procurement at local level: Chances for Improvement of local self-government and public enterprises in Serbia, (2015) 13 (3) Lex Localis - Journal of Local Self-Government, 789-807, https://doi.org/10.4335/13.3.789-807(2015); D. S. Jones, Procurement practices in the Singapore civil service: Balancing control and delegation, (2002) 2 (1) Journal of Public Procurement, 29-53, https://doi.org/10.1108/JOPP-02-01-2002-B002; J. D. Coggburn, Exploring differences in the American states' procurement practices, (2003) 3 (1) Journal of Public Procurement, 3-28, https://doi.org/10.1108/JOPP-03-01-2003-B001

[39] P. R. Schapper, J. N. V. Malta and D. L. Gilbert, An analytical framework for the management and reform of public procurement, (2006) 6 (1/2) Journal of Public Procurement, 1-26, https://doi.org/10.1108/JOPP-06-01-02-2006-B001

[40] PMI, A guide to Project Management Body of Knowledge, http://www.cs.bilkent.edu.tr/~cagatay/cs413ZPMBOK.pdf (Last accessed: 20 June 2017); R. G. Rendon, Procurement process maturity: Key to performance measurement, (2008) 8 (2) Journal of Public Procurement, 200-214, https://doi.org/10.1108/JOPP-08-02-2008-B003

[41] J. Gordon Murray, Improving the validity of public procurement research, (2009) 22 (2) International Journal of Public Sector Management, 91-103, https://doi.org/10.1108/09513550910934501

[42] S. Tadelis, Public procurement design: Lessons from the private sector, (2012) 30 (3) International Journal of Industrial Organization, 297-302, https://doi.org/10.1016/j.ijindorg.2012.02.002

[43] PMI, A guide to Project Management Body of Knowledge.

[44] P. Biernacki and D. Waldorf, Snowball sampling: Problems and techniques of chain referral sampling, (1981) 10 (2) Sociological Methods & Research, 141-163, https://doi.org/10.1177/004912418101000205

[45] T. Bodin, G. Johansson, T. Hemmingsson, C. Nylén, K. Kjellberg, B. Buström and P. O. Östergren, S09-2 Respondent-driven sampling in sampling hard-to-reach precarious workers, (2016) 73 (Suppl 1) Occupational and Environmental Medicine, A109-A109, https://doi.org/10.1136/oemed-2016-103951.294

[46] K. Avrachenkov, G. Neglia and A. Tuholukova, Subsampling for chain-referral methods, (2016) Lecture Notes in Computer Science, 17-31, https://doi.org/10.1007/978-3-319-43904-4_2

[47] See Table 2.

[48] M. Tavakol and R. Dennick, Making sense of Cronbach's alpha, (2011) 2 InternationalJournal of Medical Education, 53-55, https://doi.org/10.5116/ijme.4dfb.8dfd

[49] A. I. Alonso, The shaping of local self-government and economic development through city strategic planning: A case study, (2014) 12 (3) Lex Localis - Journal of Local Self-Government, 373-391, https://doi.org/10.4335/12.3.373-391(2014)

[50] T. Dezelan, A. Maksuti and M. Uršič, Capacity of local development planning in Slovenia: Strengths and weaknesses oflocal sustainable development strategies, (2014) 12 (3) Lex Localis - Journal of Local Self-Government, 547-573, https://doi.org/10.4335/12.3.547-573(2014)

[51] W. C. Lawther and L. L. Martin, Innovative practices in public procurement partnerships: The case of the United States, (2005) 11 (5-6) Journal of Purchasing and Supply Management, 212-220, https://doi.org/10.1016/j.pursup.2005.12.003

[52] Erridge and Nondi, Public procurement, competition and partnership, 169-179, https://doi.org/10.1016/0969-7012(94)90006-X

[53] S. Seshadri, K. Chatterjee and G. L. Lilien, Multiple source procurement competitions, (1991) 10 (3) Marketing Science, 246-263, https://doi.org/10.1287/mksc.10.3.246

[54] P. Furlong, F. Lamont and A. Cox, Competition or partnership?, (1994) 1 (1) European Journal of Purchasing & Supply Management, 37-43, https://doi.org/10.1016/0969-7012(94)90041-8

[55] O. Mamavi, H. Nagati, G. Pache and F. T. Wehrle, How does performance history impact supplier selection in public sector?, (2015) 115 (1) Industrial Management & Data Systems, 107-128, https://doi.org/10.1108/imds-07-2014-0222

[56] R. H. Porter and J. D. Zona, Detection of bid rigging in procurement auctions, (1993) 101 (3) Journal of Political Economy, 518-538, https://doi.org/10.1086/261885

[57] P. C. Van Duyne, E. Stocco, V. Bajovic, M. Milenović and E. E. Lojpur, Searching for corruption in Serbia, (2010) 17 (1) Journal of Financial Crime, 22-46, https://doi.org/10.1108/13590791011009356

[58] M. Milosavljevic, N. Milanović and S. Benković, Politics, policies and public procurement efficiency: A quantitative study of 25 European countries, (2016) 14 (3) Lex Localis - Journal of Local Self-Government, 537-558, https://doi.org/10.4335/14.3.537-558(2016)

[59] Å. B. Grødeland and A. Aasland, Fighting corruption in public procurement in post-communist states: Obstacles and solutions, (2011) 44 (1) Communist and Post-Communist Studies, 17-32, https://doi.org/10.1016/j.postcomstud.2011.01.004

[60] K. Raik, EU accession of Central and Eastern European countries: Democracy and integration as conflicting logics, (2004) 18 (4) East European Politics and Societies, 567-594, https://doi.org/10.1177/0888325404269719

[61] Lynch, Public procurement and contract administration...

[62] A. A. Amirkhanyan, Collaborative performance measurement: examining and explaining the prevalence of collaboration in state and local government contracts, (2008) 19 (3) Journal of Public Administration Research and Theory, 523-554, https://doi.org/10.1093/jopart/mun022

[63] M. Plaček, The effects of decentralization on efficiency in public procurement: Empirical evidence for the Czech Republic, (2017) 15 (1) Lex Localis-Journal of Local Self-Government, 67-92, https://doi.org/10.4335/15.1.67-92(2017); M. Murray Svidroňová and J. Nemec, E-Procurement in self-governing regions in Slovakia, (2016) 14 (3) Lex Localis - Journal of Local Self-Government, 321-335, https://doi.org/10.4335/14.3.321-335(2016)

[64] Public Procurement Office, Report on public procurement for 2016, http://www.ujn.gov.rs/ci/izvestaji/izvestaji_ujn (Last accessed: 6 July 2017) 6.