Analysis of a Conceptual Model and Assessment of the e-Procurement System in Bangladesh

Md. Abdur Rashid¹, Mohammad Shorif Uddin^2*

¹Bangladesh University of Professionals, Dhaka, Bangladesh.
²Department of Computer Science and Engineering, Jahangirnagar University, Dhaka, Bangladesh.

DOI: 10.4236/jcc.2021.911005                

Abstract

The public procurement system in Bangladesh has been running by traditional manual process under the flagship of Public Procurement Act (PPA 2006) and Public Procurement Rules (PPR 2008). Public procurement agencies have been facing challenges in this manual tendering system. To overcome this problem and to bring reality to the “Digital Bangladesh” slogan, the Government of Bangladesh introduced the e-Procurement system under the e-GP (Electronic Government Procurement) guideline 2011. After the inception of e-procurement, there is no e-procurement assessment model to improve the e-GP system. The purpose of this research is to develop a conceptual framework and to design an e-procurement assessment model. With this view, we have considered one of the biggest entity Roads and Highways Department (RHD) of the Government of Bangladesh, for field study. Mixed methods along with FGD (Focus Group Discussion), KII (Key Informant Interview), and survey questionnaires are used to collect data from RHD, and then Statistical Package for Social Science (SPSS) software is used for regression analysis and hypothesis testing to develop the e-Procurement assessment model. The novel contribution of the study lies in the test of the hypothesis that focuses on developing the conceptual model of the e-procurement assessment system in Bangladesh. Findings of the study are essential for all Procurement Entity (PE) and suppliers i.e. contractors of RHD who are engaged in the construction of the infrastructure project development project.

Keywords

e-GP Guideline 2011, Conceptual Framework, e-Procurement Assessment Model, Hypothesis Test, Regression Analysis

1. Introduction

Since 2011, Bangladesh has started the e-Procurement system [1] in public procurement developed and managed by the Central Procurement Technical Unit (CPTU) under the Ministry of Planning. The electronic Government Procurement (e-GP) system is the flagship technology of CPTU and was introduced by the recommendation of the World Bank [2] to bring efficiency, transparency, and fairness in bidding competition in all public procurements. Though the e-GP system was being implemented in 2011, the bidders and procurement entities (PEs) are still facing some challenges [3] [4]. Any e-procurement assessment model is not being designed since the inception of e-GP system. The primary purpose of this research is to get the e-procurement assessment model. Three objectives of the study are: 1) to find the challenges experienced during the e-Procurement implementation, 2) to compare the procurement efficiency between traditional purchase and e-Procurement purchase, and 3) to predict significant effects of different factors that influence e-Procurement implementation. For our investigation, we have considered the most significant development procuremententity of the Government of Bangladesh—the Roads and Highways Department (RHD) [5]. The broad topic taken for this research is an assessment [6] of the e-procurement system in implementing the development projects of the RHD. Through a detailed study of e-Procurement literature, it is found that no in-depth research has been done yet in assessing e-GP system implementation since its inception. The main motivation of this research is to fill this gap.

According to the seventh five-year plan [7], the target of the Bangladesh government is to conduct 100% of public procurement using the e-tender system through e-Procurement software by 2020. Now the time has come to assess the e-Procurement system to find the contributing factors [8] affecting the e-Procurement implementation. This study performs the hypothesis test by field data analysis in proposing a conceptual framework for the e-procurement implementation assessment.

This paper is structured as follows. Section 2 describes related literature, Section 3 provides a conceptual framework for the e-procurement implementation assessment, Section 4 presents the methodology used in this research, and Section 5 contains the key results and discussions. Finally, Section 6 concludes this work.

2. Literature Review

1) e-Procurement and e-GP Vision

E-procurement [9] [10] uses secure web-based application platforms by the government as a buyer to communicate with the bidder’s community in a paperless environment free from inconveniences, delays, and physical insecurities of any kind. According to the definition, e-procurement involves the procurement of goods and services using the internet or electronic network in its implementation [11]. CPTU made the e-GP vision to enhance the efficiency and transparency of the Bangladesh public procurement system [1].

2) Theoretical Motivation

Every researcher gets important clues designing the research and data analysis technique from the relevant theory. The researcher here studied 2 theories like Maps inititative assessment model and Gardenal e-Procurement assessment model.

a) Gardenal e-Procurement assessment model

Empirical researches on the practical applications, leveraging benefits, and the value of e-Procurement in the public sector of Bangladesh has been done by researchers in a limited way. For the sake of this current review, the model [16] which has been applied four successive times since 2008 in Italy for assessment of e-Procurement in the public sector of Lombardia Region of Italy, will be followed here. Francesco Gardenal considered five dependent variables viz, efficiency, effectiveness, competitiveness, dematerialization, and transparency as impact dimensions for assessment.

b) Maps initiative assessment model

Around the world, Governments are spending about USD 9.5 trillion [6] in public procurement each year. On average, public procurements contribute about 12% - 20% country’s GDP around the globe.

The Methodology for Assessing Procurement Systems (MAPS) is a widely used measurement tool for assessing public procurement in the world which was developed in 2003/2004. Universally accepted MAPS assessments [6] have already been conducted in more than 90 countries globally. Following SDG, MAPS will be used by all countries soon. MAPS tools are very much helpful to assess the quality and effectiveness of the public procurement system.

MAPS follows [6] SDG Goal 12, which calls for promoting sustainable procurement practices in line with national priorities and policies, and SDG Goal 16, which calls for effective and accountable institutions.

The following considerations guided the MAPS correction [6]:

• Value for money

• Transparency

• Fairness

• Good governance

MAPS Pillars to assess public procurement are:

i) Regulatory, Legal, and Policy Framework

ii) Institutional Framework and Management Capacity

iii) Public Procurement Operations and Market Practices

iv) Accountability, Integrity, and Transparency of the Public Procurement System.

3) Research Gap Analysis

A critical review of the literature has been made and a summary of the research gap is presented below. In India, the GePNIC system [12] catered to around 75% of the country’s electronic procurement needs. This helped the government of India to gather some experience on saving procurement expenditures, majorly cost and time. In Sri Lanka, its government is currently considering a range of procurement reforms including the adoption of the e-GP system intending to increase the efficiency and timeliness of the public procurement process for the betterment of the country [13]. In Nepal, the e-Procurement system has been introduced to increase the much-needed transparency and remove bias to ensure equal access which in turn will proliferate open competition in Nepal public procurements [14]. e-Perolehan [15] started in Malaysia and became a very popular mechanism for e-Procurement. It helped to improve the relationship between the government and bidding communities. It removed the red-tapes and enforced compliance. It also helped to enhance e-commerce, accountability, and transparency in government contracts and projects.

3. Conceptual Framework

The conceptual framework is the output of the in-depth study after a comprehensive critical literature review and research gap analysis on the adoption of e-procurement in an organization of a country. Following is the list of associated variables in Table 1 that was obtained from the study.

RHD’s bidder and PE community surveys (questionnaires) were considered to measure the significant effects of variables of the conceptual framework. The following 4 hypotheses of Figure 1 are considered in this paper to test the proposed e-Procurement assessment model.

H₁.

H_o: Effective e-Procurement implementation assessment does not significantly depend on accountability.

H_a: Effective e-Procurement implementation assessment significantly depends

Table 1. Variables list for the proposed model and hypothesis test.

on accountability.

H₂.

H_o: Effective e-Procurement implementation assessment does not significantly depend on public procurement governance.

H_a: Effective e-Procurement implementation assessment significantly depends on public procurement governance.

H₃.

H_o: Effective e-Procurement implementation assessment does not significantly depend on dematerialization.

H_a: Effective e-Procurement implementation assessment significantly depends on dematerialization.

H₄.

H_o: Effective e-Procurement implementation assessment does not significantly depend on legal, regulatory, and policy framework.

H_a: Effective e-Procurement implementation assessment significantly depends on legal, regulatory, and policy framework.

Variables in Table 1 are separated into dependent and independent variables to make the proposed conceptual framework of the study and to analyze the hypothesis data properly using SPSS software. Total four variables are treated as dependent variables those will be treated as factors that influence the assessment of the e-procurement implementation model.

Using Table 1, the essential proposed conceptual framework of the study has been developed and shown in Figure 1. The proposed model is the hybridization of the Maps initiative [6], Francesco Gardenal’s e-Procurement impact model [16], and from the summary of the literature of the study.

Figure 1. The proposed conceptual framework of the e-Procurement implementation assessment model.

4. Methodology

The essence of research design is to translate a research problem into data for analysis to impart pertinent answers to research questions at the least cost [17]. The mixed-method [18] based on the qualitative and quantitative investigation has been used here to triangulate the result. A stratified sampling method has been used for the survey.

The population of the study is the PE officers and bidders of RHD. In this study, the population is known. Considering the known population, the total sample size of the survey is determined about 402 where PE officers, bidders, and FGD are 206, 145, and 51, respectively. KII, FGD, and survey questionnaire tools are adopted to conduct the research study. The primary data was collected from 4 wings, 11 zones, 31 circles, and 70 divisions of RHD in Bangladesh to maintain space triangulation.

Hypothesis tests were done on 4 dependent variables: Accountability, Procurement Governance, Dematerialization, Legal, Regulatory, and Policy Framework. Multiple Linear Regression (MLR) model using SPSS version 23 software was used for data analysis to test 4 hypotheses of the proposed conceptual framework. Each dependent variable had some influencing factors i.e. independent variables shown in Table 1. The survey questionnaires were developed on a 5-point liker scale to predict significant effects of different factors that influence e-Procurement implementation in RHD development project procurement. Analysis of the 4 hypotheses helped to test the proposed conceptual framework and infer about the assessment of the e-Procurement implementation model in RHD.

The survey used both structured and unstructured questionnaires keeping in line with the three objectives of the study. Data were collected from remote PE offices and bidders using email and telephone. Telephonic communication helped to clarify some answers to specific questions. The researchers collected the secondary data through document study and by reviewing the relevant literature.

5. Results and Discussions

5.1. Descriptive Statistics of PE Officers Demographic Data

The findings of gender issues indicated that 93.7% of PE officers were male while 6.3% of respondents were female. This finding implies that the majority of the PE officers in RHD are males.

The findings regarding the age of the respondents state that the age of PE officers started from 22 years and the maximum age is 59 years. The mean age is 41.21 years. The standard deviation of the PE officer’s age is 9.661 that indicates the average respondents are mid-level officers and they possess sufficient knowledge to be involved with the e-procurement systems.

The study about respondents’ education level is that principal PE officers i.e. 53.4% hold Bachelor of Science in engineering degrees. It is argued that significant respondents are educated who handle the e-procurement system in RHD.

The findings indicate that Sub Asst. Engineers, Asst. Engineers, Sub Divisional Engineers, Executive Engineers, Superintendent Engineers to senior Additional Chief Engineers are involved in giving opinions as of KII. It proves that data have been collected from all available posts in RHD who are actively engaged in the e-Procurement system at their offices in RHD.

From the findings, it is found that the PE officers’ e-GP roles are as PE users, TOC members, PE admin, and TEC members. Among them, a major 29.6% is PE users. These findings indicate that data was collected from the very relevant PE officers in RHD who had an active role in the e-Procurement system for public procurement. It also reveals that the credibility of the data is high.

In response to the question “how many times PE officers got e-GP training?” the findings indicate that 16.5% of PE officers did not participate in any e-GP related training. 44.2% of PE officers participated in e-GP related training 1 time, 29.1% of PE officers participated in e-GP related training 2 times, only 4.4% of PE officers participated in e-GP related training 3 times. Therefore, a significant proportion of PE officers had not attained at least 1 time of e-procurement training in RHD. This causes challenges and difficulties in handling e-Procurement related tender documents at the stage of the e-procurement process. The data suggest that PE officers training on e-procurement software at least once is a must to handle e-procurement software smartly.

5.2. Descriptive Statistics of Bidders Demographic Data

The findings express that 100% of respondent bidders are male. The data reveals that there is no female bidder in RHD. It reflects the gender imbalance in bidders’ perception. The presence of a gender-equity policy will bring a balance between the male and female bidders.

The answer relating to the question of the highest educational degree of the bidders revealed that 62% of bidders possess a bachelor’s degree. Therefore, the major portion of the respondents has the potentiality to handle the e-procurement bidding process.

In response to the question “how many bidders got e-Procurement training?” indicates that 71% of bidders did not participate in any e-GP related training that was provided by CPTU. 26.2% of bidders participated in e-GP related training 1 time. Therefore, the major portion of bidders had not attained the necessary training on the implementation of the e-Procurement system in RHD. It creates challenges and difficulties in filling up the e-Procurement related tender documents at the stage of the bidding process. The data suggests that all registered bidders should be ensured to participate in the e-GP training.

5.3. Inferential Statistical Analysis

One of the three objectives was “To predict significant effects of different factors that influence e-Procurement implementation”. The study tested four hypotheses H₁ to H₄, for an effective e-Procurement implementation assessment model.

Hypothesis test on accountability H₁:

Let, dependent variable Y = accountability

And 4 independent variables are:

X₁ = Collaboration between PE and bidders, X₂ = PE officer’s behavioral changes,

X₃ = Bidder’s behavioral changes, X₄ = Value for money.

Regression Model-1 is: $Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_3+{\beta }_4{X}_4+{ϵ}_i$

SPSS data interpretation:

The above model summary Table 2 (determining how well the model fits) provides the value of R and R².

The R-value represents the degree of correlation which is 0.556 (the “R” Column).The analyzed correlation value of 55.6% indicates a high degree (Positive strong) of correlation.

The R² value (the “R Square” column) indicates how much of the total variation in the dependent variable “accountability”. By the R² value of the above model summary Table 2, it can be concluded that the dependent variable “accountability” has a 30.9% dependency on 4 independent variables.

The regression model predicts the dependent variable accountability significantly well which reports that the regression equation fits the data well.

Here we got, sig = 0.000; i.e. p < 0.05

So, overall, the regression model is significant and statistically fit. Findings show that overall regression has a significant influence on increasing accountability by 4 independent variables.

Fitted regression model-1 is:

$\begin{array}{l}Y=0.882+0.072X_1+0.184X_2+0.232X_3+0.344X_4\\ t2.1980.9022.0772.9755.838\\ \text{Sig}\left(0.029\right)\left(0.368\right)\left(0.039\right)\left(0.003\right)\left(0.000\right)\end{array}$

Hypothesis test on procurement governance H₂—

Let, dependent variable Y = procurement governance

And the 6 independent variables are

X₁ = capacity development, X₂ = integrating principles, X₃ = procurement planning publish, X₄ = quality government tendering process, X₅ = free from red-tapes, X₆ = free from after contracts awarding

Regression Model-2 is $Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_3+{\beta }_4{X}_4+{\beta }_5{X}_5+{\beta }_6{X}_6+{ϵ}_i$

SPSS data interpretation:

The above model summary Table 3 (determining how well the model fits) provides the value of R and R².

Table 2. Model summary of correlation on accountability.

^aPredictors: (Constant), Value for money, Collaboration between PE and bidders, bidders behavioral changes, PE officers behavioral changes.

Table 3. Model summary of correlation on procurement governance.

^aPredictors: (Constant), Free from after contracts awarding, Capacity development, Procurement planning publish, Quality government tendering process, Integrating principles, Free from red-tapism.

The R-value represents the degree of correlation, which is 0.883 (the “R” Column). The analyzed correlation value of 88.3% indicates a high degree (positive strong) of correlation.

The R² value (the “R Square” column) indicates how much of the total variation in the dependent variable “procurement governance”. By the R² value of the above model summary Table 3, it can be concluded that the dependent variable “procurement governance” has 77.9% dependencyon6 independent variables.

The regression model predicts the dependent variable procurement governance significantly well which reports that the regression equation fits the data well (i.e., predicts the dependent variable).

Here we got, sig = 0.000; i.e. p < 0.05

So, overall, the regression model is significant and statistically fit. Analysis tells that overall regression has a significant influence on increasing procurement governance by 6 independent variables. In another word, the independent variables reliably predict the dependent variable.

Fitted regression model-2 is:

$\begin{array}{l}Y=0.353+0.060X_1-0.024X_2+0.002X_3+0.757X_4+0.094X_5+0.044X_6\\ t1.8911.592-0.5680.06418.5032.4631.211\\ \text{Sig}\left(0.0\text{6}0\right)\left(0.\text{113}\right)\left(0.\text{571}\right)\left(0.\text{949}\right)\left(0.000\right)\left(0.0\text{15}\right)\left(0.\text{227}\right)\end{array}$

Hypothesis test on dematerialization H₃—

Let, dependent variable Y = dematerialization and the 2 independent variables are

X₁ = paperless procurement process, X₂ = reduced archiving cost

Regression Model-3 is $Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{ϵ}_i$

SPSS data interpretation:

The above model summary Table 4 (determining how well the model fits) provides the value of R and R².

The R-value represents the degree of sample correlation which is 0.946 (the “R” Column). The analyzed correlation value is 94.6% which indicates a high degree (positive strong) of correlation.

The R² value (the “R Square” column) indicates how much of the total variation in the dependent variable “dematerialization”. By the R² value of the above

Table 4. Model summary of correlation on dematerialization.

^aPredictors: (Constant), Reduced archiving cost, Paperless procurement process.

model summary Table 4, it can be concluded that the dependent variable “dematerialization” has 89.4% dependency on 2 independent variables.

The regression model predicts the dependent variable dematerialization significantly well which reports that the regression equation fits the data well (i.e., predicts the dependent variable).

Here we got, sig = 0.000; i.e. p < 0.05

So, overall, the regression model is significant and statistically fit. Analysis tells that overall regression has a significant influence on increasing dematerialization by 2 independent variables.

Fitted regression model-3 is:

$\begin{array}{l}Y=0.205+0.894X_1+0.065X_2\\ t2.20929.5332.244\\ \text{Sig}\left(0.028\right)\left(0.000\right)\left(0.026\right)\end{array}$

Hypothesis test on legal, regulatory, and policy framework H₄—

Let, dependent variable Y = legal, regulatory, and policy framework and the 5 independent variables are

X₁ = overall CPTU performance, X₂ = PPA 2006 (and subsequent amendments), X₃ = PPR 2008 (and subsequent amendments), X₄ = e-GP guideline 2011, X₅ = implementation of digital signature

Regression Model-4 is: $Y={\beta }_0+{\beta }_1{X}_1+{\beta }_2{X}_2+{\beta }_3{X}_3+{\beta }_4{X}_4+{\beta }_5{X}_5+{ϵ}_i$

SPSS data interpretation:

The above model summary Table 5 (determining how well the model fits) provides the value of R and R².

The R-value represents the degree of sample correlation which is 0.913 (the “R” Column). The analyzed correlation value of 91.3% indicates a high degree (positive strong) of correlation.

The R² value (the “R Square” column) indicates how much of the total variation in the dependent variable “legal, regulatory and policy framework” can be explained by the influence of the 5 independent variables. By the R² value of the above model summary Table 5, it can be concluded that the dependent variable “legal, regulatory and policy framework” has 83.4% dependency on 5 independent variables.

The regression model predicts the dependent variable legal, regulatory, and policy framework significantly well fit which reports that the regression equation fits the data well (i.e., predicts the dependent variable).

Here we got, sig = 0.000; i.e. p < 0.05

So, overall, the regression model is significant and statistically fit. Analysis tells that overall regression has a significant influence on increasing legal, regulatory, and policy framework by 5 independent variables.

Fitted regression model-4:

$\begin{array}{l}Y=0.390+0.013X_1+0.022X_2+0.047X_3+0.846X_4+0.006X_5\\ t2.5390.4270.4111.02224.7180.469\\ \text{Sig}\left(0.0\text{12}\right)\left(0.\text{67}0\right)\left(0.\text{66}0\right)\left(0.\text{3}0\text{8}\right)\left(0.000\right)\left(0.0\text{64}\right)\end{array}$

This section presents the key points of the results of the previous subsection without statistical calculations. Key findings of each hypothesis are summarized in the following Table 6 for purpose of discussion.

R values of hypothesis H₁ - H₄ indicate that dependent variables accountability, procurement governance, dematerialization, (legal, regulatory, and policy frame-work) have a mutually strong positive correlation with its corresponding independent variables.

R² values of hypothesis H₁ - H₄ indicate that dependent variables accountability, procurement governance, dematerialization, (legal, regulatory, and policy framework) have dependencies on their corresponding independent variables. Noted that this was an overall measure of the strength of association and did not reflect the extent to which particular independent variables are associated with the dependent variables.

So, overall, the regression model is highly significant and statistically fit.

The coefficient value of hypothesis H₁ - H₄ indicates that independent variables have a significant influence on the dependent variable.

Table 5. Model summary of correlation on legal, regulatory, and policy framework.

^aPredictors: (Constant), Digital signature implementation, Overall CPTU Performance, e-GP guideline 2011, PPR 2008 and amendments, PPA 2006 and amendments.

Table 6. Results of hypothesis testing.

Source: Researcher’s field survey data analysis, 2020.

The test result of Table 6 indicates that the effective e-Procurement implementation assessment significantly depends on the dependent variables accountability, procurement governance, dematerialization, (legal, regulatory, and policy framework. It also confirms that the proposed conceptual framework of Figure 1 for the e-procurement assessment model is justified.

6. Conclusion

This paper proposed a conceptual framework for e-Procurement assessment model based on the critical literature review and research gap analysis comprising 4 dependent variables like accountability, procurement governance, dematerialization, and (legal, regulatory, and policy framework) through FGD, KII, and survey questionnaires on RHD of the Government of Bangladesh. Hypothesis tests H₁ - H₄ using a regression model from the gathered data confirmed that there exists a strong correlation between each dependent variable with its corresponding independent variables. Test findings indicated that for 4 cases, H_o (null hypothesis) is rejected and Ha (alternative hypothesis) is accepted. The key message of this paper is that the proposed model for e-Procurement implementation assessment can be implemented to improve the e-GP policy guidelines.

Conflicts of Interest

The authors declare no conflicts of interest regarding the publication of this paper.

References

[1]	CPTU (2011) Bangladesh Electronic Government Procurement (e-GP) Guideline. Ministry of Planning, Bangladesh Gadget.
[2]	World Bank (2011) Bangladesh Country Procurement Assessment Report. World Bank Report No. 24144-BD, Washington DC.
[3]	Corsi, M. (2006) e-Procurement Overview. Università “La Sapienza” di Roma, 1-9.
[4]	Akando, Z.I. (2016) Challenges and Prospects of e-Procurement in Bangladesh: A Study on Roads and Highways Department. http://dspace.bracu.ac.bd/xmlui/bitstream/handle/10361/6516/14272015_MAGD.pdf?sequence=1&isAllowed=y
[5]	RHD. http://www.rhd.gov.bd/Strategy/rhd_organogram_revised.pdf
[6]	Methodology for Assessing Procurement Systems, 2018. http://www.mapsinitiative.org/methodology/MAPS-methodology-for-assessing-procurement-systems.pdf
[7]	Planning Commission (2015) Seventh Five Year Plan of Bangladesh: FY2016-FY2020. Bangladesh.
[8]	KEVIN (2016) Critical Success Factors in the Implementation of e-Procurement in Public Entities in Kisumu County, Kenya. http://erepository.uonbi.ac.ke/bitstream/handle/11295/99538/Obat_Criti-cal%20Success%20Factors%20In%20The%20Implementation%20Of%20EProcure-ment%20In%20Public%20Entities%20In%20Kisumu%20County,%20Kenya.pdf?sequence=1&isAllowed=y
[9]	CPTU, e-Government Procurement (e-GP) System. https://www.eprocure.gov.bd/aboutUs.jsp
[10]	Nasidai, S.E. (2016) Factors Influencing Implementation of e-Procurement: A Case Study of Small and Medium-Size Businesses in Voi Town. European Journal of Logistics, Purchasing and Supply Chain Management, 4, 11-20.
[11]	Masudin, I., Aprilia, G.D., Nugraha, A. and Restuputri, D.P. (2021) Impact of E-Procurement Adoption on Company Performance: Evidence from Indonesian Manufacturing Industry. Logistics, 5, 16. https://doi.org/10.3390/logistics5010016
[12]	Ministry of Finance, India (2017) Annual Report on Central Public Procurement Portal. 1-59. https://eprocure.gov.in/cppp/sites/default/files/eproc/CPPP-AnnualReport-2017-18.pdf
[13]	Mohammed, R. (2017) E-Government Procurement: Enabling Business through Efficient Systems. Verité Research, 1-34.
[14]	National Electronic Government Procurement System 2021. https://bolpatra.gov.np/egp
[15]	Aman, A. and Kasimin, H. (2011) E-Procurement Implementation: A Case of Malaysia Government. Transforming Government People Process and Policy, 5, 330-344. https://doi.org/10.1108/17506161111173586
[16]	Francesco, G. (2013) A Model to Measure e-Procurement Impacts on Organizational Performance. Journal of Public Procurement, 13, 215-242. https://doi.org/10.1108/JOPP-13-02-2013-B003
[17]	Asenahabi, B.M. (2019) Basics of Research Design: A Guide to Selecting Appropriate Research Design. International Journal of Contemporary Applied Researches, 6, 76-89.
[18]	Creswell, J.W. (2007) An Introduction to Mixed Methods Research. https://sbsrc.unl.edu/Introduction%20to%20Mixed%20Methods.pdf