Continuous results-driven innovation management program

Iara Sibele Silva (Department of Electrical Engineering, Pontificia Universidade Catolica de Minas Gerais, Belo Horizonte, Brazil)
Patrícia Bernardes (Department of Administration, Pontificia Universidade Catolica de Minas Gerais, Belo Horizonte, Brazil)
Felipe Diniz Ramalho (Department of Electrical Engineering, Pontificia Universidade Catolica de Minas Gerais, Belo Horizonte, Brazil)
Petr Iakovlevitch Ekel (Department of Electrical Engineering, Pontificia Universidade Catolica de Minas Gerais, Belo Horizonte, Brazil)
Carlos Augusto Paiva da Silva Martins (Department of Electrical Engineering, Pontificia Universidade Catolica de Minas Gerais, Belo Horizonte, Brazil)
Matheus Pereira Libório (Department of Administration, Pontificia Universidade Catolica de Minas Gerais, Belo Horizonte, Brazil)

Revista de Gestão

ISSN: 2177-8736

Article publication date: 19 September 2019

Issue publication date: 5 December 2019

1328

Abstract

Purpose

The purpose of this paper is to present the innovation management program (IMP) (FAZ Program) and analyze its results according to the public policy goals that support it (Pró-Inova) suggesting improvements.

Design/methodology/approach

Intensive-direct-observation method in 43 companies; systematic data gathering and analysis (172 meeting documents); and innovation maturity diagnostics in 30 companies between August 2013 and May 2016.

Findings

The FAZ Program success rate according to the Pró-Inova goals achieved 81 percent. The percentage of completion of FAZ activities decreases during its implementation from 100 percent (strategic module) to 74 percent (management module) and ending at 46 percent (project module). The maturity for innovation of these committees/teams is decisive for those percentages. Companies whose maturity for innovation of the strategic committee and the organizational team are above average or excellent have, respectively, 1.8 and 1.7 times greater probability of implementing the program successfully.

Research limitations/implications

The FAZ Program represents only 4 percent of the programs supported by Pró-Inova. The innovative products, processes and businesses produced by the FAZ Program implementation are not measured. These innovations usually happen several years after an innovative management models implementation.

Practical implications

The maturity for innovation diagnosis is useful both to evaluate the company’s innovation capacity and to predict its chances of implementing the program successfully. Adjusting the structure of the model (e.g. PDCA cycle for the organizational module) and improving the program’s implementation (e.g. ensure management module resources and maturity for innovation capacity) can increase the program’s success rate.

Originality/value

Previous research works on IMPs supported by Pro-Inova focus on describing their methodology or benefits. The results allow answering what and how one of these programs offers in a return to the public innovation support received.

Keywords

Citation

Silva, I.S., Bernardes, P., Ramalho, F.D., Ekel, P.I., Martins, C.A.P.d.S. and Libório, M.P. (2019), "Continuous results-driven innovation management program", Revista de Gestão, Vol. 26 No. 4, pp. 389-408. https://doi.org/10.1108/REGE-01-2019-0006

Publisher

:

Emerald Publishing Limited

Copyright © 2019, Iara Sibele Silva, Patrícia Bernardes, Felipe Diniz Ramalho, Petr Iakovlevitch Ekel, Carlos Augusto Paiva da Silva Martins and Matheus Pereira Libório

License

Published in Revista de Gestão. Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode


1. Introduction

Although there are cases of innovation success in Brazil involving universities, research institutes and companies (e.g. the textile industry of the 1920s and 1930s, Embrapa and Embraer) (Rapini, Chaves, Albuquerque, Silva, Souza, Righi, & Cruz, 2009; Suzigan, & Albuquerque, 2011; Chaves, Carvalho, Silva, Teixeira, & Bernardes, 2012), studies have revealed a scenario where less than 1 percent of Brazilian companies invest in innovation activities (IBGE, 2014, 2016).

Since the Innovation Law (Zuniga, Negri, Dutz, Pilat, & Rauen, 2016), public policies to support innovation, such as the Lei do Bem (Law of Good), Embrapii, Sibratec (Fischer, Schaeffer, & Vonortas, 2019) and Finep-Pro (Arbix, Salerno, Amaral, & Lins, 2017), seek to increase the cases of innovation success involving universities, research institutes and companies (Lee, 2013). In the Finep-Pro scope, Pro-Inova is a public policy that supports innovation (Finep, 2010), which provides financial support for the development and implementation of innovation management programs (IMPs) in companies (Zen, Jaramillo, Dambros, Menezes, & Machado, 2014) as a response to the Business Mobilization for Innovation (BMI) (Mattos, Stoffel, & Teixeira, 2010) aiming to increase the companies’ innovation management capacity, and consequently the innovations’ success (Zawislak, Alves, Tello-Gamarra, Barbieux, & Reichert, 2013).

The number of researches on IMPs supported by Finep (2010) is small, but this number started to increase after the implementation in 2014 (Zen et al., 2014). Research works such as the Innovation Route (Zen, Machado, Jaramillo López, Borges, & Callegaro-de-Menezes, 2017), NUGIN Methodology (D’Barsoles, Iata, & Lezana, 2017; Feuerschütte, da Cunha Lemos, Hoffmann, & Fernandes, 2017), Pró-Inova Seridó Region (Santos, Motta, Luna, Barbosa, Brito, Nunes, & Romero, 2014), Matrix JOIN (Brandao, da Silva Gomes, & Segundo, 2015) and Model of Two Wheels (Bagno, & Faria, 2017) focus on describing the IMPs methodology and benefits.

These research works have an important role in showing the benefits generated by public policies that support innovation (Negri, & Rauen, 2018), especially for the innovative management models literature (Birkinshaw, & Mol, 2006; Birkinshaw, & Ansari, 2015). However, these research works fail in pointing out whether the results generated by these IMPs have met the Pró-Inova goals (Finep, 2010), and fail in pointing out possible improvements to these programs and their respective public policies (Monte Silva, & Guimarães, 2016).

This paper presents the IMP of the FAZ Program filling a gap with a research that analyzes an IMP’s results according to the public policy goals that support it. The analysis of the results from the FAZ Program is important because it gives insights about the expected economic return of the public policies that support it (Fischer et al., 2019). Besides, suggesting improvements is important because it may increase the innovation capacity of Brazilian companies (Saunila, & Ukko, 2014; Rothwell, 1992).

Besides this introduction, in Section 2 we present the environment of public policies that support innovation in Brazil and the private initiatives that shape the IMPs, detailing the FAZ Program’s financial support, development (model of two wheels and innovation management system (IMS)) and implementation. In Section 3, we present the methodology. In Section 4, we show the results from the implementation of FAZ Program (completed activities and innovation diagnosis), discussing whether these results met the Pró-Inova goals or not. Finally, in Section 5 we present the research contributions, limitations and suggestions for further research.

2. Innovation management programs

Public innovation policies (e.g. Innovation Law 2004) in Brazil have increased the effectiveness of governmental support instruments for innovation in companies (Zuniga et al., 2016). However, these policies were not enough to correct the relative disconnection (compared to other countries) between universities and companies (Lee, 2013) or to correct the mismatch between public efforts for innovation and the results generated by the economy (Monte Silva, & Guimarães, 2016). To correct these problems, a new institutional framework developed by the government with universities and companies added to the Innovation Law programs regarding tax deductions for companies (Law of Good), subsidy programs for research centers (e.g. Embrapii) and collaboration centers (e.g. Sibratec) (Fischer et al., 2019), economic grant programs for the development of high technological risk projects, and retaining researchers at companies (e.g. Finep-Pro) (Arbix et al., 2017).

As these programs increase the public resources allocation to companies and universities, there is an increased need for better control and evaluation of these resources as a way to improve the public policies that support innovation (Monte Silva, & Guimarães, 2016). Negri and Rauen (2018) emphasize that the control and evaluation of public policies that support innovation are an important way to give transparency and publicity to public spending. For the authors, it is not enough to know how and where public resources are being used, but what are the benefits generated by those resources. In addition, improvements in public policies that support innovation only occur when there is transparency regarding the direct and indirect impacts of these policies. Arbix, Salerno, Amaral, and Lins (2017) show that proactive stances such as the Business Mobilization for Innovation (BMI) of the National Confederation of Industry, the Competitive Brazil Movement and forums dedicated to connect companies, universities and public agencies are also ways to improve public policies that support innovation.

In 2010, a strong awareness of BMI began among Brazilian companies on the importance of innovation, offering capacity development in innovation management. This capacity was developed through models created by Innovation Management Support Centers (IMSCs) of State Industry Federations (Mattos et al., 2010). This increase in innovation management capacity is a BMI concern because the innovative activity performance of the companies depends as much on operational, commercial and technological aspects as on the innovation management (Zawislak et al., 2013). In this sense, developing capacity in management innovation is a way to produce specific managerial skills to reduce internal frictions between different areas of the company (Birkinshaw, & Mol, 2006). It also facilitates cognitive changes and acquisition of skills needed for organizational learning and to implement innovations (Birkinshaw, & Ansari, 2015).

On the one hand, the development of innovation management models involves risks and uncertain returns like any other innovation. Furthermore, management innovation is more difficult to justify, implement and quantify than technological innovations (Birkinshaw, & Mol, 2006). Best practices of innovation management do not always work in all companies and generate adverse effects during the effort to build capacity for innovation (Bagno, Leiva, & Oliveira, 2016). On the other hand, the authors state that IMS allow a more complete understanding of the gaps between best practices, results achieved by the company and expectations from future gains.

To reduce the risks and difficulty of implementing best practices of innovation management, Finep’s Pró-Inova public call offered financial support to Brazilian IMSCs (Mattos et al., 2010) for developing innovative models of innovation management (Birkinshaw, & Mol, 2006; Birkinshaw, & Ansari, 2015). These models should include 11 integrated or isolated topics: strategic planning; organization for innovation; methodologies and tools for evaluating and developing new products and processes; methodologies and tools for evaluating and developing new business; competitive intelligence systems; structuring of learning processes; structuring innovative environments and stimulating creativity (e.g. generation, selection and implementation of innovative ideas and projects); development of information systems for innovation (e.g. public news and innovation funding programs); systems for monitoring and evaluating innovation (e.g. performance indicators); innovative marketing systems; and technology prospecting and intellectual property management actions (Finep, 2010).

In total, 24 IMSCs received financial support from the Pró-Inova public call for developing IMPs (Zen et al., 2014). From these IMSCs, we found five research works about IMPs in the literature. In the state of Rio Grande do Sul, the IMP focuses on the resource-based view, linking the innovation process to a navigation route to meet the needs of micro and small technology-based enterprises (Zen et al., 2017). In the state of Santa Catarina, the IMP focuses on developing innovation management modules and tools to plan and manage innovation projects, identifying technological bottlenecks (D’Barsoles et al., 2017) and on developing a tool for identifying process and cultural patterns to enhance innovation (Feuerschütte et al., 2017). In the state of Rio Grande do Norte, the IMP focuses on the learning and diffusion of organizational innovation, presenting actions aimed at the innovation management culture development (Santos et al., 2014). In the state of Bahia, the IMP focuses on developing a collaborative, dynamic and playful tool to identify the company’s innovation maturity level, and to include a systematized innovation management in the business routine (Brandao et al., 2015). In the state of Minas Gerais, the IMP focuses on developing a continuous innovation management model and its implementation in companies by an IMS. Despite the difficulties of interaction (government, university and industry) faced by the IMSCs (Zen et al., 2014), the cited research works only reveal positive aspects or methodological descriptions of the IMP implementation. Even though some research works analyze the IMP implementation (Aniceto, Bagno, Alfradique, & de Souza, 2016) and its innovation management model (Silva, Libório, Bernardes, & Ramalho, 2018) or report and highlight specific points of the program (Oliveira, & Resende, 2016; Oliveira, Rocha, Camargos, & Bagno, 2016; Oliveira, Liborio, Fujimoto, Silva, Silva, & Laudares, 2016; Liborio, Fujimoto, Silva, Silva, Laudares, & Martins, 2016), these researches are still descriptive and do not show if the IMP results met all Pró-Inova goals.

This research goes further than an IMP description by analyzing its implementation results (highlighting problems and proposing improvements) and verifying the IMP results according to the Pró-Inova goals. The analysis focuses on the IMP developed by the IMSCs of the Federation of Industries of Minas Gerais. Through Pro-Inova, the IMP FAZ Program received from CNPq (Brazilian National Council for Scientific and Technological Development) and Finep (Funding Authority for Studies and Projects) a financial support of R$ 1.9m to develop and implement the first eight Pró-Inova topics (Fiemg, 2015). Figure 1 brings the operationalization of the conceptual framework of the FAZ Program and shows its financial support, development and implementations according to Pró-Inova topics.

According to Fiemg (2015), the FAZ Program development is divided into two fronts. The first front is the Innovation Management Model of Two Wheels (MTW) development. The second front is the IMS development. The IMS development includes systems (Pró-Inova topics 5 and 8) to monitor, manage and analyze information (technological, marketing and innovation financing) and to (Pró-Inova topics 3–4 and 7) systematize the methodologies and tools elaborated in the MTW. The MTW is a model developed by the Federal Universities of Minas Gerais and Viçosa (UFV) exclusively for the FAZ Program (Fiemg, 2015). This model adopts the method of control and continuous improvement of processes and products (Bagno, & Faria, 2017) to make innovation a continuous process in companies (García-Sánchez, García-Morales, & Martín-Rojas, 2018).

The FAZ Program implementation involves innovation capacity development and IMS training. The capacity development involves activities associated with the Pró-Inova topics 1–2 and 6 that comes from the MTW and do not integrate the FAZ Program IMS. The training involves activities associated with the Pró-Inova topics 3–5 and 7–8 that integrate the FAZ Program IMS (Fiemg, 2015). FAZ Program implementation takes one year and occurs through modules (strategic, manager, project and organizational) that have a set of specific activities and committees/teams (Silva et al., 2018). Figure 2 shows that the strategic, manager and project modules implementation is interactive, and that the interaction process occurs in cycle D (Do) of the PDCA.

This PDCA structure is known as an easy method to implement, execute and operate (Sokovic, Pavletic, & Pipan, 2010). The other advantage of the PDCA structure is to allow a sequential or concurrent implementation, or even a specific module implementation (Aniceto et al., 2016).

The MTW strategic module is associated with Pró-Inova topic 1 and aims to train companies for the strategic innovation planning elaboration. Strategic innovation planning involves processes and procedures that allow the alignment of values and corporate behaviors toward innovation activity, and allocate resources for innovation (Saunila, & Ukko, 2014). Strategic innovation planning makes it possible for the company to extract benefits and to optimize resources through the synergy of parallel execution of investment and innovation projects (Van der Panne, Van Beers, & Kleinknecht, 2003). In MTW, the strategic innovation planning is developed by the company’s top management (strategic committee) after seven activities (Bagno, & Faria, 2017) as shown in Figure 3.

In the P cycle (planning), the activities of innovation diagnosis (identification of innovation challenges and opportunities), the strategic innovation goals definition and the strategic innovation plan elaboration take place. In cycle D (Do), the committee performs the activities foreseen in the strategic innovation plan. In cycle C (Check) the committee analyzes the strategic innovation goals accomplishment. In cycle A (Adjust) the committee suggests refinements and improvements in the company’s innovation strategic planning and goals (Bagno, & Faria, 2017; Silva et al., 2018). Evidence of this module implementation shows that the innovation diagnosis allows the strategic committee to define innovation strategies, projects performance indicators and ensures the alignment between the innovation model processes and the company strategy. This alignment is important because it contributes to the standardization of innovation management, including the control and design of innovation projects (Oliveira, Rocha, Camargos, & Bagno, 2016).

The MTW management module is associated with the Pró-Inova topics 3–4 and 7 and focuses on two aims. The first aim is to capacitate companies on new products, processes and businesses evaluation and development. The second aim is to train the management team in managing the program of ideas and projects generation, selection and implementation through the FAZ Program IMS. These activities involve a well-defined process that contributes to engaging people in the company’s innovation activities, and to keep their ideas aligned with the company’s strategy (Van der Panne et al., 2003). To adopt a well-defined process is also recognized as an important way to increase the company’s innovative capacity (Saunila, & Ukko, 2014). In MTW, these innovation management processes are part of the FAZ Program IMS. The company managers (management committee) are responsible to operate the FAZ Program IMS and its set of eight activities (Bagno, & Faria, 2017) as shown in Figure 4.

In the P cycle (Plan), the portfolio management activities (organization, selection and prioritization of ideas and projects) and project management activities (cancellations, suspensions or change of project priorities) take place. The management committee executes both activities by the FAZ Program IMS. The portfolio management uses the funnel method to select the innovative ideas and projects (Clark, & Wheelwright, 1992). This method follows stages of funneling with multiple inputs and intermediate outputs to allocate resources to the most promising ideas and projects (Bagno, Salerno, & Silva, 2017). The project management team uses the stage-gates method (Cooper, 1993) to monitor and control the project deployment process (Utterback, 1970). This method follows a set of stage-gate decisions to manage the phases of ideation, development, implementation and launch of innovation (Bagno et al., 2017). In cycle D (Do), the committee elaborates on budgets and schedules of the selected projects and designates the members of the project team. In cycle C (Check), the committee uses the bubble diagram to prioritize projects of the portfolio and the stage-gates method to verify the fulfillment of the prioritized project budgets and schedules. In cycle A (Adjust), the management committee reports to the strategic committee deviations and failures that occurred in the portfolio and project’s management (Bagno, & Faria, 2017). Evidence of this module implementation shows that the portfolio management and project management systematization stimulates the employees involvement in the program, aligns ideas to strategy, and increases the innovative ideas and projects quality (Liborio et al., 2016). However, practical evidence also reveal that this committee concentrates on a lot of activities, including activities of other modules (e.g. manage the program of generation, selection and implementation of innovative ideas) (Silva et al., 2018).

The MTW project module is associated with the Pró-Inova topics 3 and 4 and aims to qualify and train companies in project management. Project management is relevant for at least two reasons. First, it can avoid those financial constraints increasing the probability of abandoning an innovation project (García-Quevedo, Segarra-Blasco, & Teruel, 2018). Second, project management tools (e.g. stage-gates) contribute in reducing financial losses by allowing management of risks and uncertainties based on partial results, and enabling decision in advance, such as change, interrupt and correct the projects development (Cooper, & Kleinschmidt, 1993). In MTW, the project module achievement depends on technicians and specialists (project team) (Bagno, & Faria, 2017) whose main responsibility is to develop and manage innovation projects (Silva et al., 2018). For this, the project team uses the FAZ Program IMS that systematizes the six activities stage-gates method shown in Figure 5.

The stage-gates method allows companies to make decisions according to the stage of projects, such as eliminating them through internal limitations or interrupting them through external constraints (García-Quevedo et al., 2018). Furthermore, the stage-gates method favors the interaction between the project teams and the strategic and tactical committees (Silva et al., 2018). This contributes to ensuring the alignment between the innovation project development and the company strategic objectives, budgets and schedules (P stages of the strategic and management modules) (Fiemg, 2015). Practical evidence of this module shows that pre-established goals in each stage-gate (development, testing and control of results) avoid technical failures or budget change and favor the project execution on schedule (Oliveira, Liborio, Fujimoto, Silva, Silva, & Laudares, 2016).

The MTW organizational module is associated with the Pró-Inova topics 2 and 6 and aims to capacitate companies into innovation organizations, learning processes elaboration and innovation culture introduction. To develop those capacities is important because the success of the innovative process, and its continuity (Cobo-Benita, Rodríguez-Segura, Ortiz-Marcos, & Ballesteros-Sánchez, 2016) depends much more on people than on financial resources or technological structures (Rothwell, 1992; Cobo-Benita et al., 2016). In MTW, the activities of this module are performed by any employee regardless of their level or function (team organization for innovation). The activities of the organization module are based on the exchange of experimental and technical knowledge to develop innovation skills. Different from the other modules, the activities of the organizational team do not follow a well-defined process (Bagno, & Faria, 2017) as shown in Figure 6.

The Figure also does not show links between the organizational activities and the Pró-Inova topics. The lack of methods, processes and tools to plan, execute, check and adjust the activities makes it more difficult to measure and analyze the implementation of this module (Silva et al., 2018). There are also no links between the organizational module and the strategic, management and project modules. On the other hand, we observed in Figure 7 that the strategic, management and project modules are linked by internal and external inputs and outputs.

We highlight in Figure 7 the activities of registering and selecting ideas. The MTW addresses these activities in the organizational module, but it is the management committee through the FAZ Program IMS that perform this activity. Evidence of this activity implementation show that the FAZ Program IMS makes the registering and selecting of ideas management more agile and transparent, contributing to the involvement of people in the innovation (Oliveira, & Resende, 2016). For the authors, this involvement increases creative capacity and motivation for generating good ideas, and to execute innovation projects with efficiency. Figure 7 shows that MTW follows the characteristics of the innovation management model based on interaction processes (Rothwell, 1992), which are usually implemented gradually (Birkinshaw, & Mol, 2006) with a strong presence of graphical representations (Bagno et al., 2017) and IMS to standardize and systematize its activities or process (Mir, Casadesús, & Petnji, 2016).

3. Methodology

In order to meet the research goal, we developed an intensive-direct-observation between August 2013 and May 2016. We opted for intensive-direct-observation because the researchers could be directly involved with the sources, which does not occur in the extensive-direct-observation. Despite that, we do not take part in the actions of solving the FAZ Program implementation problems, which is a characteristic of the research-action. In our case, we adopted as a data collection technique the participant observation (Michel, 2009). The intensive-direct-observation took one year and was accomplished in each one of the 43 companies that implemented the FAZ program. Through 200 h of meetings, we collected two sets of data. The first set of data is related to the completion of the activities of each module. The second set of data is related to the company’s maturity to innovate. The framework of the research method is summarized in Figure 8.

The diagnosis of maturity to innovate allows the organization to identify its strengths and areas that need improvements, making necessary changes visible, and building a framework for developing strategies to optimize resources (Torres, Camilo, Ziviani, & Parreiras, 2015). Most of the maturity model’s basis come from the capability maturity model integration (CMMI) model (O’Regan, 2017) and address a wide variety of topics (Li, 2007; Serpell, & Alvarez, 2014). In the innovation management, the maturity to innovate is associated with the company’s information technologies (Achi, Salinesi, & Viscusi, 2016), maturity level of their employees (Nauyalis, 2017) and other structural variables and managerial skills (Demir, 2017). Although it is still in the process of improvement, the maturity models for innovation management present a solid and reliable basis for the development and improvement of other maturity models for innovation management (Torres et al., 2015). Therefore, according to the CMMI dimensions, characteristics (O’Regan, 2017) and, applications (Li, 2007) we developed and applied a maturity model based on the analysis of the maturity level of people, processes, tools and innovation (Nauyalis, 2017), classifying the company maturity to innovate according to the Likert scale (Michel, 2009, p. 74) into: very poor; below average; average; above average; and excellent.

The maturity to innovate diagnosis carried out in the FAZ Program consists of a questionnaire with 80 questions (20 per module). The questions answered by 30 companies were related to: planning and management experiences, technical and intellectual capacity, knowledge about innovation, cultural and organizational aspects. After 200 h of meetings, we produced the company’s final report, tabulating the information about the completed activities of each module. Based on this report, we detail all the activities carried out in the companies (e.g. schedule of activities, indicators and targets) and reported to Finep for approval. This approval occurred in 2017. Except for company data, the Finep report is not confidential and is available upon request. Finally, we calculated the FAZ Program success rate according to the Pró-Inova goals. To do this, we compared the number of companies that completed each module to the number of Pró-Inova topics present in these modules.

4. FAZ program implantation results and discussion

The first set of results is associated with the intensive-direct-observation in the 43 companies that implement the FAZ Program and is summarized in Table I. We observe in Table I that 100 percent of the companies completed the FAZ Program IMS training. Second, there is a drop in the percentage of companies that completed the three initial modules of MTW. The percentage of completed activities decreases from 100 percent in the strategic committee module to 74 percent in the management module and to 46 percent in the project module. The organizational module does not take part in this decrease trend for two reasons. First, because its implementation does not depend on inputs from previous modules. Second, because its conclusion does not depend on the execution of all four MTW proposed activities.

During the FAZ Program implementation, we observed that the success in completing the strategic module depends on the management committee performance in the strategic module activities execution, which may have contributed to 100 percent of the companies having completed this module. On the other hand, the management committee at 26 percent of the companies did not complete the activities of the management module, suggesting that the strategic module activities execution impacted the performance of the management module activities. As a result, project teams did not receive enough inputs to complete the project module, making only 46 percent of companies complete this module. Finally, we observed that 95 percent of companies completed at least one activity to stimulate the culture of innovation.

These results show how the activities completion impact the FAZ Program implementation and provide empirical evidence to confirm previous researches of innovative management models in the literature. First, management models innovations (e.g. FAZ Program) may take several years to implement (Birkinshaw, & Mol, 2006). Second, innovative management models involve processes with many parts (modules), which are difficult to separate from each other (Birkinshaw, & Ansari, 2015). Furthermore, the results also provide empirical evidence to refute that the MTW modules implementation can occur separately (Aniceto et al., 2016).

The second set of results is associated with the maturity to innovate diagnosis in 30 companies (see Table). Table II summarizes this result showing the maturity innovation of the committees/teams that completed their respective modules.

Considering only the maturity of the committees/teams that completed the modules, the results show that the committee/team maturity for innovation is relevant to the completion of the strategic and management modules, but not relevant to the completion of the project and organizational modules. First, because the success of the implementation of the project module depends on the inputs that are generated from the successful implementation of the previous modules. Second, because the completion of the organizational module depends on the completion of only one activity. This explains why 95 percent of the companies completed the organizational module even when the maturity of the organizational team was below average, and at an average of 77 percent of them.

In a more detailed analysis of the diagnostic data (Table), we observed that the strategic committee has the highest number of excellent concepts of maturity for innovation (50 percent of the total). Companies whose strategic committee has above average or excellent maturity for innovation have a 1.8 times greater chance of successfully implementing the FAZ Program. We also notice that the strategic committee’s maturity for innovation affects the possibility of the company completing the management module. Diagnostic data reveal that the maturity of the strategic committee is below average in 16 percent of companies that do not complete the management module, but this percentage is only 5 percent in the companies that complete the management module. In relation to the management committee, we observed that the maturity for innovation is above average in 45 percent of the companies that implemented the FAZ Program, above the general average of 25 percent of the committees/teams of the other modules. Among companies that did not complete the project module, project teams’ maturity for innovation is below average or average in 76 percent of them. On the other hand, the completion of the project module is associated with the strategic committee’s maturity for innovation. First, because the strategic committee’s maturity for innovation is below average or average in 82 percent of companies that have not completed this module. Second, because the strategic committee’s maturity for innovation is above average or excellent in 64 percent of the companies that have completed this module. Finally, we analyze the impact of maturity for innovation of the organizational team in the completion of the other modules. While maturity for innovation of organizational teams is above average or excellent in 50 percent of the companies that implemented the three modules, this percentage is 11 percent of the companies that implemented two modules and 6 percent of the companies that implemented only one module. In general, companies whose organizational team has maturity for innovation of above average or excellent have a 1.7 times greater chance of implementing the FAZ Program successfully.

These results provide empirical evidence that confirm how much the companies’ maturity for innovation (Dalmarco, Zaniol, Selao, & Drumm, 2014), especially from the organizational team (Van der Panne et al., 2003; Cobo-Benita et al., 2016) and from top management (strategic committee (Rothwell, 1992; Ortt, & Van der Duin, 2008), increases the chances of implementing innovation in companies successfully. In addition, these results provide information such as the Pró-Inova goals – rate of success (Table III) to show if the FAZ Program results achieved the set of Pró-Inova goals, as well as to point out improvements for the program and the public policies that support innovation.

These results show more than 81 percent of the FAZ Program success rate according to the Pró-Inova goals. These results give transparency and publicity to public spending, as well as to show benefits generated by these expenditures (Negri, & Rauen, 2018). However, the results do not allow us to definitively affirm that Pro-Inova is a successful public innovation incentive policy. First, because the FAZ Program represents only 4 percent of the total IMPs developed by the Brazilian IMSCs (Zen et al., 2014). Second, because the implementation of the innovative management model does not mean innovative products, processes and businesses. The innovations originated from innovative management models usually take several years to happen (Birkinshaw, & Mol, 2006). However, we conclude that 81 percent of the companies were able to adapt their management models, and with this innovative management model, they tend to increase their competitive advantage and sustainability through innovative products, processes or businesses (Birkinshaw, & Ansari, 2015).

However, these conclusions do not address some problems in the MTW structure. Some questions still need an answer. Such as, why the MTW organizational module is not structured in PDCA cycles? Why the organizational module activities are not systematized in the FAZ Program IMS? Why does the management committee carry out activities of other MTW modules? Table IV suggests some answers aiming to discuss how to improve the MTW and FAZ Program.

These suggestions can help to adjust the structure of the MTW as well as to improve the effectiveness of the FAZ Program implementation. For a more effective implementation we suggest not starting the FAZ Program in the following three situations: before the maturity for innovation diagnosis and the strategic innovation plan elaboration; without obtaining the minimum human resources required to complete the activities of the management committee; and without the minimum maturity for innovation of the project team. Taken together, we expect that these improvements will contribute to increasing the return on the public policy that supports the development and implementation of the FAZ Program.

5. Conclusions

Pro-Inova (Finep, 2010) does not correct the relative disconnection between universities and companies in developing innovation in Brazil (Lee, 2013), but finds in the FAZ Program a successful case of this connection, and brings empirical evidence that public policies that support innovation can bring benefits to the economy (Monte Silva, & Guimarães, 2016). These benefits occur because they promote innovations, as long as innovation depends not only on operational, business and technological aspects but also on management (Zawislak et al., 2013). If on the one hand it is impossible to predict whether the benefits of the FAZ Program (an innovative innovation management model) will exceed its costs (Birkinshaw, & Mol, 2006); on the other hand, the FAZ Program IMS increases the company’s chances of receiving benefits from the innovation management (Mir et al., 2016).

We show that the maturity for innovation diagnosis is useful to evaluate the company’s innovation capacity (Bagno et al., 2017), as well as to predict the company’s chances of success in implementing the FAZ Program. We conclude that the success of public policies that support innovation does not depend exclusively on IMPs and IMS, but also on the maturity for innovation of the companies that implement these programs.

We bring empirical evidence to state that companies’ maturity for innovation is decisive for the successful implementation of these innovative management models. The literature on innovative management models states that innovative management models have been a key-factor of success for the competitive advantage of many companies (Birkinshaw, & Mol, 2006) and that innovative management models facilitate innovative and promising business initiatives implementation (Birkinshaw, & Ansari, 2015). Therefore, we argue that a company’s innovation or competitive advantages depends first on maturity to innovate and second on the management models. In this way, we suggest that future research works seek the promotion of activities for developing the maturity for innovation before starting the program implementation. We also suggest defining processes and systematizing the MTW organizational module in PDCA cycles.

Figures

The conceptual framework of the FAZ program

Figure 1

The conceptual framework of the FAZ program

Interaction points of the two wheels model

Figure 2

Interaction points of the two wheels model

Strategic committee activities

Figure 3

Strategic committee activities

Management committee activities

Figure 4

Management committee activities

Project team activities of stage-gates method

Figure 5

Project team activities of stage-gates method

Activities of the organization team

Figure 6

Activities of the organization team

MTW modules and activities links

Figure 7

MTW modules and activities links

Research method framework

Figure 8

Research method framework

Modules completed by the 43 companies

Module Modules completed Percentage of completion
Strategic 43 100
Management 32 74
Project team 20 46
Organizational 41 95

Note: 100 percent of the companies completed the innovation management system (IMS) training

Maturity to innovate of the committees/teams that completed their respective modules

Maturity to innovate Strategic Management Projects Organizational
Below average 10% 10% 9% 17%
Average 50% 48% 64% 60%
Above average 33% 37% 18% 23%
Excellent 7% 5% 9% 0%
Total of companies 30 21 11 30

FAZ program results achieved according to Pró-Inova goals

Pró-Inova topic FAZ Program (MTW module) Successful implantations Rate of success (%)
(1) Strategic planning Strategic 43 100
(2) Organization for innovation Organizational 41 95
(3a) Methodologies and tools for evaluating and developing new products and processes Management and project 60
 Project portfolio management Management 32 74
 Innovation project management Project 20 47
(4a) Methodologies and tools for evaluating and developing new business Management and project 60
 Project portfolio management Management 32 74
 Innovation project management Project 20 47
(5) Competitive intelligence systems Management 32 74
(6) Structuring of learning processes Organizational 41 95
(7a) Structuring innovative environments and stimulating creativity Management and organizational 85
 Register and select ideas (e.g. generation, selection and implementation of innovative ideas and projects) Management 32 74
 Create innovative environments and stimulate creativity Organizational 41 95
(8) Development of information systems for innovation (e.g. public news and innovation funding programs) Management 32 74
Total 4 43 81

Note: aThe success rate of Pró-Inova topics that are in two MTW modules is calculated by the average

How to improve the MTW and the FAZ program

Module Problem Suggestion
Strategic The innovation strategic plan impacts the conclusion of the other modules Diagnose the maturity for innovation and elaborate the innovation strategic plan before starting the program
Management Execute strategic and organizational module activities Indicate minimum human resources required to execute the module
Project Maturity for innovation is below average and average in 73% of the teams Promote maturity for innovation to similar levels of other committees
Organizational Lack of systematized processes and connection to other modules Structuring in PDCA cycles and systematization in the IMS

Diagnosis and activities completion data

Company IMS T SM MM PM OM SC MC PT OT Faz Program start date
 4 Yes Yes Yes Yes Yes Average Average Average Average November 2013
 5 Yes Yes Yes No Yes Above average Average Average Average June 2014
 6 Yes Yes Yes Yes Yes Excellent Above average Above average Above average July 2014
 7 Yes Yes Yes No Yes Average Average Average Average July 2014
 8 Yes Yes No No Yes Average Above average Average Average July 2014
 9 Yes Yes Yes No Yes Above average Above average Below average Average July 2014
10 Yes Yes Yes Yes Yes Above average Excellent Excellent Above average July 2014
12 Yes Yes No No Yes Below average Average Average Below average September 2014
13 Yes Yes Yes Yes Yes Above average Above average Average Above average September 2014
17 Yes Yes Yes No Yes Average Above average Above average Average November 2014
18 Yes Yes Yes No Yes Above average Average Above average Average November 2014
19 Yes Yes Yes Yes Yes Above average Above average Average Average November 2014
20 Yes Yes Yes No Yes Average Above average Average Average November 2014
21 Yes Yes No No Yes Average Above average Average Average December 2014
22 Yes Yes Yes Yes Yes Average Below average Below average Below average December 2014
24 Yes Yes Yes Yes Yes Above average Average Average Average January 2015
26 Yes Yes No Yes Yes Average Average Average Average February 2015
27 Yes Yes No No Yes Excellent Above average Above average Above average March 2015
28 Yes Yes No No Yes Above average Above average Above average Above average April 2015
29 Yes Yes Yes No Yes Below average Below average Below average Below average April 2015
30 Yes Yes Yes No Yes Average Average Below average Average April 2015
31 Yes Yes No No Yes Below average Average Average Average May 2015
32 Yes Yes No No Yes Average Above average Above average Average May 2015
33 Yes Yes Yes Yes Yes Above average Above average Above average Above average May 2015
34 Yes Yes No No Yes Average Average Above average Average June 2015
35 Yes Yes Yes No Yes Average Above average Average Average June 2015
36 Yes Yes Yes No Yes Average Average Average Average July 2015
37 Yes Yes Yes No Yes Average Average Average Below average July 2015
38 Yes Yes Yes Yes Yes Above average Average Average Below average August 2015
40 Yes Yes Yes Yes Yes Average Average Average Above average November 2015
 1 Yes Yes Yes Yes Yes August 2013
 2 Yes Yes Yes Yes Yes August 2013
 3 Yes Yes Yes No Yes August 2013
11 Yes Yes Yes No Yes July 2014
14 Yes Yes Yes Yes Yes October 2014
15 Yes Yes Yes Yes Yes October 2014
16 Yes Yes Yes Yes Yes November 2014
23 Yes Yes No Yes Yes January 2015
25 Yes Yes No No Yes February 2015
39 Yes Yes Yes No Yes September 2015
41 não sim sim sim não December 2015
42 não sim sim sim não May 2016

Notes: IMS T, innovation management system training; SM, strategic modules; MM, management module; PM, projects module; OM, organizational modules; SC, strategic committee; MC, management committee; TC, projects team; and OT, organizational team

Appendix

Table AI

References

Achi, A., Salinesi, C., & Viscusi, G. (2016). Innovation capacity and the role of information systems: a qualitative study. Journal of Management Analytics, 3(4), 333360.

Aniceto, M. D., Bagno, R. B., Alfradique, P. H. M., & de Souza, G. F. M. (2016). Avaliação de um modelo de sistema de gestão da inovação a partir da experiência de implantação em empresas do estado de Minas Gerais. Gestão da inovação de produtos e serviços: pesquisas e práticas atuais 2016 anais do X Workshop do instituto de Inovação e Gestão de Desenvolvimento de Produto (pp. 110–114). em Betim.

Arbix, G., Salerno, M. S., Amaral, G., & Lins, L. m. (2017). Avanços, equívocos e instabilidade das Poláticas de Inovação no Brasil. Novos Estudos. CEBRAP, 36(3), 927.

Bagno, R., Leiva, T., & Oliveira, L. (2016). Innovation management: Lessons learned from innovation diagnostic tools. Product: Management & Development Journal, 14(1), 1221.

Bagno, R. B., & Faria, A. F. (2017). O modelo das duas rodas. Minas Gerais: Editora Viçosa.

Bagno, R. B., Salerno, M. S., & Silva, D. O. (2017). Models with graphical representation for innovation management: a literature review. R&D Management, 47(4), 637653.

Birkinshaw, J., & Ansari, S. (2015). Understanding management models. Going beyond “what” and “why” to “how” work gets done in organization. In Foss, J. N., & Saebi, T. (Eds), The Organizational Dimension to Business Model Innovation (pp. 85–103). New York, NY: Oxford University Press.

Birkinshaw, J. M., & Mol, M. J. (2006). How management innovation happens. MIT Sloan Management Review, 47(4), 8188.

Brandao, L. A., da Silva Gomes, A., & Segundo, G. S. A. (2015). Política de competitividade e desenvolvimento: uma análise da maturidade da gestão de inovação de empresas do sul da Bahia. Revista Gestão Industrial, 11(3), 105124.

Chaves, C. V., Carvalho, S. S. M., Silva, L. A., Teixeira, T. C., & Bernardes, P. (2012). The point of view of firms in Minas Gerais about the contribution of universities and research institutes to R&D activities. Research Policy, 41(9), 16831695.

Clark, K. B., & Wheelwright, S. C. (1992). Structuring the development funnel. In Wheelwright, S. C., & Clark, K. B. (Eds), Revolutionizing Product Development (pp. 111–132). New York, NY: The Free Press.

Cobo-Benita, J. R., Rodríguez-Segura, E., Ortiz-Marcos, I., & Ballesteros-Sánchez, L. (2016). Innovation projects performance: analyzing the impact of organizational characteristics. Journal of Business Research, 69(4), 13571360.

Cooper, R. G. (1993), Winning at new products: accelerating the process from Idea to launch, Reading, MA: Addison-Wesley Publishing.

Cooper, R. G., & Kleinschmidt, E. J. (1993). Stage gate systems for new product success. Marketing Management, 1(4), 2029.

D’Barsoles, G. W., Iata, C., & Lezana, Á.G.R. (2017). Projeto NAGI-IEL/SC: núcleo de apoio a gestão da inovação. Revista Ciencias Estratégicas, 25(37), 111137.

Dalmarco, G., Zaniol, C., Selao, D., & Drumm, E. C. (2014). Empreendedor-coordenador brasileiro: Análise através da metáfora médico e monstro. Revista Eletrônica de Administração e Turismo, 4(2), 294312.

Demir, F. (2017). Measuring strategic management maturity: an empirical study in Turkish public and private sector organizations. World Academy of Science, Engineering and Technology, International Journal of Social, Behavioral, Educational, Economic, Business and Industrial Engineering, 11(11), 25302537.

Feuerschütte, S. G., da Cunha Lemos, D., Hoffmann, M. G., & Fernandes, R. F. (2017). Desenvolvimento de uma ferramenta para identificação de padrões culturais para a inovação: relato da trajetória de um projeto de cooperação técnico-científica. Race: revista de administração, contabilidade e economia, 16(1), 327352.

Fiemg (2015), FAZ Program – Inovação para Resultados: guia da gestão da inovação, Belo Horizonte, Brasil: , Federação das Indústrias de Minas Gerais.

Finep (2010). Chamada pública MCT/FINEP – At – pró-inova – núcleos de apoio à gestão da inovação. Financiadora de Estudos e Projetos, Rio de Janeiro, Available from: www.finep.gov.br/chamadas-publicas/chamadapublica/500 (accessed 9 January 2019).

Fischer, B. B., Schaeffer, P. R., & Vonortas, N. S. (2019). Evolution of university-industry collaboration in Brazil from a technology upgrading perspective. Technological Forecasting and Social Change, 145(8), 330340.

García-Quevedo, J., Segarra-Blasco, A., & Teruel, M. (2018). Financial constraints and the failure of innovation projects. Technological Forecasting and Social Change, 127(2), 127140.

García-Sánchez, E., García-Morales, V. J., & Martín-Rojas, R. (2018). Influence of technological assets on organizational performance through absorptive capacity, organizational innovation and internal labour flexibility. Sustainability, 10(3), 125.

IBGE (2014). Cadastro Central de Empresas. Instituto Brasileiro de Geografia e Estatística, Rio de Janeiro, Available from: https://sidra.ibge.gov.br/pesquisa/cempre (accessed 9 January 2019).

IBGE (2016). Pesquisa de Inovação 2014. Instituto Brasileiro de Geografia e Estatística, Rio de Janeiro, Available from: https://biblioteca.ibge.gov.br/visualizacao/livros/liv99007.pdf (accessed 9 January 2019).

Lee, K. (2013), Schumpeterian Analysis of Economic Catch-Up: Knowledge, Path-Creation, and the Middle-Income Trap. New York, NY: Cambridge University Press.

Li, J. (2007). Application of CMMI in innovation management. Wireless Communications, Networking and Mobile Computing 2007 Proceedings of the WiCom International Conference, (pp. 4966–4969). Shanghai.

Liborio, M. P., Fujimoto, N., Silva, I. S., Silva, W., Laudares, S., & Martins, C. A. S. P. (2016). Programa inovar de gestão da inovação: características e dificuldades em PD&I no setor de telecomunicações. Gestão da inovação de produtos e serviços: pesquisas e práticas atuais 2016 anais do X Workshop do instituto de Inovação e Gestão de Desenvolvimento de Produto, em Betim, 228–232.

Mattos, J. F., Stoffel, H. R., & Teixeira, R. A. (2010), Mobilização Empresarial pela Inovação: cartilha: gestão da inovação, Confederação Nacional da Indústria, Brasília, Available from: www.ipdeletron.org.br/wwwroot/pdf-publicacoes/8/cartilha_gestao_inovacao_cni.pdf (accessed 9 January 2019).

Michel, M. H. (2009), Metodologia e pesquisa científica em ciências sociais: um guia para acompanhamento da disciplina e elaboração de trabalhos monográficos, São Paulo: Editora Atlas.

Mir, M., Casadesús, M., & Petnji, L. H. (2016). The impact of standardized innovation management systems on innovation capability and business performance: an empirical study. Journal of Engineering and Technology Management, 41(3), 2644.

Monte Silva, L., & Guimarães, P. B. V. (2016). Law and innovation policies: An analysis of the mismatch between innovation public policies and their results in Brazil. Law and Development Review, 9(1), 95151.

Nauyalis, C. (2017). A new framework for assessing your innovation program: introducing the innovation management maturity model by plane view. Available from: www2.planview.com/im3/docs/Planview-Innovation-Maturity-Model.pdf (accessed 9 January 2019).

Negri, F., & Rauen, A. T. (2018). Innovation policies in Brazil during the 2000s: the need for new paths. Ipea Texto para discussão, 235(1), 152.

O’Regan, G. (2017). “Capability maturity model integration. In O’Regan, G. (Ed.), Concise guide to software engineering, (255–277). Cham: Springer.

Oliveira, C. C., Liborio, M. P., Fujimoto, N. A., Silva, I. S., Silva, W. L., & Laudares, S. (2016). Desenvolvimento de sistema de atendimento a incidentes em redes de telecomunicações. Gestão da inovação de produtos e serviços: pesquisas e práticas atuais 2016 anais do X Workshop do instituto de Inovação e Gestão de Desenvolvimento de Produto (pp. 245–249). em Betim.

Oliveira, M., & Resende, D. (2016). Programa inovação: fonte de ideias para melhoria contínua. Gestão da inovação de produtos e serviços: pesquisas e práticas atuais 2016 anais do X Workshop do instituto de Inovação e Gestão de Desenvolvimento de Produto, (pp. 260–263). em Betim.

Oliveira, O. G., Rocha, S. D. S., Camargos, M. D., & Bagno, R. B. (2016). Estabelecendo um sistema de gestão da inovação: o caso da Mediphacos. Gestão da inovação de produtos e serviços: pesquisas e práticas atuais 2016 anais do X Workshop do instituto de Inovação e Gestão de Desenvolvimento de Produto (pp. 250-254). em Betim.

Ortt, J. R., & Van der Duin, P. A. (2008). The evolution of innovation management towards contextual innovation. European Journal of Innovation Management, 11(4), 522538.

Rapini, M. S., Chaves, C. V., Albuquerque, E. M., Silva, L. A., Souza, S. G. A., Righi, H. M., & Cruz, W. M. S. (2009). University-industry interactions in an immature system of innovation: Evidence from Minas Gerais, Brazil. Science and Public Policy, 36(5), 373386.

Rothwell, R. (1992). Successful industrial innovation: critical factors for the 1990s. R&D Management, 22(3), 221240.

Santos, J., Motta, K., Luna, M., Barbosa, A. G., Brito, E., Nunes, M., & Romero, F. (2014). Pró-Inova: projecto to implement innovation management in industrial SMEs-Rio Grande do Norte, Brazil. ICOPEV 2014 – 2nd proceedings of the International Conference on Project Evaluation, (pp. 283–288). Guimaraes.

Saunila, M., & Ukko, J. (2014). Intangible aspects of innovation capability in SMES: impacts of size and industry. Journal of Engineering and Technology Management, 33(3), 3246.

Serpell, A., & Alvarez, R. (2014). A systematic approach for evaluating innovation management in construction companies. Procedia Engineering, 85(17), 464472.

Silva, I. S., Libório, M. P., Bernardes, P., & Ramalho, F. D. (2018). O modelo das duas rodas: uma referência para o sistema de gestão da inovação em pequenas e médias empresas. Revista Brasileira de Inovação, 17(2), 445452.

Sokovic, M., Pavletic, D., & Pipan, K. K. (2010). Quality improvement methodologies–PDCA cycle, RADAR matrix, DMAIC and DFSS. Journal of Achievements in Materials and Manufacturing Engineering, 43(1), 476483.

Suzigan, W., & Albuquerque, E. M. (2011). The underestimated role of universities for the Brazilian system of innovation. Brazilian Journal of Political Economy, 31(1), 330.

Torres, H. K. M. L., Camilo, R. D., Ziviani, F., & Parreiras, F. S. (2015). Dimensões e características dos modelos de maturidade e de mensuração da gestão da inovação: uma revisão sistemática da literatura. in anais do IV Simpósio Internacional de Gestão de Projetos, Inovação e Sustentabilidade, São Paulo.

Utterback, J. M. (1970). Process of innovation – a study of origination and development of ideas for new scientific instruments. IEEE Transactions on Aerospace and Electronic Systems, 18(4), 124131.

Van der Panne, G., Van Beers, C., & Kleinknecht, A. (2003). Success and failure of innovation: a literature review. International Journal of Innovation Management, 7(3), 309338.

Zawislak, P. A., Alves, A. C., Tello-Gamarra, J., Barbieux, D., & Reichert, F. M. (2013). Influences of the internal capabilities of firms on their innovation performance: A case study investigation in Brazil. International Journal of Management, 30(1), 329348.

Zen, A. C., Jaramillo, A. I. L., Dambros, Â. M. F., Menezes, D. C., & Machado, B. D. (2014). O papel do governo no apoio à gestão de inovação das empresas: um estudo exploratório do programa de núcleos de apoio à gestão de inovação. Proceedings of XXIV seminário nacional de parques tecnológicos e incubadoras de empresas, Belém.

Zen, A. C., Machado, B. D., Jaramillo López, A. I., Borges, M. C., & Callegaro-de-Menezes, D. (2017). Rota da inovação: uma proposta de metodologia de gestão da inovação. Revista de administracao contemporânea, 21(6), 875892.

Zuniga, P., Negri, F., Dutz, M. A., Pilat, D., & Rauen, A. (2016). Conditions for innovation in Brazil: a review of key issues and policy challenges. Ipea Texto para discussão, 218(1), 1102.

Acknowledgements

This study was financed in part by: Euvaldo Lodi Institute of Minas Gerais; National Scientific and Technological Development of Brazil (CNPq), Process No. 311032/2016-8; Coordination for the Improvement of Personnel in Higher Education – Brasil (CAPES) – Finance Code 001; Funding Authority for Studies and Projects (Finep).

Corresponding author

Matheus Pereira Libório can be contacted at: m4th32s@gmail.com

Related articles