The impact of crowdsourcing and user-driven innovation on R&D departments’ innovation activity: Application of multivariate correspondence analysis

Abstract

Research background: In enterprise practices, innovation management is taking on more and more open forms. New, creative sources of inspiration are being sought. More and more companies are opening up to cooperation with external people or entities, thus entering a path of open innovation and crowdsourcing. This development is facilitated by increasingly large numbers of online and virtual communities. The idea is to use the potential of the crowd — collective intelligence and creativity. As the strategy is still developing, knowledge in this area is limited. There is a paucity of research on the impact of crowdsourcing on R&D departments and their innovative activities.

Purpose of the article: The study aims to determine whether the use of customer support in the form of crowdsourcing affects the innovative activities of R&D departments that implement user-driven innovation (UDI).

Methods: An original questionnaire was used for the study. Fifty-seven R&D departments in Poland participated in the research. A correspondence analysis, performed on the basis of the Burt matrix, was applied for the analysis. The authors also used Cramer's V correlation coefficients. Examination of the acquired correlation coefficients reveals the existence of four distinct categories of enterprises regarding the usage of crowdsourcing and implementation of product innovations, completed R&D projects, employment in R&D departments, and percentage of revenue allocating in R&D works.

Findings & value added:  Research has shown that R&D departments positively evaluate the effects of using UDI in market research analyses of customer purchasing behavior. It helps to create or improve products or services offered on the market, especially in the field of customization, using the information from national research and development units at the same time. Their purpose is mainly to increase the product range. The findings can help academics and practitioners improve the body of knowledge about the determinants of innovation activity, especially concerning relationships with customers and user-driven innovation practices.

Keywords

crowdsourcing, innovation, user-driven innovation, Internet, research and development (R&D), marketing orientation, customer purchase behavior, multivariate correspondence

References

1. Abdelaty, H., & Weiss, D. (2023). Coping with the heterogeneity of external knowledge sources: Corresponding openness strategies and innovation performance. Journal of Innovation & Knowledge, 8(4), 100423. DOI: https://doi.org/10.1016/j.jik.2023.100423
   View in Google Scholar
2. Abdi, H. (2003). Multivariate analysis. In M. Lewis-Beck, A. Bryman & T. Futing (Eds.). Encyclopaedia for research methods for the social sciences, (pp. 699–702). Thousand Oaks, CA: Sage.
   View in Google Scholar
3. Abdi, H., & Valentin, D. (2007). Multiple correspondence analysis. In N. J. Salkind (Ed.). Encyclopaedia of measurement and statistics, (pp. 651–657). Thousand Oaks, CA: Sage.
   View in Google Scholar
4. Abdi, H., Williams, L. J., & Valentin, D. (2013). Multiple factor analysis: Principal component analysis for multi-table and multi-block data sets. Wiley Interdisciplinary Reviews: Computational Statistics, 5, 149–179. DOI: https://doi.org/10.1002/wics.1246
   View in Google Scholar
5. Acar, O. A. (2019). Motivations and solution appropriateness in crowdsourcing challenges for innovation. Research Policy, 48(8), 103716. DOI: https://doi.org/10.1016/j.respol.2018.11.010
   View in Google Scholar
6. Afuah, A., & Tucci, Ch. L. (2012). Crowdsourcing as a solution to distant search. Academy of Management, 3(37), 355–375. DOI: https://doi.org/10.5465/amr.2010.0146
   View in Google Scholar
7. Ang, K. L. M., Seng, J. K. P., & Ngharamike, E. (2022). Towards crowdsourcing Internet of Things (Crowd-IoT): Architectures, security and applications. Future Internet, 14(2), 49. DOI: https://doi.org/10.3390/fi14020049
   View in Google Scholar
8. Bagheri, S. K., Raoufi, P., Eshtehardi, M. S. A., Shaverdy, S., Akbarabad, B. R., Moghaddam, B., & Mardani, A. (2018). Using the crowd for business model innovation: The case of Digikala. R&D Management, 50(1), 3–17. DOI: https://doi.org/10.1111/radm.12353
   View in Google Scholar
9. Bai, W., Lee, D., & Lee, K. Y. (2017). Stochastic dynamic AC optimal power flow based on a multivariate short-term wind power scenario forecasting model. Energies, 10, 2138. DOI: https://doi.org/10.3390/en10122138
   View in Google Scholar
10. Brem, A., Bilgram, V., & Marchuk, A. (2019). How crowdfunding platforms change the nature of user innovation - from problem solving to entrepreneurship. Technological Forecasting and Social Change, 144, 348360. DOI: https://doi.org/10.1016/j.techfore.2017.11.020
    View in Google Scholar
11. Cacuci, D. G. (2021). Fourth-order comprehensive adjoint sensitivity analysis (4th-CASAM) of response-coupled linear forward/adjoint systems: I. Theoretical framework. Energies, 14, 3335. DOI: https://doi.org/10.3390/en14113335
    View in Google Scholar
12. Callaghan, Ch. W. (2020). The ‘death of innovation’ paradox, R&D and the scientific potential of crowdsourcing. African Journal of Science, Technology, Innovation and Development, 12(2), 141–150. DOI: https://doi.org/10.1080/20421338.2019.1634899
    View in Google Scholar
13. Chai, L., Sun, H., & Wang, Z. (2022). An error consistency based approach to answer aggregation in open-ended crowdsourcing. Information Sciences, 608, 1029–1044. DOI: https://doi.org/10.1016/j.ins.2022.07.001
    View in Google Scholar
14. Chan, K. W., Li, S. Y., & Zhu, J. J. (2018). Good to be novel? Understanding how idea feasibility affects idea adoption decision making in crowdsourcing. Journal of Interactive Marketing, 43, 52–68. DOI: https://doi.org/10.1016/j.intmar.2018.01.001
    View in Google Scholar
15. Chesbrough, H. (2006). Open innovation: A new paradigm for understanding industrial innovation. In H. Chesbrough, W. Vanhaverbeke & J. West (Eds.). Open innovation: Researching a new paradigm (pp. 43–62). Oxford: Oxford University Press. DOI: https://doi.org/10.1093/oso/9780199290727.001.0001
    View in Google Scholar
16. Chesbrough, H., & Bogers, M. (2014). Explicating open innovation: Clarifying an emerging paradigm for understanding innovation. In H. Chesbrough, W. Vanhaverbeke & J. West (Eds.). New frontiers in open innovation, (pp. 3–28). Oxford: Oxford University Press. DOI: https://doi.org/10.1093/acprof:oso/9780199682461.003.0001
    View in Google Scholar
17. Chesbrough, H. W. (2003). Open innovation. The new imperative for creating and profiting from technology. Boston: Harvard Business School Press.
    View in Google Scholar
18. Ciasullo, M. V., Carli, M., Lim, W. M., & Palumbo, R. (2022). An open innovation approach to co-produce scientific knowledge: An examination of citizen science in the healthcare ecosystem. European Journal of Innovation Management, 25(6), 365–392. DOI: https://doi.org/10.1108/EJIM-02-2021-0109
    View in Google Scholar
19. Cricelli L., Grimaldi M., & Vermicelli, S. (2022). Crowdsourcing and open innovation: a systematic literature review, an integrated framework and a research agenda. Review Management Science, 16, 1269–1310. DOI: https://doi.org/10.1007/s11846-021-00482-9
    View in Google Scholar
20. de Mattos, C. A., Kissimoto, K. O., & Laurindo. K. O. (2018). The role of information technology for building virtual environments to integrate crowdsourcing mechanisms into the open innovation process. Technological Forecasting and Social Change, 129, 143–153. DOI: https://doi.org/10.1016/j.techfore.2017.12.020
    View in Google Scholar
21. Devece, C., Palacios, D., & Ribeiro-Navarrete, B. (2019). The effectiveness of crowdsourcing in knowledge-based industries: The moderating role of transformational leadership and organisational learning. Economic Research-Ekonomska Istraživanja, 32(1), 335–351. DOI: https://doi.org/10.1080/1331677X.2018.1547204
    View in Google Scholar
22. Dorado, J., & Almendros, G. (2021). Organo-mineral interactions involved in herbicide sorption on soil amended with peats of different maturity degree. Agronomy, 11, 869. DOI: https://doi.org/10.3390/agronomy11050869
    View in Google Scholar
23. Egger, R., Gula, I., & Walcher, D. (2016). Open toursim: Open innovation, crowdsourcing and co-creation challenging the tourism industry. Springer. DOI: https://doi.org/10.1007/978-3-642-54089-9
    View in Google Scholar
24. Estellés-Arolas, E., & González-Ladrón-de-Guevara, F. (2012). Towards an integrated crowdsourcing definition. Journal of Information Science, 38(2), 189–200. DOI: https://doi.org/10.1177/0165551512437638
    View in Google Scholar
25. Estellés-Arolas, E., Navarro-Giner, R., & González-Ladrón-de-Guevara, F. (2015). Crowdsourcing fundamentals: Definition and typology. In F. Garrigos-Simon, I. Gil-Pechuán & S. Estelles-Miguel (Eds). Advances in crowdsourcing (pp. 33–48). Springer. DOI: https://doi.org/10.1007/978-3-319-18341-1_3
    View in Google Scholar
26. Felin, T., Lakhani, K., & Tushman, M. (2017). Firms, crowds, and innovation. Strategic Organization, 15(2), 119–140. DOI: https://doi.org/10.1177/1476127017706610
    View in Google Scholar
27. Fiechter, J. L., & Kornell, N. (2021). How the wisdom of crowds, and of the crowd within, are affected by expertise. Cognitive Research: Principles and Implications, 6, 5. DOI: https://doi.org/10.1186/s41235-021-00273-6
    View in Google Scholar
28. Flores, C. C., & Rezende, D. A. (2022). Crowdsourcing framework applied to strategic digital city projects. Journal of Urban Management, 11(4), 467–478. DOI: https://doi.org/10.1016/j.jum.2022.08.004
    View in Google Scholar
29. Greenacre, M. J. (2007). Correspondence analysis in practice. Boca Raton (FL): Chapman & Hall/CRC. DOI: https://doi.org/10.1201/9781420011234
    View in Google Scholar
30. Gurca, A., Bagherzadeh, M., & Velayati, R. (2023). Aligning the crowdsourcing type with the problem attributes to improve solution search efficacy. Technovation, 119, 102613. doi:10.1016/j.technovation.2022.102613. DOI: https://doi.org/10.1016/j.technovation.2022.102613
    View in Google Scholar
31. Howe, J. (2006). The rise of crowdsourcing. Wired Magazine, 14, 1–5.
    View in Google Scholar
32. Howe, J. (2008). Crowdsourcing: How the power of the crowd is driving the future of business. Random House.
    View in Google Scholar
33. Hwang, H., Tomiuk, M. A., & Takane. Y. (2009). Correspondence analysis, multiple correspondence analysis and recent developments. In R. Millsap & A. Maydeu-Olivares (Eds.). Handbook of quantitative methods in psychology. London: Sage Publications. DOI: https://doi.org/10.4135/9780857020994.n11
    View in Google Scholar
34. Jespersen, K. R. (2018). Crowdsourcing design decisions for optimal integration into the company innovation system. Decision Support System, 115, 52–63. DOI: https://doi.org/10.1016/j.dss.2018.09.005
    View in Google Scholar
35. Jo, H., & Bang, Y. (2023). Factors influencing continuance intention of participants in crowdsourcing. Humanities Social Science Communications 10, 824. DOI: https://doi.org/10.1057/s41599-023-02335-0
    View in Google Scholar
36. Jugend, D., De Camargo Fiorini, P., Armellini, F., & Ferrari, A.G. (2020). Public support for innovation: A systematic review of the literature and implications for open innovation. Technological Forecasting and Social Change, 156, 119985. DOI: https://doi.org/10.1016/j.techfore.2020.119985
    View in Google Scholar
37. Kaewchur, O., Anussornnitisarn, P., & Pastuszak, Z. (2013). The mediating role of knowledge sharing on information technology and innovation. International Journal of Managements, Knowledge and Learning, 2(2), 227–242.
    View in Google Scholar
38. Karachiwalla, R., & Pinkow, F. (2021). Understanding crowdsourcing projects: A review on the key design elements of a crowdsourcing initiative. Creativity and Innovation Management, 30(3), 563–584. DOI: https://doi.org/10.1111/caim.12454
    View in Google Scholar
39. Kearney, M., Cramer’s, W., & Sage V. (2017). Encyclopedia of communication. Research Methods Publisher: Sage Editors.
    View in Google Scholar
40. Khan, V. J., Papangelis, K., Markopoulos, P., & Lykourentzou. I. (2019). Marco-task crowdsourcing: engaging the crowds to address complex problems. Springer. DOI: https://doi.org/10.1007/978-3-030-12334-5
    View in Google Scholar
41. Khasraghi, H. J., & Mohammadi, S. (2012). An innovative crowdsourcing approach for Amazon mechanical turk. International Journal of Computer Applications, 52(4), 20–13. DOI: https://doi.org/10.5120/8190-1556
    View in Google Scholar
42. Kokkoris, I. P., Mallinis, G., Bekri, E. S., Vlami, V., Zogaris, S., Chrysafis, I., Mitsopoulos, I., & Dimopoulos, P. (2020). National set of MAES indicators in Greece: Ecosystem services and management implications. Forests, 11, 595. DOI: https://doi.org/10.3390/f11050595
    View in Google Scholar
43. Kroonenberg, P. M., & Greenacre, M. (2004). Correspondence analysis. Wiley interdisciplinary reviews computational statistics. Wiley.
    View in Google Scholar
44. Larrick, R. P., Mannes, A. E., & Soll, J. B. (2012). The social psychology of the wisdom of crowds. In J. I. Krueger (Ed.). Frontiers of social psychology: Social psychology and decision making. Philadelphia: Psychology Press.
    View in Google Scholar
45. Laursen, K., & Salter, A. (2004). Searching high and low: What types of firms use universities as a source of innovation? Research Policy, 33(8), 1201–1215. DOI: https://doi.org/10.1016/j.respol.2004.07.004
    View in Google Scholar
46. Lin, L., Shen, H., Liu, S., Xu, L., & Cheng X. (2022). On the cybernetics of crowdsourcing innovation: A process model. IEEE Access,10, 27255–27269. DOI: https://doi.org/10.1109/ACCESS.2022.3154101
    View in Google Scholar
47. Luo, Ch., Lan, Y., Luo, X., & Li, H. (2021). The effect of commitment on knowledge sharing: An empirical study of virtual communities. Technological Forecasting and Social Change, 163, 120438. DOI: https://doi.org/10.1016/j.techfore.2020.120438
    View in Google Scholar
48. Mangiaracina, R., Perego, A., Seghezzi, A., & Tumino, A. (2019). Innovative solutions to increase last-mile delivery efficiency in B2C e-commerce: A literature review. International Journal of Physical Distribution and Logistics Management, 49(9), 901–920. DOI: https://doi.org/10.1108/IJPDLM-02-2019-0048
    View in Google Scholar
49. Mariani, A., Annunziata A., Nacchia, F., & Vastola, A. (2016). Crowdsourcing in wine business: Co-creation and fundraising experiences. In G. Szolnoki, L. Thach & D. Kolb (Eds.). Successful social media and ecommerce strategies in the wine industry (pp. 93–114). Book Series: Palgrave Pivot. DOI: https://doi.org/10.1057/9781137602985_6
    View in Google Scholar
50. Martinez, M. G. (2017). Inspiring crowdsourcing communities to create novel solutions: Competition design and the mediating role of trust. Technological Forecasting and Social Change, 117, 296–304. DOI: https://doi.org/10.1016/j.techfore.2016.11.015
    View in Google Scholar
51. Mladenow, A., Bauer, C., & Strauss, C. (2014). Social crowd integration in new product development: Crowdsourcing communities nourish the open innovation paradigm. Global Journal of Flexible Systems Management, 15(1), 77–86. DOI: https://doi.org/10.1007/s40171-013-0053-6
    View in Google Scholar
52. Mortara, L., Napp, J., Ford, S., & Minshall, T. (2011). Open innovation activities to foster corporate entrepreneurship. In L. Cassia, T. Minola & S. Paleari (Eds.). Entrepreneurship and technological change. Cheltenham: Edward Elgar. DOI: https://doi.org/10.4337/9781781002018.00024
    View in Google Scholar
53. Mubarak, M. F., & Petraite, M. (2020). Industry 4.0 technologies, digital trust and technological orientation: What matters in open innovation? Technological Forecasting and Social Change, 161, 120332. DOI: https://doi.org/10.1016/j.techfore.2020.120332
    View in Google Scholar
54. Naeem, H. M., & Di Maria, E. (2022). Customer participation in new product development: An Industry 4.0 perspective. European Journal of Innovation Management, 25(6), 637–655. DOI: https://doi.org/10.1108/EJIM-01-2021-0036
    View in Google Scholar
55. Nevo D., & Kotlarsky J. (2020). Crowdsourcing as a strategic IS sourcing phenomenon: Critical review and insights for future research. Journal of Strategic Information Systems, 29(4), 101593. DOI: https://doi.org/10.1016/j.jsis.2020.101593
    View in Google Scholar
56. O’Hern, M., Akdeniz, B., & Du, S. (2021). The effects of crowdsourcing contribution type and temporal consistency on new product development success. R&D Management, 52(1), 126–138. DOI: https://doi.org/10.1111/radm.12481
    View in Google Scholar
57. Palacios-Marqu´es, D., Gallego-Nicholls, J. F., & Guijarro-Garc´ıa, M. (2021). A recipe for success: Crowdsourcing, online social networks, and their impact on organizational performance. Technological Forecasting and Social Change, 165, 120566. DOI: https://doi.org/10.1016/j.techfore.2020.120566
    View in Google Scholar
58. Pan, M., Chandrasekaran, A., Hill, J., & Rungtusanatham, M. (2022). Multidisciplinary R&D project success in small firms: The role of multiproject status and project management experience. Production and Operations Management, 31(7), 2806–2821. DOI: https://doi.org/10.1111/poms.13716
    View in Google Scholar
59. Protasiewicz, J. (2023). Supporting innovativeness and information sharing. In Knowledge recommendation systems with machine intelligence algorithms. Studies in Computational Intelligence (pp. 51–65). Springer. DOI: https://doi.org/10.1007/978-3-031-32696-7_4
    View in Google Scholar
60. Pussinen, P., Wallin, A., & Hemilä, J. (2023). The hope of exponential growth – Systems mapping perspective on birth of platform business. Digital Business, 3(2), 100060. DOI: https://doi.org/10.1016/j.digbus.2023.100060
    View in Google Scholar
61. Randhawa, K., Wilden, R., & West, J. (2019). Crowdsourcing without profit: The role of the seeker in open social innovation. R&D Management, 49(3), 298–317. DOI: https://doi.org/10.1111/radm.12357
    View in Google Scholar
62. Rauhut H., & Lorenz J. (2011). The wisdom of crowds in one mind: How individuals can simulate the knowledge of diverse societies to reach better decisions. Journal of Mathematical Psychology, 55, 191–197. DOI: https://doi.org/10.1016/j.jmp.2010.10.002
    View in Google Scholar
63. Ruiz, E., & Beretta, M. (2021). Managing internal and external crowdsourcing: An investigation of emerging challenges in the context of a less experienced firm. Technovation, 106, 102290. DOI: https://doi.org/10.1016/j.technovation.2021.102290
    View in Google Scholar
64. Sawhney, M., Verona, G., & Prandelli, E. (2005). Collaborating to create: The Internet as a platform for customer engagement in product innovation. Journal of Interactive Marketing, 19(4), 4–17. DOI: https://doi.org/10.1002/dir.20046
    View in Google Scholar
65. Schall, D. (2012). Service-oriented crowdsourcing: Architecture, protocols and algorithms. Springer. DOI: https://doi.org/10.1007/978-1-4614-5956-9
    View in Google Scholar
66. Schuhmacher, A., & Kuss, M. (2020). The impact of crowdsourcing in modern drug discovery. Expert Opinion on Drug Discovery, 15(8), 865–867. DOI: https://doi.org/10.1080/17460441.2020.1751116
    View in Google Scholar
67. Song, J., Dong, F., Zhao, J., Lu, S., Li, L., & Pan, Z. (2016). A new design optimization method for permanent magnet synchronous linear motors. Energies, 9, 992. DOI: https://doi.org/10.3390/en9120992
    View in Google Scholar
68. Spithoven, A., Clarysse, B., & Knockaert, M. (2011). Building absorptive capacity to organise inbound open innovation in traditional industries. Technovation, 30(2), 130–141. DOI: https://doi.org/10.1016/j.technovation.2009.08.004
    View in Google Scholar
69. Spithoven, A., Vanhaverbeke, W., & Roijakkers, N. (2013). Open innovation practices in SMEs and large enterprises. Small Business Economics, 41, 537–562. DOI: https://doi.org/10.1007/s11187-012-9453-9
    View in Google Scholar
70. Srisathan, W. A., Ketkaew, Ch., & Naruetharadhol, P. (2023). Assessing the effectiveness of open innovation implementation strategies in the promotion of ambidextrous innovation in Thai small and medium-sized enterprises. Journal of Innovation & Knowledge, 8(4), 100418. DOI: https://doi.org/10.1016/j.jik.2023.100418
    View in Google Scholar
71. Stefan, I., Hurmelinna-Laukkanen, P., Vanhaverbeke, W., & Oikarinen, E. L. (2022). The dark side of open innovation: Individual affective responses as hidden tolls of the paradox of openness. Journal of Business Research, 138, 360–373. DOI: https://doi.org/10.1016/j.jbusres.2021.09.028
    View in Google Scholar
72. Steininger, D. M., & Gatzemeier, S. (2019). Digitally forecasting new music product success via active crowdsourcing. Technological Forecasting and Social Change, 146, 167–180. DOI: https://doi.org/10.1016/j.techfore.2019.04.016
    View in Google Scholar
73. Stieger, D., Matzler, K., Chatterjee, S., & Ladstaetter–Fussnegger, F. (2012). Democratizing strategy. California Management Review, 54(4), 44–68. DOI: https://doi.org/10.1525/cmr.2012.54.4.44
    View in Google Scholar
74. Surowiecki, J. (2004). The wisdom of crowds: Why the many are smarter than the few and how collective wisdom shapes business, economies, societies and nations. Doubleday: Random.
    View in Google Scholar
75. Surowecki, J. (2005). The wisdom of crowds. NewYork: Anchor Books.
    View in Google Scholar
76. Tapscott, D., & Williams, A. (2007). Wikinomics: How mass collaboration changes everything. Old Saybrook, CT: Tantor Media Inc.
    View in Google Scholar
77. Teirlinck, P., & Spithoven, A. (2008). The spatial organisation of innovation: Open innovation, external knowledge relations and urban structure. Regional Studies, 42(5), 689–704. DOI: https://doi.org/10.1080/00343400701543694
    View in Google Scholar
78. Thompson, D. C., & Bentzien, J. (2020). Crowdsourcing and open innovation in the drug discovery: Recent contributions and future directions. Drug Discovery Today, 25(12), 2284–2293. DOI: https://doi.org/10.1016/j.drudis.2020.09.020
    View in Google Scholar
79. Toral, S. L., Martinez-Torres, M. R., Barrero, F., & Cortes, F. (2009). An empirical study of the driving forces behind online communities. Internet Research, 19(4), 378–392. DOI: https://doi.org/10.1108/10662240910981353
    View in Google Scholar
80. Toroslu, A., Schemmann, B., Chappin, M. M. H., Castaldi, C., & Herrmann, A. M. (2023). Value capture in open innovation: A literature review and a research agenda. Industrial Marketing Management, 114, 297–312. DOI: https://doi.org/10.1016/j.indmarman.2023.08.012
    View in Google Scholar
81. Tóth, G., Bao, J., Nguyen, N., Safouane Benazzouz, M., Monteiro Paiva Chaves, M., & Grill, A. (2021). The promise of crowdfunding to finance R&D of novel diagnostics and therapeutics for incurable diseases. Drug Discovery Today, 26(10), 2205–2208. DOI: https://doi.org/10.1016/j.drudis.2021.04.027
    View in Google Scholar
82. Usman, M., Vanhaverbeke, W., & Roijakkers, N. (2023). How open innovation can help entrepreneurs in sensing and seizing entrepreneurial opportunities in SMEs. International Journal of Entrepreneurial Behavior & Research, 29(9/10), 2065–2090. DOI: https://doi.org/10.1108/IJEBR-11-2022-1019
    View in Google Scholar
83. Vermicelli, S., Cricelli, L., & Grimaldi, M. (2020). How can crowdsourcing help tackle the COVID-19 pandemic? An explorative overview of innovative collaborative practices. R&D Management, 51(2), 183–194. DOI: https://doi.org/10.1111/radm.12443
    View in Google Scholar
84. Vignieri, V. (2020). Crowdsourcing as a mode of open innovation: Exploring drivers of success of a multisided platform through system dynamics modelling. Systems Research and Behavioral Science, 38, 108124. DOI: https://doi.org/10.1002/sres.2636
    View in Google Scholar
85. Wazoku.com (12.02.2023).
    View in Google Scholar
86. West, J., & Bogers, M. (2014). Leveraging external sources of innovation: A review of research on open innovation. Journal of Product Innovation Management, 31(4), 814–831. DOI: https://doi.org/10.1111/jpim.12125
    View in Google Scholar
87. Wu, H., Corney, J. R., & Grant, M. (2014). Crowdsourcing measures of design quality. In Proceedings of the ASME 2014 international design engineering technical conferences & computers and information in engineering conference IDETC/CIE 2014 (DETC2014-34967, V01BT02A010). American Society of Mechanical Engineers. DOI: https://doi.org/10.1115/DETC2014-34967
    View in Google Scholar
88. Zhang, Z., Jing, J., Wang, X., Kwang, K., Choo, R., & Gupta, B. B. (2020). A crowdsourcing method for online social networks security assessment based on human-centric computing. Human-centric Computing and Information Sciences, 10, 23. DOI: https://doi.org/10.1186/s13673-020-00230-0
    View in Google Scholar
89. Zheng, H., Li, D., & Hou, W. (2011). Task design, motivation and participation in crowdsourcing. International Journal of Electronic Commerce, 15(4), 57–88. DOI: https://doi.org/10.2753/JEC1086-4415150402
    View in Google Scholar
90. Zhitomirsky-Geffet, M., & Maman, Y. (2014). Wisdom of the crowds and online information reliability: A case study of Israeli real estate websites. Online Information Review, 38(3), 417–435. DOI: https://doi.org/10.1108/OIR-07-2013-0176
    View in Google Scholar