Crowdsourcing In Software Engineering

Siddharth
6 min readNov 28, 2020

Back in 2017, a 10 year old girl from Michigan, USA had made the news when Elon Musk himself had replied to a letter she had sent to him with a tweet saying “That sounds like a great idea. We’ll do it”. The idea that Elon liked and which the girl had suggested was hosting a competition for the general public on who could come up with the best advertisement for the previously unadvertised Tesla Cars. The winner would get an year’s worth of supercharging or a Model 3 Easter egg, both products of Tesla. While the idea may not have been a brilliant business plan or something innovative, it did surmise the essence of crowdsourcing for benefit.

Crowdsourcing is not a new concept by any stretch. In fact, I would like to believe it is originating from primitive times, a shoutout to the meme “Apes together strong”. Over the years, there have been countless stories of crowdsourcing used for the innovating ideas and forming solutions to complex puzzles.

The term itself was coined in 2006 by Jeff Howe who had carefully observed this centuries old phenomenon and introduced it to the world in an article detailing this phenomenal practice. And the timing of this was just perfect for crowdsourcing to make it’s entry in the world of tech for 2006 was an year of big technological trends with wireless internet access, video on phones, video on demand(everywhere), the start of vlogs(video blogs) and a boom in internet phone service. In a nutshell, the perfect ingredients for tech companies to use crowdsourcing to influence the people with ideas they collect from the people(an ironical reflection of Lincoln’s immortal words describing democracy).

Models

Crowdsourcing as a concept is regularly used in software engineering, but for efficient usage, it has to be deployed in the form of various models. The purpose of these models is to keep a track of crowdsourcing so that once the model is implemented, the results of crowdsourcing could be analyzed. To understand further how software companies see crowdsourcing, here is Jeff Howe’s definition :

Crowdsourcing represents the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call.

There are 3 points which stand out in the definition : The work is solicited through an open call to which basically anyone can respond. The workers who volunteer are unknown to the organization needing the work done. The group of workers can be large. Ke Mao echoing these points gave the definition of crowdsourced software engineering as :

Crowdsourced Software Engineering is the act of undertaking any external software engineering tasks by an undefined, potentially large group of online workers in an open call format.

Peer Production
This is one of the earliest and most popular crowdsourcing model in software engineering. Peer production has been responsible for some of the biggest breakthroughs and development in software history. Linux, Apache, Rails, Firefox, all these great technologies would have not been possible without crowdsourcing.
The most well known form of peer production in software is open source development. A model in which control is decentralized and contributions are made without monetary reward. The contributors decide the project’s scope and goals rather than a paying client. Contributors are typically motivated by the opportunity to gain experience with new technologies, bolster their reputation, and contribute to a good cause. Expert questions on Stack Overflow receive an answer in a median of just 11 minutes,9 another example of crowdsourcing’s power.

Competitions
A second crowdsourcing model, competitions, has recently gained significant attention in software development. Pioneered for software by Top Coder, competitions are similar to traditional outsourcing, in which a client requests work and pays for its completion. However, they treat workers as contestants rather than collaborators. First, a client proposes a project. Then a copilot (an experienced worker paid for this work) decomposes the project into a series of competitions that might cover requirements, architecture, UI design, implementation, and testing.

Dimensions of Crowdsourcing
Peer production, competitions, and micro tasking have important differences. To compare crowdsourcing models and provide a sense of the space in which they exist, we identified eight foundational and orthogonal dimensions along which the models vary.

The dimensions of crowdsourcing

Motivations

It’s interesting to consider the forces driving the current emergence of crowdsourcing models, platforms, and environments, particularly in terms of their use in software development organizations. Many of the models are relatively novel, and their long-term benefits and drawbacks remain poorly understood. Businesses, however, must see tangible benefits to adopt and use crowdsourcing — even if experimentally. Here, we review several motives driving software development organizations to adopt crowdsourcing, as well as the forces motivating developers to participate.

Reduced Time to Market
Increased development speed is a frequent reason for crowdsourcing. The possibility to perform usability testing of a system in a few hours is tantalizing, given that such efforts typically take much longer in-house.

Generating Alternative Solutions
Organizing work into self-contained tasks enables multiple workers to independently complete the same task. Because workers have diverse perspectives, backgrounds, and experiences, this often leads to the creation of alternative solutions. In Stack Overflow, developers can compare many alternative answers to their question, selecting the best for their needs.

Employing Specialists
Decomposing large development tasks into smaller tasks enables greater flexibility in the use of specialist freelancers. Crowdsourcing might make it possible to rely less on in-house developers who are generalists (and thus perhaps less fluent in a particular area of work) or on recruiting specialists into the team when necessary (which often requires substantial lead time).

Learning through Work
As we mentioned before, a key reason developers contribute to open source is to learn a new technology. They might want to learn a new framework (for example, AngularJS) or get up to speed on the style and idioms of a new project by reading some code and contributing a bug fix. Whereas millions use communities such as Codecademy (www .codecademy.com) to learn the basics of new technologies, going beyond requires jumping into a larger, real project. Different crowdsourcing models let developers do so with varying commitment levels.

Challenges

Even as crowdsourcing slowly but surely enters mainstream software development, key challenges remain for it to reach its true potential and for the scenarios sketched in the introduction to become reality — if that’s indeed possible. In particular, further realizing many of crowdsourcing’s bene ts requires further decomposition of tasks and even greater participation in crowdsourcing platforms. Essentially, this asks whether we can increase the crowd’s size, shrink the tasks, and lower the expertise and coordination demands. it’s likely no coincidence that many of the tasks for which crowdsourcing has found the most success — testing, creating UI mockups, and answering questions — have clear goals and require minimal context. For example, can the authoring of a software architecture be decomposed into short, self-contained tasks that don’t require contributors to understand the complexity of the whole?

Conclusion

Crowdsourcing in its various forms has already changed software development. Open source aside, the number of new crowdsourcing platforms, the number of workers signing up and actively contributing, and the number of organizations actively experimenting with crowdsourcing all indicate a phenomenon that in many ways has crept up on the industry rather than taking it by storm. The potential advantages are tangible. The increasing shift of development work to labor markets — for example, more than $1 billion annually in freelancing is brokered through Upwork alone — portends the potential for even more dramatic shifts.

References

The article is a summarization of the splendid paper “Crowdsourcing in Software Engineering Models, Motivations, and Challenges” written by Thomas D. LaToza, George Mason University and André van der Hoek, University of California, Irvine.

--

--