Historically, most organizations in our society (governments, armies, companies) structure themselves in a hierarchy, with knowledge and power concentrated at the top and segmented visibility of smaller decomposed tasks distributed along its base.

Crowdsourcing technology is enabling less structured and specialized organizations to solve complex knowledge problems competitively. It is not a cheap way to outsource work to volunteers. When done right, it is an entire new form of knowledge production with the potential to revolutionize our approach to solving simple and complex problems.

Knowledge has a long tail

The Open Source Software Community and the volunteers of Wikipedia show that in at least some knowledge domains, unstructured crowds of regular people produce results that are competitive with hierarchical organization of professional experts (professional software teams, the Phd’s of Encyclopedia Britannica).



To understand how that happens, let’s look at how knowledge distributes among a population (picture above). “Long Tail” refers to a statistical property of a distribution where its “tail” is larger than its “head”.

The group at the head of the curve (“experts”) accumulates personal knowledge that is individually higher than the average person. But the total knowledge held by the experts is still relatively small compared to the knowledge held by the broader population.

Our current organizations are very efficient expressing the knowledge of experts at the top of the pyramid or at the head of segmented functional groups, but it fails to capture the knowledge at the base. There are good reasons why they evolved that way…

The Medium is the Message

The history of human civilization is tightly connected with our ability to use language to organize groups to tackle complex problems that are beyond the abilities of a single person.

The most effective way to exchange ideas with other people is through rich direct interaction face-to-face. But because it is difficult to have that kind of interaction with a large number of people, most large organizations have relied primarily on the printed word (books, documents, memos, email messages) to collaborate.

The print medium, being unidirectional, is particularly good at transferring knowledge from someone who knows more (“expert”) to a number of people who know less. The predominance of print led to the evolution of highly specialized and hierarchical organizations, where experts accumulate knowledge in narrow functional fields and lead a large group of less knowledgeable people.

In this environment, collective intelligence does not have channels of expression. The voice of the expert is the voice of knowledge.

Crowdsourcing Technology

Digital technologies, the Internet and Social Media are starting to provide a communication medium that emulates some of the characteristics of rich, direct interaction. It is real-time (not linear), it links peer-to-peer (not hierarchically), and it is interactive.

So, if that is true, why don’t we see the effects of co-creation emerge in the most popular social (Twitter, Facebook, Google+) and Q&A websites (like Quora or Yahoo Answers)?

The reason is that we are still using the new medium as we used the old one. Quora is still dominated by the individual desire of owning the truth and position as an expert. True co-creation requires collaboration, not ego, as the driving force from the process of expressing collective knowledge.

Also, we are already using medium interactivity, but not yet created the mechanisms behind the interaction to allow for true co-creation and collective intelligence to emerge and be expressed. That back-end infrastructure is still emerging in the form of new Crowdsourcing Technology.

Conclusion

In domains being affected earlier by the digital medium, (e.g. software development and publishing), the long tail of knowledge held by the unstructured crowd is now able to express itself in ways that are competitive with hierarchical organizations.

As Crowdsourcing technologies evolve and spread over other domains, more and more complex problems can be tackled using a new form of human organization.

We will be able to solve problems, not by analytically decomposing big problems into smaller ones, but by presenting the complex problem to the collective intelligence and let it holistically express the solution.

–

Marcio Saito’s (@Marcio_Saito) interest in Collaboration and Co-Creation originates in his early involvement with the Open Source Software community in the early 90′s. He writes about Social Media and Collective Intelligence and is an advisor to Ledface, a startup using Crowdsourcing to create a new kind of Intelligence.