Building Open Source Software in a Large Corporation

Santiago Basaldúa

d&a blog

The world runs on data, but without the dynamic, accessible and adaptable nature of OSS (Open Source Software) the pace of exploitation of data-rich fields would be painfully slow. Imagine a world of Data Science without Linux, Python, Anaconda or Tensorflow, just to cite some relevant examples of Open Source Software.

In the last years, the trend of using OSS for data processing and analytics has been increasing up to the point where most solutions today are OSS. The largest corporations, including Google, Microsoft or Facebook are actively contributing to it. Despite the success stories, adopting an open source philosophy requires challenging a lot of established ideas, especially if we want to attract active supporters instead of just users that are not providing knowledge to the project, or free riders.

We identified three fears that keep corporations from embracing OSS:

  • Fear of letting competitors benefit from their investments.
  • Corporate production roadmaps do not favor long-term development of necessary tools to create future products.
  • Not controlling the narrative. Legacy corporations are used to building a perfect narrative and dislike open competition despite their claims.

OSS 101 for executives

  • Contributing to OSS always pays back. In a recent study from the Strategy Unit at Harvard Business School Professor Frank Nagle states: “Paying employees to contribute to such [OSS] software boosts the company’s productivity from using the software by as much as 100 percent, when compared with free-riding competitors.” Active contributors produce more and better code.
  • On production roadmaps: Tools necessary to create applications typically result in large projects that, today, are mostly OSS. What corporations develop are typically business applications built on top of that. The schedules and incentives are different in each case. OSS communities are meritocratic while in corporate development legacy, hierarchy, budgets and schedules shape the products.
  • In OSS you can control the landing experience by creating introductions, tutorials, and documentation, but you cannot control the narrative. Users try it and decide if it suits their needs. All claims can be verified. To corporations this may sound like a nightmare, it is not. Actually, transparency is one of the main reasons why OSS has already replaced the old style business model. Technical users are saturated with overhyped unverifiable narratives and seek transparency.

The Community

We use to think about OSS as a piece of stable, useful, well documented code that is released continuously. That is the visible result in a success case. But most OSS projects fail before that stage. The reasons are diverse, including creating a product nobody wants, but a crucial one is failing to build a community. Bootstrapping a community is the hardest part and is a requirement for success. Founders must invest a lot of resources to produce something usable enough and reliable enough to attract early adopters in a stage when there is not much incentive to use the product. This is where large corporations have the power to support new endeavours. They can help by supporting branding, communication, infrastructure, part-time contributors and budget. Companies should envision OSS as an opportunity to invest in ambitious projects in their own benefit for an insignificant fraction of what they invest in business solutions. This improves employee motivation and productivity. It also gives them the power to choose what they support or cut their funding at no cost anytime.

At BBVA D&A

As an example, BBVA released Jazz in December 2017 as OSS. Jazz was originally a highly efficient distributed data store with an R client (now in cran) developed for internal use at BBVA Data & Analytics. After its release as OSS, its main author decided to redesign the product pivoting it into an AI platform that addresses issues that are poorly or not at all supported in mainstream platforms.

Jazz has been advancing through the first nine months of 2018 steadily, refactoring is now complete and a list of pending tasks is being addressed. Similarly ambitious projects had a two to four year “under the radar” period before they became relevant. By those standards, Jazz is still young. Time will tell if it becomes another abandoned project or a successful story of OSS groom by a big corporation, on that could one day tell its creators: This mission is too important for me to allow you to jeopardize it.1)Hal in 2001: A Space Odyssey, 1968

References   [ + ]

1. Hal in 2001: A Space Odyssey, 1968