Will Open Source Enabled Big Data, IoT / API Enabled Innovation prevail – YES or No?

In this section (4) we consider the question

– Will Open Source Enabled Big Data, IoT / API Enabled Innovation prevail – YES or No?

It is also clear that unless IT Functions embrace and lead in an API / IoT enabled economy we will continue to see the development of “Shadow IT” capabilities that are closely aligned to and embedded within the individual lines of business (sales, marketing, supply chain, manufacturing, distribution, multi-channel, partner enablement).

Indeed I believe we will continue to observe a switch from Business to Consumer (B2C) towards Business to Individual (B2I) insight based targeted enablement based on location, weather, preference, event, location insights (which follows IBM’s acquisition of the Weather Company) in addition to prior IBM alliances with Twitter, Apple and/or more recently Cisco in the IoT / Edge and Data Analytics area .

Indeed IBM already delivers solutions in this area with our Metro Pulse solution for Consumer Products Industry clients where multiple sources of un or semi structured Big Data (SQL Schema After) and/or Little Data (SQL Schema Before) are seamlessly combined with location, weather, preference, local event, historical POS data and promotional data etc to increase sales and product availability in “Metro” city based locations like London, New York or Singapore.
Also a high rate of innovation (and change) is currently being observed in the Big Data platforms and analytics solutions area where it seems that the majority of the Enterprise IT Architects and Clients I’ve spoken to are firmly committed to Open Source aligned Big Data solutions and platform choices. This then naturally raises the following question

Will Open Source aligned Big Data solutions eventually prevail?

In my view the answer is a 100% YES, although I also believe that a balance between open source driven innovation and large enterprise scale IT non functional requirements is required as summarized below:

Open Innovation

Indeed in the area of Proprietary vs Open Systems (at one time these used to be defined as Unix based Client / Server systems vs the IBM Mainframe), IBM previously tried a relatively closed and proprietaty approach when the IT market was rapidly transitioning towards Unix or “Open” distributed client / server platforms in the early 1990’s.
IBM subsequently / consequently suffered a near death experience in business terms as prior continued M/Frame MIPS platform capacity growth rapidly switched towards these alternative Distributed / Client Server platforms. Indeed SAP delivered SAP R/3 vs R/2 over the Mainframe with DB2 before to align with this “Open Systems” choice and market trend.

Although it is also true to say in more recent times IBM M/Frame MIPS capacity growth (combined with Open platform M/Frame Linux enablement) continues at a pace, often for mission critical systems of record / big batch scenarios.

Something rather similar happened in the PC market where IBM developed a technically superior but incompatible IBM PS/2 MCA (Micro Channel Architecture) as a follow on to the original IBM PC and/or PC AT IO adapter architecture.

Just as we technically turned right the rest of the market turned left with an ISA (Industry Standard Architecture) PC Input / Output (IO) adapter and bus strategy the rest is history as IBM’s PC Company went from having a largely dominant “IBM PC” market share to a significantly smaller share over time, is the same thing happening now in the core ERP / Systems of Record market?

As a direct result and subsequently IBM’s commitment and contribution to Open Source driven projects and innovation has been second to none amongst the major IT vendors, in summary IBM essentially previously learnt a very hard business into IT lesson in IT innovation and industry change terms.

This commitment includes significant investments and technical alignment to the following:

  • The Apache Software Foundation (1999) subsequently Eclipse (2001)
  • Linux (2007), OpenStack (2012), Cloud Foundry (2014)
  • jS (2014), Docker and the very significant Apache SPARK in-memory analytics operation system (2015) investment
  • In addition to the more recent innovative Blockchain based Hyperledger project (2016).

These commitments are in addition to the ODPi (Open Data Platform) Hadoop initiative are now both pervasive and very significant within IBM, indeed IBM recently published a paper that summarises this commitment and the resulting rates of Open Source driven innovation that in the longer term, in the view of the author will always eventually prevail over proprietary aligned alternatives no matter how big a single vendor or aligned partner eco system commitment.

Hence in my view, It’s not really a case of if, simply a case of when Open Source based innovation prevails.

Indeed in support of this viewpoint, Vinnie Mirchandani (in SAP Nation v1.0) mentions the success and growth of the Cloud Integrator Appirio with BoB / SaaS integration solutions and a large TopCoder community, in addition the rapid growth IBM is experiencing in the Bluemix and/or API Connect area’s.

Of course this Open Source commitment does not mean clients will not require a prior trusted solution partners to help them to safely bridge between their existing system of record and planned front office API enabled strategic IT platform investments, either Public, Hybrid Cloud or indeed prior Private Cloud / On-Premise often for mission and business critical data protection and/or privacy / IP reasons – It all starts with the data !.

The above mentioned paper can be found here, it nicely summarises the evolution of various Open Source platforms over time.


More recently one of the potentially most significant and innovative Open Source projects is the rapidly emerging Blockchain “Hyperledger” distributed ledger project that will in my view be truly transformative for many clients and industries.

Indeed I’d also recommend a rather detailed report published by the UK Government Office for Science, Chief Scientific Advisor, Mark Walport in December 2015 called :

Distributed Ledger Technology: beyond block chain, which can be found at:


Commencing initially with the Financial Services industry but then likely rapidly extending into other industries like Consumer Products and/or Discreet Manufacturing where complex extended and distributed supply chains and resulting financial transaction flows and ledger entries are the norm.

I’d also recommend the following short youtube video that describes the future impact of the this project in addition to this item on the Financial Times ft.com as follows:



Having covered some of the strategic IT investment choices in the above lets now dive back into the details of some of the “hype” and largely commercially driven pressure to migrate to SAP S/4 HANA and/or HANA OS/DB migrations (vs prior Oracle, DB2, MS SQL etc SAP AnyDB platform choices)

Now we move onto the final section in this series of blog entrances – In-memory marketing hype vs reality, section 5.

Disclaimer – This blog represents the authors own views vs a formal IBM point of view

The views expressed in this blog are the authors and do not represent a formal IBM point of view.

They do represent an aggregate of many years (20+) of successful ERP / SAP Platform deployment and IT strategy development experience that is supplemented with many hours of reading, respective DB2 and/or SAP HANA roadmap materials and presentations at various user conferences and/or user groups, in addition to carefully reading input from a range of respected industry / database analyst sources (these sources are respected and quoted).




One thought on “Will Open Source Enabled Big Data, IoT / API Enabled Innovation prevail – YES or No?”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s