Enteprise Search Bus : the intelligent fondation of the entreprise 2.0

The Enterprise Search Bus is becoming key in the information and IT infrastructure of organisations. It announces a decline in the strategic importance of the relational database, emphasizing its greatest weakness: its rigidity that seen from an intelligence perspective makes it a rather dumb tool. Here's why and here are some consequences to the software ecosystem.

The context has two axes: the content and the container

Axe 1: there are three main types of data:

1. Structured (numbers, values, positions…),
2. Unstructured (contracts, catalogues, documents…),
3. Issues of exchanges and collaboration (messages, discussions, directories,...).

Axe 2: containers of all types of content share three main missions:

1. The first is historically their main purpose: to guarantee transactions, because this is ultimately where the data is born and/or makes sense in the real economic activity of the company.

2. The second is a corollary to the first but has grown increasingly important with the growth of volumes to be searchable, for one must be able to put the right data in front of the right operator or to ensure a good transaction.

3. And the third arises from the info-explosion and issues of cost and energy control: managing the archiving, and organized retention of data.

In a predictable Darwinian phenomenon, each of these major functions has become a playing field for specialized players. Depending on whether the data is structured or unstructured, the actors may be different. But there is a groundswell of interest: the database was the backbone of the company and this is now changing. The success of the database was based on its ability to organize and secure transactions. These are now, at the time of redundancy and the Cloud, fairly basic if not commonplace qualities.

To sum up the software landscape, the table below explains how software has emerged at the 9 key intersections of the two axes mentioned above.

Function/ type of data	Structured	Unstructured	People
Transaction and process management	ERP, CRM, business applications + Database.	BPM + CMS + Database	Mail tool, enterprise social network
Search and navigation, analysis	Business Intelligence + Data warehouse	Integrated Search Engine	Integrated Search Engine
Archiving, backup,…	Dedicated archiving tools	Dedicated archiving tools	Dedicated archiving tools

The info-explosion is relativizing the hegemony of the relational database

Until the info-explosion, it was vertical integration: Oracle is a good example starting from the relational database and going to the business application managing structured data. Those prospered inside a silo. For this to work, application servers with the ambition to make apllications talk to applications were the next logical step in this paradigm. This was the core fondation of the enterprise 1.0 information system. Such a system was busy properly handling its transactions, and counting and analyzing its business data: its core component was naturally the relational database. Outside the File System and the email box, long regarded as tools reserved for minor office uses, almost all enterprise computing has been built around the relational database.

But the relational database system is cumbersome and expensive on one hand, when in addition Business Intelligence tools lack agility. On the other hand, the info-explosion of unstructured data (emails and files in directories that are stored "flat" without a relational model) has stimulated the advance of the technology of search and navigation. As unstructured data has become increasingly business critical, the tools to access data have become increasingly professional. Finally, if one can say ‘who can do more can do less’. The relational database becomes a source of information as any other, it is moving from the status repository of system information to a simple container of structured data. It is as such, neither more nor less important and strategic than the CMS.

Two important consequences on the value of the search engine in the structured world

1. The Business Intelligence tools are being marginalized by the search engines to access information and provide a 360° view. In 90% of the time, the search engine is more flexible and better than the BI tool. See the recent testimony of the Laser Group in 01 Informatique in French (the recent choice of Sinequa in a database offload bench against a known French rival. A very interesting project which aims to index the entire transaction history of the 22 million credit cards of Laser / Cofinoga).
2. With server virtualization, not to mention the gradual migration to the Cloud, the challenge is not to allow applications to talk to each other, but to enforce a consistent repository of data. From this perspective, this is the announcement of the victory of the Enterprise Search Bus. An intelligent Enterprise Search Bus is capable of ensuring the consistency of all data by making it searchable by business relevant categories/metadata, either inherited from the source or generated on the fly. In this respect, one can imagine that a vision based on the Enterprise Search Bus will absorb MDM (Master Data Management) type approaches that are in fact a very accomplished, but very specialized response to the question.

The Enterprise Search Bus becomes strategic for the enterprise

The important point for the construction or development of an information system is - but it is not surprising to see me push this argument - that the search engine has gone from a fun gadget (it’s true, who really needs a search engine to find data a little faster on the Intranet or to find lost documents on shared directories?) to an essential element of the enterprise information system. The Enterprise Search Bus is:

• Key to unify the data by structuring it within the business context,
• Essential for help not in searching but to achieve effective results with the right information in the right place within the right application.
• Major in the logic of supporting a simple work ethic. Simple as it digests the real complexity of our business ecosystem by reducing it to a logical layout, that is easy and intuitive. Talk to consultants from Atos Origin, who have recently selected Sinequa to access their unstructured data, or consultants from Mercer (see Press Release attached).

Consequences on the market of the enterprise search engine

To be up to the challenges, your Enterprise Search Bus will have to validate several key criteria:

1. Integrate the generation of smart technology and metadata processing. If the relational database is losing ground, the search engine needs to provide the necessary business structure more than ever. In this area, you have the choice of linguistics (Sinequa, FAST) or Bayesian (Autonomy).
2. Absorb all the volumes and heterogeneity of your information system. You needlinear scalability and secure connectors that are rapidly deployable without any specific development.
3. Integrate in a logical and portable platform. You are Java or .Net, private servers or Cloud: your Enterprise Search Bus must be agnostic on the subject.
4. And many other criteria such as ergonomics as simple as an iphone. A true 2.0 product in the way it is used (interactivity) and in its way of being deployed (pilot site), ...

Sinequa validates these criteria better than anyone I think. Our vision for many years, the work we have with our customers in particular since 2005, and our flexibility allow us today to have the right product at the right time.

I must confess: 5 years ago, we thought the topic was to translate the value proposition of Google in the enterprise. Our technical department was obsessed with Google. Today, this seems ridiculous, because our mission is at the heart of a business increasingly virtualized, both literally and figuratively (Cloud computing, outsourced supply chain management, outsourcing of most enterprise processes, offshore development, outsourced marketing, etc.). The heart of the intelligence and agility of a company needs an Enterprise Search Bus. Some speak of meaning-based computing; I dare to say simply Intelligent business. The following definition from Wikipedia seems eloquent to me.

“Intelligence comes from the Latin Intelligentsia (ability to understand), derived from Latin intellegere meaning understand, and whose prefix inter (between), and the radical legere (to choose, pick) or ligare (link) suggests essentially the ability to link elements who would otherwise be separated.”

“Intelligence is all mental faculties to understand things and events, to discover relationships between them. Intelligence is also recognized as being what in fact it allows: adaptability. Also, practical intelligence is the ability to act appropriately to situations. In terms of evolution of human understanding cannot be conceived without a diversified coding system. It therefore comes to conceptual intelligence, inseparable from a mastery of language (and therefore "words") to complex reasoning and the reasoning is the mental process of analysis for determining the relationships between elements. Finally, and at this level, the purpose of intelligence is the conceptual and rational knowledge.”

The market for Enterprise Search is exciting. We realized that Google was a mediated and talented follower, little present in value-added projects. Today, all eyes are turned to FAST / Microsoft. But I think that innovation and value this time will still come from others. I think the winning visions cannot be based on an Enterprise Search Bus choked by a complete proprietary chain from the OS to the browser through the portal, CMS and database.

I think that the winning vision will be an intelligent and flexible Enterprise Search Bus, able to keep up with new innovative technologies, delivering a capacity to create the link between the user and the data. It is a Darwinian market, it is not enough to be right, we must constantly adapt. So see you in 5 years, when the Cloud is ubiquitous.