A Data Base is just another source, a tribute to Molière’s play “le Bourgeois Gentilhomme”

<w

A lot has been said about the convergence between Business Intelligence and Enterprise Search. I've discussed this with many customers and partners wondering if yes or no, like Mister Jourdain who was speaking in prose without knowing it, Sinequa was already doing Business Intelligence. My view is that sometimes we talk about Business Intelligence when it would be more fair to talk about DataBase Offloading.

« Database Offloading » means using a search engine to query and use the content of a database. The database was designed to manage transactions, and not to optimize access to its content in order to rapidly populate third party applications. For example, a database where all the transactions of a bank are stored and managed contains information that could be accessible in order to give an immediate unified view of a given customer's history.

• The "IT 1.0" solution was to copy the database content in a datawarehouse, then allow querying of this datawarehouse. This solution was the result of technical limitations or hard facts inherited from the past (cost of hardware, databases performance,…). Today this looks too heavy and costly for the job done.
  
• The "IT 2.0" solution is to use an Enterprise Search engine to index the database content and therefore facilitate access to relevant information. Some parameterization may be necessary to complete this. The search engine must be precise, robust and scalable, completely designed using Web standards in terms of architecture and technology. Moreover, a next generation Enterprise Search engine will also be able to generate distributions on quantitative criteria related to a specific column. Sinequa allows this approach. A pioneer of this intelligent solution is Jean-Paul Figer, former CTO of Cap Gemini and today running his own IT architecture company; he promotes this in a REST styleJ (cf. sorry this is in French. REST, un style plus qu'un standard). Jean-Paul Figer has been able to take advantage of the disruption brought by search technologies and he can divide the cost of a project by 10 or more; more important he can reduce the time of implementation. A good Enterprise Search engine, like Sinequa, contributes in this case to huge improvements in productivity. However, this approach is not properly addressing Business Intelligence but more specifically « Database Offloading » and application rewriting in REST.
  

I'll conclude this post underlying that a second step is possible, when a Search Bus with good management of security rights is available: expand applicative possibilities to content beyond the database perimeter, indexing information from less structured sources, and therefore contributing to a 360° vision of the customer (or any other relevant subject). Here again, scalability, security management, connectivity, make all this possible.

The idea of this post came from the post of Adriaan Bloem, Analyst at CMS, who explains that using a search technology to access the content of a database at a lower cost is smart, but has more to do with « Database offloading » than with Business Intelligence.

Google Docs, Security and respect of access rights: is the possibility of a leak acceptable?

As published by TechCrunch, Google Docs has shared docs without permission. So it seems Google Docs has had a security leak. Some users were able to see what others had produced, despite the sharing and access rules in place. It is true that technology bugs are common, but is this kind of problem serious or not?

I think there are two issues: one is factual and related to what has been unduly shared and the damages occurred. The other is more intangible, it is the lack of confidence generated by the room for error. How can an individual work confidently if the fruit of their labor and their intellectual property rights are likely to be violated? How can we accept from the enterprise point of view, that confidential information is subject to leaks?

The principle of the Enterprise 2.0 is the sharing and exchange of information. This works because there is confidence in the tools available and particularly with one underlying condition: respect for the integrity of all user data. Several CIOs of Sinequa customers, particularly in banking, consulting and administration sectors, have rightly chosen our solution because it guarantees the respect of security rules. Conversely, I know a bank that installed a search solution (that I will not name) for their shared directories: the first day when the service went live, an employee searched for "executive bonus" and got the list of the bonuses of the executive team...

When it comes to security, we must demand zero risk. If for example the search solution is not designed to manage security at both the application and document levels, if the user access rights are not taken into account at the heart of the index, but "a posteriori", we are in danger. This is one reason that led Sinequa to develop its own application connectors. If the search solution does not permanently refresh user access rights in conjunction with new security rules (user profile changes, a public document that is now confidential...), there will always be a security risk leading to periods in which users can ask a question and get information that they should not see...

Personally, I think that non-compliance with user access rights and the risk of security leaks is unacceptable. And you, what is your opinion?

 

SINEQUA : SPEED AND VOLUME, keeping the relevancy and the rich functionalities

 

Sinequa just finished a first series of tests on our new version of Sinequa CS. I must confess I'm very proud.

Without any specific optimization, the results generate a lot of enthusiasm here. Sinequa has long been ahead in terms of relevancy and functionalities. When others did not see the point of managing security, linguistics or connectivity, we already solved these issues three years ago. We now have developed a new architecture including the necessary options to fulfill enterprise search needs at the kernel level of the technology, while at the same time generating first class performance. Sinequa technology is today at an unparalleled level of performance for this level of functionality. There will be detailed product data sheets coming soon, but in the mean time, here are a few points:

Number of queries on a large volume of users: up to 1700 simultaneous queries per second on one bi-processor server (average response time around 10 milliseconds). In production, our most demanding customer today manages up to 400 queries per second but with multiple servers, we actually generate here an improvement of around 50 times compared to the previous release of the technology. More importantly, it's highly sufficient to serve any customer needs.

Capacity to manage large volumes per server: one single server has indexed around 100 million documents (enterprise documents) in a few dozens of hours, and the server limits had not been reached. The server is a quadri-processor with 32 Giga of RAM (yes… it takes what it takes), so this is very promising; it represents a huge improvement for Sinequa, especially considering the performances come with a complete linearity based on the number of servers. We can now index the integrality of the enterprise content without consuming a lot of hardware resources, and this will be done in a reasonable time, and with sufficient refresh. For precise indexation time and volumes, I'll wait to have all the data per types of documents, since a PDF or a word document , an excel spreadsheet or a html document can be quite different. As an example, one entry level server(4 processors and 8 Giga of RAM): can index a little bit more than 1000 press documents per second, which means around 100 million documents in 24 hours (per server).

Capacity to index a database on an entry level server
(4 processors and 8 Giga of RAM): 5000 lines (or database objects) per second, which gave around 20 million lines per hour and finally 100 million database objects indexed in 5 hours. Maximal number of insertion per seconds: 10,000 which means in the end 100 million in less than three hours. I have recently read the performances of a competitor who was proudly indexing 30 million database objects in ten hours on a server. Sinequa does 6 to 7 times faster, and we are talking about a competitor who's main competitive advantage is supposed to be scalability.

We are impatient to see this new release of Sinequa being exposed to the users and content inside the enterprise; the rich functionalities of Sinequa combined with this level of performance, should give results that users will notice and vote for. We don't have long to wait as next month the first customer will be in production…

Desktop Search vs. Enterprise Search: a very different game

I was pleased by my conversation Friday with a knowledge management executive from a large international firm. He considers that desktop search has little to do with enterprise search, which was not how he saw things six months ago. Customers or analysts sometimes ask me why Sinequa doesn’t create a desktop search product, except when we adress very specific customer needs. There are two reasons: one is functional and linked to the usage, to our vision and our value proposition, and the other one is technical. The two work quite well together.

The functional reason is simple, Sinequa is an Enterprise 2.0 specialist. This means that through our enterprise search solution, we deliver individual productivity as well as collective intelligence.

• I am convinced that the Enterprise 2.0 serves this goal, making sure that anyone in the organisation is efficient and in phase with the rest of the company (what is new and disruptive here is the idea that productivity comes just as much from rich interactions as from organisation schemes and processes, cf. my december 2008 post « Taking advice from the ants »). In other words, collective intelligence comes from better interactions between employees. A prerequisite is that each employee must have access to shared information within the appropriate context. That means access to shared knowledge, according to his/her profile (i.e. a sales person must not have access to the knowledge of the CFO). This knowledge includes but is not limited to: documents, information within applications, employees who could provide valuable advice, or those who are interested in the same topic, or who would be relevant for the user to know of, customers that will be impacted, and so on...

• An exhaustive enterprise search solution such as Sinequa CS, equipped with all necessary connectors managing security and access rights, providing advanced extraction functionalities and appropriate scalability can offer all this. All that needs to be done is to deploy the indexation on all the applications (CRM, ERP, PLM, HRS,...), the Intranets, the file systems, the mail servers, …

• Some say that desktop content should be added to that shared content. I think this is highly inapropriate. As a matter of fact, information on the desktop happens to be… personal. Sure it must be easily searchable, but it should not be mixed with enterprise shared information and knowledge. The two applications (desktop search and enterprise search) should be different including the functionalities they offer. If not, you would get the worst of both worlds. One can actually legitimately compare desktop information with real world desktop and office documents: everyone of us organises his/her files according to his/her own needs. I file documents in a way that helps me stay efficient. What is on my desktop or in my drawers is there to help me do my job, and there is no capitalisation or sharing preocupation there. When I capitalise or share, it's from outside of my desktop. It does not mean that things should not be easily accessible and archived on my desktop (of course I need to be able to retrieve quickly from my drawer). But it could be dangerous to mix those contents with the rest of the enterprise content. That could lead to a massive slowdown of individual productivity. Indeed, it is important that when an employee searches something other than his/her own files, he/she searches only on the updated, validated, complete data sources, the ones that are on the shared environment. If the enterprise search always brings back personal desktop results, the employee will tend to go to those first (they are already known, I don't have to read them, just recognise them), and the risk of missing the right information increases.
• On the contrary, when I search within the shared content of the Enterprise, I search, then navigate, then need to check what I have found,... It's a different mental process from retrieving a file on my hard drive. In the end, mixing both search applications is thefore dangerous and confusing and will also slow down the shift to the Enterprise 2.0. Guess what: employees are more likely to continue to work alone.
  

I’ll be more concise on the second and technical reason.

• Desktop and desktop search is a discipline in itself, it happens to fit perfectly in the ergonomy of the desktop; not using too many resources to slow down the desktop. Moreover, I'm already familiar with the documents on my desktop since I am the only one downloading them on my hard drive. As a consequence, I can be satisfied with a very basic keyword search to find a document I already know exists. I do not need to search within context: the date, format, or location on my hard drive are enough to help me remember the context of a document. And desktop search must completely integrate within the operating system. Virtualisation does not change the argument.

• It is quite interesting to notice that vendors selling a desktop search and an enterprise search solution actually sell two different solutions with no real technical integration. There are no synergies, not even commercially, as most desktop search solutions are free. In that respect, desktop search has a lot to do with World Wide Web search, I'll do a specific post on that...
  

In conclusion, I recommend Microsoft Windows Desktop Search if you are using Windows (free), or Google Desktop (free). For your Enterprise search solution, it shouldn't be a surprise if I tell you I would pick Sinequa CS. But most of all, I strongly recommend testing the solution in the real environment, to keep in mind the complete deployment scope of the project, and be sure to talk to exisiting customers of enterprise search vendors. By the way, the best enteprise search solution integrates seamlessly with good desktop search products.

Autonomy buying Interowen illustrates a revolution in Enterprise search market

Autonomy gets blamed for many things, but Autonomy sure has a strong appetite and some ambition. I respect that. Congratulations for doing such a move at a time when so many others are sitting watching financial markets go down, laying off people... Autonomy is very brave, or maybe they just have to go down that road? I have always seen a great similarity between the dynamics of Autonomy and Oracle so I naturally found Alan Pelz-Sharpe's (Analyst at CMS) comments on the subject most interesting
http://www.cmswatch.com/Trends/1481-Autonomy-acquires-Interwoven---A-first-take

When Alan compares Autonomy with Oracle, I think he's hitting something big. Oracle started as an enabling technology and chose (or had to choose because of its size or culture) to change its value proposition and to become an application vendor. A database, like a search engine, is an enabling technology, it makes many things and applications possible.

But Oracle chose to become more of an application vendor and less a technology vendor. Were they scared of Microsoft's database or of the emergence of open source, or envious of SAP's profits and customer loyalty? In other words, did they lack technological excellence or faith in their capacity to stay the best technology, or did they see that another market happened to be bigger and was generating higher margins? I guess the answer is not relevant; what is relevant is that they made that move, and let companies like Business Objects take the BI market (ironically enough to later become part of SAP). On the other end, the value always ends on the end-user desktop or i-phone or blackberry, and the application was the way from the database to reach the end-user.

I think there is a parallel between the evolutions of Autonomy and Oracle. Autonomy is turning away from search and moving towards search-enabled applications. Nobody will dare to say this could be because their product or technology is not good enough, let's all agree it is just because they see better margins elsewhere. But I think that even though Autonomy is a very well run company and is making wise financial decisions, they will miss an important market, because in the case of search, the way to get to the end-user happens to be search itself : Search is a technology that also happens to be the ultimate application...

Most of Sinequa's competitors, like Exalead for example, say that enterprise search will become a commodity, a market for Google, Microsoft or Open source, and just like the leader Autonomy, they move towards verticalized applications, BI-like search, Governance Risk Compliance solutions,... a blue ocean where a lot of money lies for those that will solve very specific information access and management needs.
This is great news for companies like Sinequa, because this time the huge market is the enabling technology: Enterprise Search itself. It is so because the value of collaboration, of sharing information, of providing better access to knowledge, happens to be the real goldmine for companies. And it is 100% delivered by enterprise search. It is anything but simple to deliver, just as, a decade ago, it was not easy to deliver a good enough exhaustive web search engine. But we never learn...

The economic crisis will probably slow down infrastructure players in their capacity to ramp up on their enterprise search solutions, and my bet is that they will always have a lot of trouble connecting their search to the outside world anyway. The economic crisis will accelerate the verticalization of Autonomy and alike. Meanwhile, organizations absolutely need to do more with less, to develop productivity and collective intelligence and only enterprise search solutions will make this happen.

That looks like an opportunity for companies like Sinequa. This is not a dream, dreams are for higher purposes like the one President Obama realized for many of us this last November, but this is an ambition I share with most of my colleagues at Sinequa and especially with Alexandre Bilger who runs the company with me. And you will see, "Yes we can"!

2009 starts well for Sinequa

The financial crisis became an economic crisis affecting all industries and companies. In this context, I am satisfied by the good results of Sinequa and conforted that we took the right management and business decisions.

Sinequa has been recognized for the second year in a row as one of the top three French software vendors with the highest annual growth rate (we were first in 2007 and are third in 2008 of the AFDEL EuroSoftware 100 ranking). After a reorganization of the company (recrutment of a VP of alliances, reorganization of a slightly oversized management team, externalization of some of the research team whose work seemed to far away from our core business and customer needs) and thanks to our tight budget control, we should be EBITDA positive for the fifth consecutive year. And we continue to experience strong growth. Very encouraging, our indirect sales are starting to represent more and more volume and the public sector (unaffected by the crisis) represents a stable 25% of revenue. For example, one administration generated 500 K€ of revenue last year and was brought by a partner, just as a recent new signature (an English financial institution).

Controlled by its management and strongly supported by its financial VC partner X-Ange (backed by the French Post), Sinequa just received additional financial backing by OSEO and a large French Bank. All together, the available cash we can count on represents a year of revenue, this is more than sufficient for a profitable company. Beyond this financial security, we have been offered financial funding to support strategic development initiatives ... we remain very conservative on this possibility though.

On the product side, our new offering easily manages a billion corporate documents. Features tailored to the professional environment and its ease of deployment, provide the high end industrial solution for enterprises, beyond a simple search engine, a solution for collective intelligence (I will comment on it in another post). Some of our customers also deploy the research modules in video, sound and images. I am very pleased that our vision (connectivity, security, scalability, relevancy and the enterprise 2.0) meets the needs of customers. This is our main goal, which drives us and our evolution, and I am not afraid to say it is our obsession, our 'raison d'etre'. And for new customers just to speak only of France, the deployments at Saint Gobain, Sagem Communication, in SFR or Atos Origin, Le Figaro or at Courrier International or L'Equipe TV ... appear to be highly satisfactory.

I was also pleased to note in the December 2008 article in Wired magazine devoted to Ray Ozzie, the replacement of Bill Gates at Microsoft, that he put in place at Microsoft a way of working that we practice at Sinequa (i.e. small teams, a large open space, white boards everywhere...). 2009 will certainly be an eventful year for the economy, for the software industry and for search engine vendors. One of 10 forecasts for 2009 from the analyst firm IDC is a "re-invention of access to information and analysis will accelerate in 2009 driven by the fiasco of the financial industry, the increase of data ...". IDC believes that major players like EMC, Google, HP, IBM, ... will buy companies like ... Sinequa (IDC cites several others, including our fellow French Exalead who matched our revenue figures with strong growth last year while continuing to post losses of around € 5 million annually). I think not, I think this year is not conducive to mergers and acquisitions of quality. I believe that those who are well positioned and managed with modesty like Sinequa will continue their development, while unprofitable start-ups generating a lot of noise but little financial results, will face a tough period during the economic crisis and it will be difficult for them to negotiate their rescue by leading industrialists who are too busy with their own passage through the economic crisis.

Link to download the IDC article:
http://idc.com/research/predictions09_form.jsp

Link to article in French:
http://www.itrmanager.com/articles/85835/10-predictions-idc-2009-br-ralentissement-croissance-marches-emergents-cloud-computing.html

Why invest in videoconference infrastructure

Work together from anywhere!
Develop VIDEOCONFERENCING today!
A sound business decision
An investment and a mandatory evolution for companies

For intra-enterprise and sales or administrative interactions, today videoconferencing allows the organization of a meeting of two or more people with a level of quality comparable to a real-life meeting. There are two prerequisites for this: specialized equipment (cameras + software + screen) and a (very) high speed connection (between minimum 3Mbit and 20Mbit). All those who have experienced one can confirm, this is a virtual meeting that is comparable to a physical one.

This mode of interaction is clearly the future. More and more large companies are acquiring videoconferencing rooms and on average, videoconferencing can reduce travel expenses by 30% (source www.tandberg.fr). It's economical but also environmentally friendly as a large company can replace up to 20,000 annual trips by videoconference meetings, preventing the release of 2200 tonnes of CO2 in the air.

The investment required for an individual or a large meeting room has not stopped falling and will continue to decrease. Today we talk of spending a thousand or a few thousand Euros on quality equipment, then add the cost of a very high-speed connection that is now in the order of €100 per month (but this should decrease with the development of fiber optic coverage).

We are experiencing globally strong growth of people working from home. The reasoning is simple: why spend 2-4 hours per day in transportation to come to work in an open space where we will mainly sit behind a computer with a phone? Why pay rent to house employees who work primarily with people (internal or external) who are located elsewhere? In some countries, employees are increasingly moving from their workplace only for exceptional meetings, or to meet with customers. But their office is located at home. Therefore they can be home for important moments (caring for children, helping a sick spouse, ...) or simply enjoy a better quality of life with less stress of daily commuting. The company optimizes its costs and, above all becomes much more effective. This is sufficient if the employee has a home broadband connection, a computer and videoconferencing software.

Beyond the environmental issues and sustainable development (reduction of CO2 emissions, better use of space), beyond the economic issues (reducing transportation costs, lower office rents, minimization of non-productive time employees spend traveling to and from the office), this is also an essential change in the best practices of the company of the 21st century: in order for the company to have the flexibility necessary for its proper operation, it must offer its employees the ability to work together regardless of their location. In addition to the natural tools that provide the contact and context (social networks and enterprise search engine), the company must provide the proper videoconferencing infrastructure facilitating exchanges and rich collaboration.

Some would say that with e-mail and instant messaging, a mobile or VOIP phone, and webconferencing software has everything you need. I think they are wrong. What matters for virtual collaboration to be effective, is the quality of the exchange, including listening. The fact we play "together", being "in sync". To do this, human beings need an experience equivalent to a face-to-face meeting, brought by the videoconference call, unlike the conference call or the webconference.

My personal experience in watching my customers and partners, is that the companies that make this effort to transform their infrastructure benefit not only from the expected in terms of efficiency, but a gradual transformation and sustainable development of the state of the general work atmosphere. The company becomes more coherent, more consistent, more collaborative. Again, the parallel with the enterprise search engine is striking.