The purpose of this article is to explain the evolution of the AI-Compare platform. As a reminder, AI-Compare is a unique API connected to the best AI engines and combined with a management platform for these services. It is also an opportunity for us to tell you how we got to the platform and the future developments that it will undergo.
The genesis of AI-Compare:
The idea of AI-Compare first came from the experience that the team had while working as consultants on Data Science and AI projects. We have been working for the last 3 years with various companies, from startups to large corporations. Our work has often included the integration of AI engines capable of processing data (often after transformation), and here are some examples:
- Image classification engine for a telecom operator
- Audio transcription engine (speech-to-text) to analyze phone calls for an insurance company
- Text classification engine for a pharmaceutical company as part of a bibliographic watch
Many AI engines (or microservices) exist on the market. We realized that it was complicated to choose the most suitable one for your data. We observed three things:
1/ The performances reached by these engines depend on the data: We have largely talked about it in our previous articles: comparison of custom image recognition engines, speech-to-text or Optical Character Recognition (OCR).
2/ A data processing project often includes several AI engines. The best performances can be obtained by combining engines from different providers. Example for the processing of tenders in Chinese:
3/ One provider won’t always be the most expensive or the cheapest: it’s possible to optimize costs by choosing the right provider for each one of your needs.
4/ And above all, we realized the multitude of providers that exist on the market. There are two categories :
- The “big players”: Google, Amazon, Microsoft, IBM and Ali Baba, who are positioned in all AI engines types
- The “specialists” who will position themselves on a reduced set of technologies but devote all their R&D to them. We have even noted some “ultra-specialists” who, in addition to specializing in one technology, only deal with one type of data: text mining and NLP on a given language or image recognition on a given type of image, etc.
All of this led us in late 2019 to think about how to help users better choose the AI engine best suited to their data.
First version and first feedbacks:
We launched a first version of the solution in April 2020. It included two components:
1/ A catalog where we listed a set of AI engines divided into various categories:
2/ A web application connected to various AI engines and a set of technologies to allow users to test these engines using their own data.
To promote this engine, we mainly used Medium which allowed us to quickly get several users:
This rapid growth, without any marketing budget, confirmed the relevance of our platform. We were also the first to be surprised by the international origin of our users (60% outside France):
Many of these first users agreed to talk to us and give us feedbacks on the platform and on their needs. It was also an opportunity to discuss with many AI engine providers from all over the world: France, USA, Japan, Hungary, Netherlands, etc. These discussions have been a great source of information to help us improve this beta version and make it into a real product with international scope.
Evolution of the platform:
The beta version was limited to comparing the performance of the engines of different providers. One of the main feedbacks concerned the use of these engines once the choice was made. Using several providers is not easy because of the differences between the different APIs/SDK and the need to subscribe to each provider separately. This led us to work on a major evolution of our solution. Not to limite it to a simple comparaison tool but to allow the direct use of AI engines via a unique API that we propose and that covers a number of technologies:
However, offering an API is not enough. We associate it with a platform that allows users to:
1/ Facilitate the use of AI engines:
The platform still integrates the graphical “comparator” tool (original idea of the platform) allowing the users to test different competing engines with their own data in order to make very personalized benchmarks.
We also include the ability to run trainable (customizable) models directly on the platform and analyze the performance they achieve and choose which model to deploy.
2/ Management API access:
This includes first of all Identity Access Management to manage access restrictions (especially with budget limitations). Our platform also allows you to set up limitations in terms of servers in order to, for example, only use algorithms that run on servers in a specific region (in Europe for example, to comply with the GDPR).
The platform also allows users to manage the costs from these different providers and at the same time to establish budget limitations that should not be exceeded.
Using our solution has several advantages during the two stages of an AI project:
- Simplified discovery of AI vendors (including niche vendors) and time saving in account creation.
- Comprehensive comparison tool (visualization of AI prediction performance, speed of AI engines, costs, etc.)
- Single API and doc: simplified implementation of AI engines from different vendors.
- Ability to change AI engine at any time: Zero change cost
- Implementation of a simple fallback solution in case of service unavailability and the possibility to combine the prediction of different AI engines into one more powerful one.
- Governance: cost tracking and optimization, access limitations and regulatory compliance (GDPR in Europe for example).
Here is a video demonstration of our platform that will give you an overview of what it allows:
Future of the platform:
Our goal in the future is still the same: to simplify the use and integration of these AI engines for our users. This is achieved through three major features that we are developing in parallel and that have been requested by our first beta testers:
1/ Open source: We are working to quickly integrate open source engines when they exist. This will allow the user to quickly measure the relevance (or not) of paid models
2/ Genius: We are working on automatically suggesting the best AI engine, so that the user does not even have to choose the engine. This complex choice will depend heavily on the technology in question and will be based on both the testing of different engines and on the (anonymous) metadata generated by our users
3/ Deployment of home-made engines: The user will have the possibility to deploy his own algorithms on the platform to compare them to the market engines and monitor them directly on the platform.
If you want to test our solution, just contact us on the link request a demo. We will be happy to present you our platform and discuss your needs in data analysis and processing.