The Python language has become popular thanks to its ability to simplify ultra-complex activities.
There is a branch of artificial intelligence called deep learning that requires even more expertise than the above disciplines. And as we shall see, Python is back…
What exactly is “deep learning”?
Deep learning is a branch of machine learning that focuses on solving complex problems by mimicking the activity of the human mind. To achieve its goals, deep learning uses artificial neural networks.
Neural Networks
A neural network is a computational approach that can potentially learn to perform any task.
In 1943, two University of Illinois mathematicians invented the concept of virtual neural networks. From observing how neurons work in our body, they came up with the idea that we should be able to create functions that can produce an informed output signal from an input signal.
However, this concept first materialized in 1957 with Frank Rosenblatt’s Perceptron, a repetitive learning algorithm modeled on neural networks and capable of determining values and trends based on the desired outcome. The problem is that the perceptron generated linear models. However, many phenomena in our lives do not follow a linear model.
In 1986, Goeffrey Hinton invented the multi-layer perceptron, which takes into account large – and thus multi-layered – neural networks. He was able to generate non-linear models from real phenomena and set out four steps to solving a problem.
The calculation starts again at 1 and the cycle completes until a satisfactory result is obtained.
In 1990, Frenchman Yann Le Cun developed the concept by introducing “Convolutional Neural Networks” which will greatly facilitate the classification of images. Le Cun later led the Facebook/Meta Artificial Intelligence Lab and received a 2019 Turing Award for his work on deep learning.
In 2014, another important concept was developed by researchers Kyunghyun Cho, Bart van Merrienboer, Dzmitry Bahdanau and Yoshua Bengio: recurrent neural networks. In this model, the information calculated at a given point in time is stored in memory to be reused in a future calculation.
So let’s remember that deep learning works from networks of neurons that are connected to each other. “Deep” means deep and refers to the number of functions/neurons involved. The deeper the network, the better a machine can solve complex tasks.
The initial difficulty of managing deep learning
The problem with deep learning and neural networks is that they’re really not easy to implement.
Some of the “low-level” APIs (see explanations in the next section) that handle neural network processing include:
- TenserFlow by Google – this product was released on November 9, 2015 and is designed to run on multiple microprocessors and GPUs (graphics processing units) and mobile phone operating systems;
- Theano – a product developed by the MLA Group at the University of Montreal in Quebec and released in 2007. It is programmed in Python but generates C++ code;
- CNTK or Cognitive Toolkit – a Microsoft product released in January 2016 containing C++ code and Python.
However, if you are using a low-level system, writing the code is lengthy and tedious. Keras was developed with the aim of allowing quick experimentation with such tools. In other words, to go quickly from an idea to its implementation…
High and low level
Keras is a “high-level” interface. We speak of “low-level” language when it more or less directly manipulates a microprocessor. When you write a program using the instructions of a low-level language, you get very fast processing. A “high” language is much easier to use, but less efficient.
The origins of Keras
Keras was originally developed by Frenchman François Chollet, a software developer at Google. He told us about the creation of Keras.
“I founded Keras before I joined Google. I started it in March 2015.
At that time, deep learning was a whole different world. It was a small niche – there were probably less than 10,000 people in the world doing deep learning. Tensorflow and Pytorch weren’t there yet. Existing deep learning tools were aimed at experts and were generally difficult to use. All of these tools are dead today: Caffe, Lasagna, Pylearn2, Lua Torch, etc.
I started Keras because I was trying to apply recurrent neural networks to natural language processing as part of a personal project – I had started a question and answer service called quickanswers a year earlier. io: You enter a question and get an immediate answer. At the time there was no open source tool and in particular no implementation of LSTM, the neural network algorithm I wanted to use. Keras was the first implementation of this kind.
In other words, I created Keras because I needed Keras.
It’s also worth noting that until then, anything related to deep learning was particularly difficult to handle. So I did my best to make Keras easy and user-friendly. I was particularly inspired by scikit-learn, a really easy-to-learn Python library.
Six months later I joined Google. I’ve worked on various independent projects – including computer vision research for Google Photos. At the same time, I continued to improve Keras. The project quickly grew huge.
In November 2015, Google released its own deep learning system, TensorFlow.
By mid-2016, most major tech companies had their own deep learning system: MXNet, Chainer, Theano, TensorFlow, Keras, and thousands of smaller libraries. However, Keras already represented about 50% of the deep learning market – while I was updating it alone in my spare time, with a little help from the open source community.
Around the same time I got a lot of feedback from Keras users and I was surprised to say the least: people were using Keras to solve problems I didn’t know were problems – like automating a fish farm in Norway. It opened my eyes: deep learning is an incredibly powerful tool. We wouldn’t be using it to its full potential if we waited for researchers at Google, Microsoft, or Amazon to solve all the problems we could use it to solve. For what ? Because first of all, the tech industry is often not aware of the problems we could solve with it!
To unlock the full potential of AI, we needed to make these technologies fully accessible and in the hands of anyone with an idea and some programming skills. In this way, those who are familiar with certain problems can start solving them themselves. A bit like websites: these days anyone can create one without having to rely on the tech industry.
And that’s the future I wanted to build: a future where AI is a tool that’s fully accessible to everyone.
In October 2016, the manager of TensorFlow contacted me and offered me to join his team to integrate Keras into TensorFlow. I accept. Then, in early 2018, TensorFlow project leaders decided to adopt Keras as the high-level interface for their next major release. I had no influence on this decision.
Keras has since been adopted by 2.5 million developers and has become the basis for some of the world’s most advanced AI systems, including Waymo, a self-driving car, and Google’s recommendation engine. »
Keras, what is that exactly?
Let’s summarize:
- Keras is a high-level interface for managing deep learning;
- Keras is an open source Python library that makes it easy to implement neural networks;
- As a high-level application, Keras uses the power of low-level libraries such as TensorFlow, CNTK, MXNet or PlaidXML;
- TensorFlow has adopted Keras as its official interface.
Keras has shown a feat in making deep learning accessible to a large number of users. In particular, it provides the ability to easily “prototype” deep learning applications. It is estimated that the number of Keras users has doubled every year since its launch. The high-tech giants that have contributed to the development of Keras include not only Google, but also Microsoft, Amazon and Nvidia.
What are the typical applications of Keras?
Keras is commonly used to solve highly complex deep learning questions such as:
- driving a car autonomously;
- make recommendations based on their own tastes and those of a community;
- have an intelligent conversation like ChatGPT does;
- Recognize and classify images: dog, cat, etc. ;
- identify objects;
- identify people in a photograph;
- Play chess…
Who are the main users of Keras?
If one had to name the most spectacular example of the use of Keras, Waymo would probably come out on top. This autonomous vehicle – originally known as the Google Car – is one of the greatest achievements in artificial intelligence. There are more than 25,000 of these cars in the field, and they rely on deep learning algorithms programmed with Keras.
Other major users of Keras are Youtube, Netflix, Uber, Yelp, Square, Expedia (Microsoft), etc.
What are the pros and cons?
The advantages of Keras are many:
- the concepts handled by Keras are relatively easy to understand;
- it’s easy to “prototype” an application;
- it is compatible with most neural network models including Tensorflow, CNTK, Theano, MXNet and PlaidXML;
- an application can run on a conventional microprocessor, but also on a GPU (graphics card processor). In addition, Keras manages the processing in parallel;
- it is cross-platform;
- Keras is modular and therefore flexible;
- The applications that Keras can manage are varied, ranging from managing an autonomous vehicle to a smartphone app.
- The ease of development is such that Keras has been used in deep learning competitions.
Another advantage of Keras is that when it was launched it benefited from the contributions of more than 4,800 contributors and this community has continued to grow: it is estimated that 250,000 developers would participate in the continuous improvement of this library.
However, when an application’s response time is a significant factor, using Keras can be somewhat detrimental. By definition, a high-level language produces less efficient code than a low-level language. If this speed consideration were indispensable, one would have to force oneself to develop the desired application directly from a low-level language. When using Keras with Tensorflow, the response times are relatively good.
What if you want to become a verified data scientist thanks to Keras? Getting started with learning Keras is relatively easy, but if you want to achieve a high level of proficiency in the field, thorough training seems imperative.
Alternatively, if you are fluent in English, the Keras reference site has tutorials that even those with minimal programming experience can access. Once the basics are acquired, it is advisable to consult, test and refine the 10 models offered on the Keras application page and pre-trained in the Imagnet image database. To perfect your learning, you can also try the Keras samples available on Github.
What tools are similar to Keras?
If you want to approach deep learning through other tools, it might be good to look at these:
Set
Posit (https://posit.co/) is the new name of Rstudio, a popular data science platform. Like Python, R is relatively accessible, well-suited to statistical operations, and benefits from a large developer community.
Google Cloud AI
As befits Google (https://ai.google/tools/) provides access to its TensorFlow tool, but also to MLKit, well suited for developing applications for mobile phones, and various deep learning tools like AI Sight which is able to recognize objects in videos.
DataRobot
It is a cloud platform (https://www.datarobot.com/fr/) that, like Keras, aims to democratize access to data science and facilitate the use of artificial intelligence applications.
IBM Watson Studio
IBM offers in its Watson Studio offer many tools specialized in artificial intelligence tools (https://www.ibm.com/fr-fr/cloud/watson-studio/deep-learning) accessible via a cloud including specially adapted deep learning tools.