This blog shows the basic concepts of information theory needed for ML. Information theory studies the measure, communication and storage of an information.
Information Theory for ML
Expectations and (co)variances
This article talks about the expectation, variance and covariance of random variables or functions.
Important probability distributions
This article summarises some discrete probability distributions (Bernouilli, Binomial and the Poisson distributions) and some continuous ones, such as the normal, student-t and exponential distributions, and graphs them in Python.
Essential continuous probability theory
This blog post vulgarises some notions from the previous post applied to continuous cases. Probability theory is a large area of Mathematics and this post gives by no means a complete overview, only some essential background knowledge to understand ML models! Let’s get into it then!
Essential discrete probability theory
Probability theory is essential area of Mathematics in order to understand Machine Learning techniques. This post is a short introduction to it, a vulgarisation of the most important notions. This level is logical and easy to grasp through examples, so don’t be afraid of reading it through.
XLNet
XLNet: Generalized Autoregressive Pretraining for Language Understanding by Yang et al. was published in June 2019. The article claims that it overcomes shortcomings of BERT and achieves SOTA results in many NLP tasks.
In this article I explain XLNet and show the code of a binary classification example on the IMDB dataset. I compare the two model as I did the same classification with BERT (see here). For the complete code, see my github (here).
You Do Cker
There are some innovations in IT that everyone follows and hear about and then there are some others, quieter, humbler, those sort of innovations that hide behind a wall and when you understand their power and what you can do with them, you are truly astonished.
Let’s have a committed relationship … with git
Git is the most widely used version control system in the world. I have to use it during my work, and as it is not difficult to use (if you don’t have conflicts), I didn’t think much about it until recently. However, once I started to read more about it, I quickly realised its elegant way of dealing with versions and integrity control worth more than an article! We have probably all taken this amazing tool granted time to time, underestimating all the trouble it saves us from. So let’s realise the power and elegance of git, and let’s finally get into a committed relationship with it!
Continue reading “Let’s have a committed relationship … with git”
BERT: Bidirectional Transformers for Language Understanding
One of the major advances in deep learning in 2018 has been the development of effective NLP transfer learning methods, such as ULMFiT, ELMo and BERT. The Transformer Bidirectional Encoder Representations aka BERT has shown strong empirical performance therefore BERT will certainly continue to be a core method in NLP for years to come.
Continue reading “BERT: Bidirectional Transformers for Language Understanding”
Elastic vs. PostgreSQL for text lookups
During one of my missions, I could compare two database management systems and their performance on text lookups: Elasticsearch and PostgreSQL. Here is the what I found.
MLIT 