The Facebook Artificial Intelligence Research (FAIR) lab has released the code for a project called fastText. The social network says fastText helps it classify text more quickly than with traditional deep learning models thanks to techniques like bag of words and subword information. “In order to be efficient on datasets with a very large number of categories, fastText uses a hierarchical classifier, in which the different categories are organized in a tree, instead of a flat structure (think binary tree instead of list),” said Facebook’s Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov.
FastText is available for multiple languages, including English, German, Spanish, French and Czech. Facebook says it can be “trained on more than 1 billion words in less than 10 minutes using a standard multicore CPU. fastText can also classify a half-million sentences among more than 300,000 categories in less than five minutes.”
Developers interested in using the open source fastText libraries can download the code from GitHub.