Artificial Intelligence, Machine Learning & Natural Language Processing

Differences between AI, ML, and NLP.

Artificial Intelligence as we know is a subfield of computer science which helps in performing and solving various tasks just like the humans. Machine Learning on the contrary is a subset of AI which allows a machine to learn from the past data without having to explicitly program it, whereas Natural Language Processing is also a subset of Artificial Intelligence, but the working of NLP is that it requires machine learning to be used effectively. Machine Learning is a better form of learning method for the training machines as compared to the old traditional methods, and NLP as mentioned above is a subset of AI but requires ML algorithms for processing the natural languages that can also be used for translation purposes from one language to another. [2]

Artificial Intelligence: AI is the term used for machines that helps to simulate human intelligence and encompasses systems that mimic the cognitive capabilities like learning from examples and solving problems. [3] Thus, in simple words, AI is the approach and application that covers anything which is related to making machines smart, be it a robot, or a car, a software application, or any machine. There are various applications of AI such as self-driving cars, predictive systems, voice assistants, autonomous vehicles, chatbots, and many more to name.

Considering an example of Artificial Intelligence, the various smart assistants or digital assistants developed are one of the best examples of AI. These assistants are helpful to humans in daily life as these digital assistants perform the tasks on our behalf. For example, Siri is one such example of AI that can access contacts, identify the words, and call the person, and interacts with the person using speech recognition, NLP, ML, and various statistical analysis to execute a particular task.

Machine Learning: ML refers to the systems that can learn from the past experiences which is a subset of the Artificial Intelligence. Its goal is to find patterns in the data and then make predictions based on those patterns in order to solve a business question and analyze trends to solve problems. [1] The main applications of ML are online recommender systems, Google search engine and algorithms, Facebook auto tagging suggestions, image and speech recognition, predictive analytics, Google translation, email and spam filtering, etc.

Considering the example of Social Media Features, ML algorithms are used to create some effective features, like Facebook notices and records the activities, chats, likes, comments, and also the amount of time you spend on certain number of posts. Machine learning tries to learn and train from these activities and experience in order to recommend friends and page suggestions on your profile. Another example of ML is the Netflix recommendation systems which provides recommendations based on the past experiences which uses the machine learning algorithms to train the model.

Natural Language Processing: The goal of Natural Language Processing is to comprehend the texts or the languages in the same way that humans do, and once the goal is achieved, the systems will then be able to comprehend, analyze, summarize, translate, generate text and language that is required. [1] The various applications of NLP are speech recognition, sentiment analysis, text summarization, translation, chatbots, automatic grammar checking, etc.

Grammar Checker is one of the popular example and application of NLP which detects and corrects the grammatical errors in the input text. Natural Language Processing is used along with the various techniques in order to perform the automatic grammar checker system. A similar example of NLP is Text Summarization which allows to summarize the text that uses the various NLP techniques such as tokenization, lemmatization, stop words removal, bag of words, etc. in order to perform the tasks.

Thus, to summarize the difference between AI, ML, and NLP, consider the below figure to better understand the differences between each one of them.

Figure 1. Difference between AI, ML, NLP [4]

As observed from the figure, ML is a subset of AI along with DL, i.e., Deep Learning, whereas NLP and ASR fall under AI and overlap with Machine Learning. Hence, each of them are intertwined with one another.

Concepts behind Natural Language Processing.

NLP as mentioned above is a subfield of AI which helps in making human input languages decipher to software. NLP is important because of its various applications that are available for day-to-day routine, for example, the text summarization application or the grammar checker, and even the translation systems. [1] The various techniques of NLP that help to extract data from the text and perform the necessary steps are sentiment analysis, named entity recognition, summarization, topic modeling, text classification, keyword extraction, lemmatization and stemming.

The common concepts and methods of NLP which help to develop a system and that are commonly used in NLP are tokenization, stemming, lemmatization, stop-word removal, parts-of-speech tagging, dependency parsing, named entity recognition, statistical language modeling, bag of words, n-grams, and regular expressions.

Tokenization: Tokenize is the process of converting a string of text into a list of words, i.e., breaking of the sentence or text into a list of words. This is the first step of NLP where tokenization is used in order to split the chunks of paragraphs and sentences into smaller chunks or list of words for better processing.

Stemming: Stemming is the process of converting the word into its word-root format or the suffixes. [5] PorterStemmer class from the NLTK package can be used for stemming which converts the words into their respective base form or cuts of the suffix. For example, the word cooking after applying stemming would be converted to cook. One of the drawbacks of stemming is that some words that might be important and useful are reduced to a non-meaningful base and hence lemmatization is the next step that can be used to overcome this drawback.

Lemmatization: Lemmatization is the process that converts the word to a meaningful base or root word or what is called a lemma. Lemmatization is somewhat similar to stemming, with a difference that lemmatization is able to capture canonical forms of words based on a particular word’s lemma. For example, if we apply stemming on the word better, the result will fail but lemmatization would result in a word, for example, better to good.

Stop-word Removal: Stop-words are the words which are less important and that occur frequently in the text. For example, the, and, an, a, is, that, which, etc. These words contribute to little or no meaning at all in the NLP process, and hence they are usually been removed. Stop-word removal is the step performed in order to remove unnecessary words from the text in order to perform the further steps of NLP.

Named Entity Recognition: Named Entity Recognition is not an important step in the NLP process or workflow, but it is recommended to perform this step in order to classify the words. NER is the mapping of the entity into their common names, i.e., each entity or object is classified into their respective categories which helps understand the various categories available in the text for analysis. For example, Google being categorized as an organization, Chandler being categorized as a Person, etc.

Parts-of-Speech tagging (POS tagging): POS tagging or word tagging is the process of associating a tag with each of the word of the sentence in the text. It assigns a category tag to the tokenized word which identifies the words as nouns, verbs, adjectives, etc. Default Tagging is the basic step that is performed for the POS tagging. And as the name goes, this is defined as the process of assigning or identifying one of the parts of speech to the tokenized word, i.e., labeling the word with an appropriate part of speech. The three types of POS tagging are rule-based tagging, stochastic tagging, and transformation tagging. For example, consider the sentence, “I like to play football, although I hated it in my childhood”. The POS tag considering for the word ‘hated’, would be a VERB. Using the other functions of POS tagging, we can also understand and analyze the word or text further into its tenses, like for instance, the word ‘hated’ would be analyzed to the output as Verb, Past tense.

References:

[1] O’Brien, S. (2022, July 5). 7 Key Differences Between NLP and Machine Learning (and Why You Should Learn Both). https://www.marketingaiinstitute.com/blog/7-key-differences-between-nlp-and-machine-learning-and-why-you-should-learn-both

[2] Mohadikar, M. (2021, January 24). Difference between AI, Machine Learning, NLP and Deep Learning. Medium. https://becominghuman.ai/difference-between-ai-machine-learning-nlp-and-deep-learning-9f63066087f1

[3] NLP, Machine Learning & AI, Explained. (2020, June 9). MonkeyLearn Blog. https://monkeylearn.com/blog/nlp-ai/

[4] Artificial intelligence: AI vs ML vs NLP. (n.d.). Sonix. https://sonix.ai/articles/difference-between-artificial-intelligence-machine-learning-and-natural-language-processing

[5] Nisar, R. (2021, December 16). Natural Language Processing — Basic Concepts - Analytics Vidhya. Medium. https://medium.com/analytics-vidhya/natural-language-processing-basic-concepts-a3c7f50bf5d3