Lexicon and Deep Learning-Based Approaches in Sentiment Analysis on Short Texts ()
1. Introduction
Opinion mining, also known as sentiment analysis, has gained significant attention in recent years as a result of the exponential growth of user-generated content on social media platforms such as Twitter. Estimating and categorizing people’s expressed thoughts, sentiments, and attitudes in the text is known as sentiment analysis [1] . This information can be useful for various objectives, including market research, brand management, public opinion analysis, and decision-making. Sentiment analysis can be performed using either a lexicon-based or a deep learning-based method. The lexicon-based technique is based on established sentiment dictionaries or lexicons that contain words and their associated sentiment polarity [2] . It assigns sentiment labels to text depending on the presence and context of certain terms. On the other hand, deep learning-based techniques use artificial neural networks, notably recurrent neural networks (RNNs) and variants such as Long Short-Term Memory (LSTM), to learn sentiment patterns and characteristics automatically from enormous volumes of training data [3] .
This research focuses on analyzing and comparing the performance of these two approaches in sentiment analysis using Twitter data. By analyzing both approaches which are lexicon-based and deep learning-based, we can get insights into their strengths, limits, and potential applications in extracting sentiment information from social media content [4] . Twitter has become a popular forum for users to share their thoughts on various issues, including products, services, events, and current events [5] . Its microblogging nature, with restricted characters in each tweet, poses a unique obstacle for sentiment research. Because of the brevity of tweets, specialized procedures are required to capture the sentiment nuances effectively.
The lexicon-based approach provides a simple and interpretable mechanism for sentiment analysis. By correlating words with sentiment polarity in a predetermined lexicon, it can instantly classify the sentiment of a tweet based on the presence of positive or negative phrases [6] . However, lexicons may not capture the context-dependent mood, irony, or slang, resulting in erroneous results. Deep learning-based techniques, notably LSTM-based models, have shown considerable potential in capturing contextual information and dependencies inside text sequences [7] . These models can effectively capture the sentiment represented in lengthier and more complex tweets. They can learn from enormous amounts of data and adapt to varied language patterns and user expressions. Deep learning models, on the other hand, necessitate significant computational resources and a large amount of training data to reach optimal performance [8] [9] . This study compares the strengths and disadvantages of lexicon-based and deep learning-based approaches to sentiment analysis on Twitter data [10] . It will evaluate their accuracy, efficiency, adaptation to diverse domains and languages, and capacity to deal with obstacles such as noisy data, sarcasm, and context-dependent sentiment. The findings of this study will help researchers, practitioners, and decision-makers choose the best strategy for sentiment analysis tasks using Twitter data.
Furthermore, this research will help to expand sentiment analysis techniques and increase the researcher’s knowledge of the sentiments and opinions expressed by people on social media sites. The findings may have repercussions in a variety of fields, including marketing, public opinion analysis, customer feedback analysis, and political attitude analysis. Ultimately, the goal is to employ sentiment analysis to gather important insights, make educated decisions, and improve user experiences in the age of social media and online communication. Study contributions are listed below:
· This study compares lexicon-based and deep learning-based sentiment analysis methods for sentiment analysis, specifically Twitter posts.
· A transparent and reproducible method for data collection, labeling, preprocessing, and model evaluation is described.
· Showed that deep learning model can achieve substantial performance for sentiment analysis on Twitter, reaching 98% accuracy.
· Wе usеd accuracy, rеcall, F1-Score, and confusion matrix to compare dееp learning with lеxicon-basеd mеthods on a largе Twittеr datasеt.
· Showеd long short-tеrm mеmory (LSTM) nеtworks as an еffеctivе dееp lеarning architеcturе for sеntimеnt classification of short social mеdia posts.
· Analyzed trends and performance trade-offs between the two approaches— deep learning had higher recall while lexicon was more precise.
The structure of the paper is as follows: It contains an overview of relevant research and current knowledge in the field in the Literature Review section. The methodology part is divided into ten sections, which arе: Proposеd Systеm, Data Dеscription, Data Prе-Procеssing, Data Visualization, Labеl Encoding, Fеaturе Ex-traction using Word Embеdding, Modеl Training, Optimization Algorithm, Callbacks, and Lеxicon Approach. Following arе thе rеsults: thе Confusion Matrix, Accuracy, Prеcision, Rеcall, and F1-scorе arе all listеd in thе Expеrimеntal Rеsults sеction. During the Discussion part, the comparison analysis’s primary results are examined. As a final touch in conclusion, the paper summarizes the main study results and essential insights.
2. Literature Review
This section contains an overview of the relevant works and studies associated to the comparative analysis.
M. Sani et al. [11] analyze Hausa tweets’ emotions using machine learning. The research examines how machine learning and lexicon-based techniques might improve Hausa text categorization. The BBC Hausa Twitter API provided the paper’s dataset. Two machine learning-based classification techniques were Multinomial Naive Bayes (MNB) and Logistic Regression (LR) employing Count Vectorizer and TF-IDF. LR classifies Hausa texts 86% better than MNB. The paper examines how Hausa’s informality and lack of structure make sentiment analysis challenging. The paper’s focus on Hausa tweet sentiment analysis limits its applicability to other text data.
M. ur-Rehman et al. [12] evaluate actual time sentiment analysis with SVM, K-SVM, and Multinomial Naive Bayes. Twitter and contested territory news websites are used for the study. The work examines important supervised machine learning classifiers for real-time sentiment analysis and suggests a technique for new researchers to investigate real-time sentiment correlation between pairs of countries to predict conflicts with substantial casualties. This study employs SVM, Kernel SVM, and Multinomial Naive Bayes for sentiment analysis. The Kaggle dataset trains real-time sentiment classifiers. K-SVM’s real-time positive sentiment recognition is 80% accurate. Multinomial Naive Bayes and Real-time SVMs performed poorly. This study examines how real-time data distorts sentiment analysis. Update language models, add data sources, and adapt sentiment analysis algorithms to language use.
N. Imanina Zabha et al. [13] they have worked on sentiment analysis by focusing on lexicon methods for the cross-lingual sentiment analysis from Twitter data. There final result is revealed that the classifier was able to determine the sentiments finally. A. Mitra et al. [14] worked on a movie data set with the help of lexicon approaches and tried to find out which movie is best or what type of categories movie people would like to watch, which will enhance the movie producer or theater owner to adopt that kind of categories movies for public to make a good business out of it.
M. S. Hajrahimova and M. I. Ismaylova et al. [15] analyze Twitter users’ COVID-19 attitudes using machine learning. This article analyses pandemic tweets’ emotional “color” using Support Vector Machine, Naive Bayes, Random Forest, and Neural Network methods. Python and scikit-learn are used to test COVID-19 tweets from Kaggle. The Random Forest classifier, with 79.37% accuracy, performs best in experiments. The paper uses machine learning to study Twitter users’ COVID-19 pandemic reactions. The article only analyses COVID-19 tweets and excludes other factors that may affect Twitter users’ sentiments.
N. M. Sham and A. Mohamed et al. [16] want to find the best sentiment analysis approach for tweets on climate change and related topics by comparing multiple methodologies. The paper compares seven lexicon-based methods, three machine learning classifiers, and hybrid algorithms using climate change tweets. The hybrid technique outperformed the other two with an F1-Score of 75.3%. Lemmatization improved machine learning and hybrid techniques’ accuracy by 1.6%, whereas TF-IDF feature extraction was marginally better than BoW, increasing the Logistic Regression classifier by 0.6%. Near the end of the paper, the authors recommend additional research on deep learning algorithms and do-main-specific sentiment in social media.
B. S. Ainapure et al. [17] examined Indian tweets about the COVID-19 pandemic and vaccine campaign. Deep learning and lexicons sort sentiments. Training a recurrent neural network with Bi-LSTM and GRU techniques yields 92.70% and 91.24% on the COVID-19 dataset and 92.48% and 93.03% on vaccination tweet categorization. Healthcare workers and governments can use the models to make pandemic decisions. The study only considers Indian opinions.
R. Srivastava et al. [18] compare unsupervised lexicon-based system and machine learning for sentiment analysis. The dataset included 20,000 TripAdvisor hotel reviews. The authors evaluated their performance after training and testing classifiers on cleaned and preprocessed data. The Support Vector Machine model achieved 96.3 percent TFIDF accuracy and 88.7 percent VADER lexical correctness. Machine learning outperforms un-supervised lexicon-based sentiment analysis in this paper’s comparison. Its single dataset may make extrapolating its conclusions to other datasets difficult.
Zvonarev et al. [19] compare Logistic Regression (LR), Convolutional Neural Network (CNN), XGBoost classifier, and models for Russian tweet tone analysis. The study compares two binary classification-based text tone analysis methods. The dataset is Russian tweets, and models are compared for accuracy and F1-Score. CNN has the highest accuracy (79.5%) but the longest computation duration. The study also indicates how Russian language unpredictability requires rigorous data pre-processing by any model. The report advises that scientists fine-tune the hyperparameters of boosting type models and broaden the methodologies used to improve the strategy.
N. Braig et al. [20] analyze sentiment analysis literature using COVID-19 Twitter data. This research informs policymakers and public health experts on how to use sentiment analysis to stop COVID-19. Researchers examined 40 publications and five databases from October 2019 to January 2022. BERT and RoBERTa models perform best on Twitter data, although ensemble models with various machine learning classifiers perform best overall. The report includes a summary of canonical ML classification techniques and a complete list of characteristics. Lexicon-based sentiment analysis approaches, which were not included in the study, could improve the research.
Although previous research has looked at sentiment analysis methods that utilized lexicons or machine learning, there hasn’t been any direct comparison of different methods on the same dataset to measure how they differ in terms of accuracy, precision, recall, and other metrics. A lot of research has limited generalizability by focusing on specific languages or domains, such as Hausa or COVID-19 tweets. Very few studies compared different methods simultaneously. Furthermore, researchers were unable to compare state-of-the-art methods because most studies lacked a reliable experimental methodology. By comparing deep learning with lexicon-based sentiment analysis on a massive English Twitter dataset, this study hopes to fill up some of those gaps. The benefits and drawbacks can be better understood with the use of a quantitative and systematic evaluation approach that examines not only accuracy but also other indicators. To serve as a standard for the community, the study also releases the code for data preprocessing, model implementation, and evaluation. This work adds to our scientific knowledge of the best methods for social media text by thoroughly comparing the top approaches on standard sentiment analysis datasets. Table 1 presents the comparison at a glance between different research experiments on sentiment analysis based on short texts.
3. Methodology
This part detailed the proposed approach, the system model, and how we completed this task.
3.1. Proposed System
In this section, we outline the methodology workflow, and the overall workflow is visually presented in Figure 1.
· Obtain the dataset from the targeted Twitter platform.
· Label the collected data for subsequent processing.
· Cleanse the dataset through various pre-processing steps.
· Divide the dataset into two segments: Training and Testing sets.
· Employ deep learning techniques, specifically Long Short-Term Memory (LSTM), to train the model using the training set.
· Assess sentiment detection accuracy using the testing set with the trained deep learning model.
· Apply the lexicon-based approach to test sentiment detection accuracy using the same testing set.
· Conduct a comparative analysis of the accuracies obtained from the deep
![]()
Table 1. Comparison table with the existing works related to short text and short texts.
learning model and the lexicon approach.
3.2. Data Description
The Sentiment140 dataset [21] , managed by Stanford University, was used to train the algorithm. There are 1.6 million tweets in it. The polarity, tweet ID, date, username, and tweet text are all included in this dataset. The polarity of the tweet and its language are the two most important columns for the goals of this project.
There were no tweets with neutral designations despite the suggestion of a neutral class. This dataset consists of two equally balanced types (positive and negative) with no skewness. There was no need to use any target class balancing strategies because the dataset was balanced correctly.
The columns of the dataset are shown in Figure 2. Features like tweet IDs, tweet posting dates, and usernames have been dropped because they don’t improve categorization. It is clear from the sentiment column that 0 denotes a negative and 4 denotes a positive. According to the general labeling guidelines of sentimental analysis, the positive class, formerly represented by 4, is now mapped to 1. With a split ratio of 0.8, 0.1, and 0.1, the dataset was divided into train, validation, and test sets for the experiment. Here the K-fold, cross-validation is applied for the training to strengthen the approached model of deep learning. This is a good split ratio, according to AY Ng; the main test set size is significantly much less than from the train set. From Reddit the test data are scraped which is for some months and can close to almost 100,000 comments, accurately similar to the Senti-ment140 dataset, this fits in exceptionally well with the project’s framework. The size of the validation and test sets is the same. The optimal approach is shuffling the entire dataset before splitting it, which strengthens the model. For our research work, we found a good amount of data to train our model. In our dataset, there were no null values, which proves that there is no missing data on this Dataset. To check if our dataset is balanced or not, we used the data distribution plot shown in Figure 3. Equal distribution of data is a sign of a good dataset.
![]()
Figure 2. Main data in sentiment140 dataset.
3.3. Data Pre-Processing
Text model processing is an important part of working with Twitter data. Because Twitter tweets are limited in length and contain noise, it is important to deal with data spread, use the right methods, and put an emphasis on cleaning. Twitter users can only use a limited number of characters to say what they want to say, so preparing short and frequent data for labeling is necessary.
Prior processing gets rid of unnecessary and duplicate data so that the focus can be on the message itself. This not only makes it easier to pull out meaningful material, but it also makes computing simpler.
As shown in Figure 4, the dataset now has column names for “sentiment,” “id,” “date,” “query,” “user_id,” and “text.” After that, columns like “id,” “date,” “query,” and “user_id” that were not needed for mood analysis were taken out of the dataframe. Also, “0” and “1” were changed to “Negative” and “Positive,” respectively, for the sentiment numbers.
3.3.1. Cleaning Dataset
Now the “text” column contains neat text. Tweet content frequently includes references to other users, linked text, emojis, and punctuation. We cannot allow certain texts to be used for training a model to learn utilizing a language model. We must thus use a variety of preprocessing and cleansing techniques to clear the text data.
The following actions were taken to sanitize the data:
· HTML decoding was used to generate the text in the tweets. It is considerably easier to decode the text using the Python module Beautiful Soup.
· The word “@” is frequently used in the text, but it serves no purpose in this context and can be removed because it adds nothing meaningful to the tweet.
· URL addresses are also unnecessary and can be disregarded for this purpose.
· Contradiction Words “is not” from “it’s not”; “doesn’t” from the “does not” have been mapped using mapping.
When the punctuation has been removed; then some of the newly acquired words lose all meaning. For instance, “isn’t” would become “isn’t” once the punctuation is removed. A dictionary is constructed to deal with such negative words, as seen in Figure 5, where the original words serve as the keys and the new words as the values. Now the dictionary replaces every negative word that fits the key with its corresponding values—these aid in improved sentiment recognition.
By changing the word’s case to lowercase, the punctuation (unique letters) is deleted, and the case insensitivity issue is resolved. The words in these sentences are later rejoined after being tokenized. All 1.6 million tweets go through this five-step process in groups. There are 16 batches total, with 100,000 tweets in each set. Figure 6 shows the entire cleaning procedure in pictorial form. After cleaning, the data frame had roughly 3500 null elements removed because they weren’t helpful for the analysis. In the above figure, we can see the whole cleaning
![]()
Figure 5. Dictionary built to handle negative words.
process of data that we have done for implementing our work.
3.3.2. Text Pre-Processing
Short text content frequently includes references to other users, links, emojis, and punctuation [22] . We cannot allow specific texts to be used for training a model to learn utilizing a language model. Thus, we must use various preprocessing and cleansing techniques to clear the text data.
For many grammatical reasons, this research has employed several word forms, which can be like write, and writing, and writes. There were also some words of families with analogous derivations and similar meanings. Stemming often refers to a technique that removes derivational affixes from words and chops off their ends in the hopes of attaining the goal most of the time.
Lemmatization and stemming techniques are utilized to decrease the number of words that convey comparable sentimental connotations and to convert words from their complete form to their truncated parent form. Lemmatization is a method for reducing words to their lemma using a linguistic dictionary [23] . Linguistic accuracy preservation is a common practice that involves the utilization of language-specific lemmatization algorithms or rule-based methodologies.
The stemming algorithm produces the stem form of a given word. Non-STEM words are subjected to stemming and subsequently substituted with their corresponding stem words. For example, ‘like’, ‘liken’, ‘likewise’, ‘liking’, and ‘likelihood’ has been substituted with the term ‘lik’. It helps to improve model performance, grouping similar words, easier to analyze and understand [24] .
3.3.3. Mention with Hyperlinks
In the context of sentiment analysis of textual data, it is frequently imperative to perform text cleaning procedures, which involve the elimination of hyperlinks, mentions, and punctuation marks. This procedure facilitates the extraction of the factual information conveyed by users, with a specific emphasis on the text pertinent to sentiment analysis. To determine the optimal sentiment combination, it is crucial to perform text cleaning procedures such as hyperlink removal [25] .
On social media, people post videos and blogs on the internet and mention tag other users. As a result, many tweets contain hyperlinks and Twitter mentions. Examples cited by Twitter users include @switchfoot and @Kenichan.
Examples of hyperlinks https://www.youtube.com, https://www.google.com, we used a regular expression to remove hyperlinks and mentions from the text [26] . Since we only need the actual texts that users express from their minds for sentiment analysis, the second step of my text cleaning is to remove all mentions from tweets. In my code, “HTTPS?: S+” and “HTTP. I’ve used the regular expression “@S+” to remove mentions. I’ve used the word “[A-Za-z0-9]+” to remove punctuation.
3.3.4. Stop-Words
Stop-words are often used for ‘in, the, I’ etc. which have minimal meaning, and also do not help decipher the spirit of the statement. By using some built-in design for the English language, such terms can be eliminated from the vocabulary [27] .
Making a list is an additional strategy. Sorting the terms in decreasing order of frequency will do this. The vocabulary size can be reduced by eliminating stop words that don’t contribute to the analysis. Stop words are frequently used terms in English that have no semantic context. As a result, we got rid of them before classifying them. In Figure 7, some stop words have been shown for our dataset.
3.4. Visualization of Data
After cleaning the data, we visualized it by dividing it into two separate word clouds, one for positive words shown in Figure 8 and the other for negative words shown in Figure 9, based on the words that occurred most frequently in the dataset. Here are some positive words we have found, which are mostly used in the positive posts.
Here are some negative words we have found, which is mostly used in negative posts.
![]()
Figure 7. Example of Stop words of this study.
After the pre-processing, we divided the train/test ratio into 80 percent and 20 percent. Therefore, the amount of the data for the train is 1.2 million, whereas the size of the data for the test is 320 thousand.
Tokenization
Tokenization is the process of breaking down a character’s sequence into smaller sections known as tokens, as well as occasionally eliminating some characters such as punctuation. The example of tokenization [28] is shown in Figure 10.
Tokenizer generates tokens for each word in the data corpus and then maps them to an index using a dictionary [29] . The index for each word is included in the word index. Vocabulary size represents the total number of words in the data corpus. Following that, the vocabulary size increased to 290,575. We already have a tokenizer object, which can convert any word into a dictionary key (number). We must provide it with a set of numbers. Additionally, we must make sure that the input shapes of the sequences are uniform. They ought to all be the same length. However, tweets’ messages vary in their word counts. We
![]()
Figure 10. Tokenization example of this research.
asked Pad Sequence for a little assistance to complete my work to prevent this. The MAX SEQUENCE LENGTH fixed length will be used for the whole sequence. We are developing the encoded class prediction model (0 or 1 as this is a binary classification). As a result, we turned my training labels into encodings.
3.5. Feature Extraction Using Word Embedding
We use the Word Embedding technique to represent words as variables in text processing. This helps us to analyze patterns and context nuances in the language to gain new understanding. Embedding is a popular way to define a word in a document. This allows us to understand context, semantics, syntactic similarities, and word relationships in a document. We could relearn the pattern by using Word Embedding in our NLP application, but this would be time-consuming and inefficient. So, we use pre-built Word Embedding, such as GloVe or Word2Vec, to give extra meaning to words. These pre-models can be very useful for tasks such as classification. In our specific case, we use Glove Embedding from Stanford AI to optimize and contextualize text. That way we can improve our NLP model without reinventing the wheel.
3.6. Model Training-Long Short-Term Memory (LSTM)
The word cloud reveals that some terms appear often in both good and negative tweets. This might be a problem when employing training data such as Naive Bayes, SVD, or others. As a result, we use sequence models.
Sequence Model
In this research, we have implemented a sequence model shown in Figure 11. Recurrent neural networks can analyze data sequences and learn input sequence patterns to create output that can be a video sequence or a scalar value.
In our instance, the neural network predicts a scalar value. Here is the model architecture we have used:
· Embedding Layer: For every input we generate Embedding Vector sequence is what we use for model architecture. Conv1D Layer: It is used to divide large feature vectors of data into smaller ones.
· Long Short-Term Memory (LSTM): An RNN variant [30] includes a memory
state cell that could be used to remember the context of letters farther into the text in the ability to talk contextual meaning instead of always close words as in RNN [31] .
· Dense: Classification using dense connected fully layers.
This research used Adam optimization for Gradient Descent. Callbacks are special functions that are triggered after each epoch to accomplish certain tasks [32] . Two callbacks were used in this scenario. To improve the results, the LRScheduler callback alters the learning rate at a specified epoch. In this notebook, the learning rate remained constant for the first ten epochs before falling exponentially. Model Checkpoint saves the best model during training depending on specific metrics. The model is preserved with the lowest validation loss in this situation. The model generates prediction scores ranging from 0 to 1, which are then classified into two classes based on a threshold value. If the score is greater than 0.5, it is classified as positive, and this serves as the categorization threshold.
3.7. Lexicon Approach
The Lexicon-based approach [33] is a widespread sentiment analysis technique that depends on the existence of specific words or phrases in a document to infer sentiment. This method employs a sentiment lexicon, which is a pre-defined dictionary comprising words or attributes along with their associated sentiment values. Based on this lexicon, each term in the paper is correlated with sentimental polarity and conviction. To compute the overall polarity score of a document, the lexicon is matched against the document, and the occurrence of positive and negative words is calculated. The sentiment orientation of the document is then assigned using the following formulas [34] :
x1 (s) = 1 if sum (l, s) > 0 (1)
x1 (s) = 0 if sum (l, s) = 0 (2)
x1 (s) = −1 if sum (l, s) < 0 (3)
Here, sum (l, s) represents the difference between the total number of positive and negative sentiment words discovered in the document. A positive sum implies a favorable sentiment, whereas a negative sum suggests a negative sentiment. However, this strategy has limits. While lexical algorithms can produce practically faultless results, they are dependent on the availability of a lexicon, which may not be available in all languages. Furthermore, the Lexicon-based approach may struggle with sarcasm, context-dependent mood, and language nuances.
To generate sentiment lexicons, researchers use a variety of methodologies, including manual compilation, lexical methods, and corpus-based approaches. The lexicon can be enlarged by including synonyms, WordNet, SenticNet, or utilizing emoticons and emojis as sentiment indicators. In summary, the Lexicon-based approach to sentiment analysis uses sentiment lexicons to match words and phrases in a document, allowing for the computation of an overall polarity score. While it provides a simple and effective solution, it does have limitations in terms of language availability and contextual nuances. Researchers continue to improve sentiment lexicons to improve accuracy and capture sentiment in text more thoroughly.
A Python of package 2, package 3 called Text-Blob, is used for the process of textual data [35] . By using a straightforward API for getting started with some typical NLP activities, which can be a part-of-speech for tagging, which can be noun phrases for extraction, sentiment analysis for data, classification for the data, translation for the data, and other things. Text-Blob plays well with NLTK and pattern while standing on their enormous shoulders.
4. Experimental Result
Training and extrapolating results are fraught with errors. Complex models reduce training errors because the error rate decreases with complexity. Bias-variance decomposition (Bias + Variance) reduces the frequency of erroneous generalizations. “Overfitting” occurs when a drop in training error rates increases test error rates. Accuracy, accuracy, recall rate, and F1-Score can be used to evaluate categorization strategies. Writers used many approaches to evaluate their models. Most studies looked at many success markers, but some used just one. We assess the model accuracy, precision, recall, and F1-Score. This four-factor structure works effectively for analyzing prediction data. The ability to notice and classify events is linked to accuracy. Equation (4) [36] gives the formula:
(4)
Precision is the degree to which observed events match predicted ones in statistics. Equation (5) [37] expresses precision mathematically. The formula is:
(5)
The recall of a classification model represents the fraction of relevant instances that the model correctly identifies. It is a measure of the model’s ability to find all the positive samples in the dataset. Here is the formula (6) [38] for determining recall:
(6)
This technique provides a balance between precision and memorability, hence the term “harmonic mean”. Formula (7) [39] can be used to calculate the F1-Score.
(7)
4.1. Performance Evaluation
Within this phase, we will assess our work performance, which will explain the precision of our work. A deep learning model (DL) and a lexicon-based approach were compared in terms of performance. Confusion matrices were constructed for both methods, with Figure 12 displaying the deep learning model’s confusion matrix and Figure 13 indicating the lexicon-based approach’s confusion matrix. The confusion matrix evaluates the models’ accuracy and efficacy by demonstrating how well they predict distinct classes. True positives and true negatives are accurately classified cases, whereas false positives and false negatives represent misclassifications.
The performance evaluation also included a comparison of the classification reports from the deep learning (ML) model and the lexicon-based technique. The classification reports provide a full examination of the models’ performance, including recall, precision, and F1-Score for each class (negative and positive), as well as accuracy and other aggregated metrics.
4.2. Analysis of the Classification Report
The result analysis included comparing the classification reports from the deep learning (ML) model and the lexicon-based technique. The classification reports provide a full examination of the models’ performance, including precision, recall, and F1-Score for each class (negative and positive), as well as accuracy and
![]()
Figure 12. Confusion matrix of deep learning model.
![]()
Figure 13. Confusion matrix of lexicon-based technique.
other aggregated metrics.
Table 2 shows the categorization report for the deep learning model (DL). It gives an overview of the model’s precision, recall, and F1-Score performance for both the negative and positive classes.
The total accuracy of the deep learning model is stated to be 0.98. The macro average for both courses is 0.86, recall is 0.96, and F1-Score is 0.87. The weighted
![]()
Table 2. Classification report of deep learning model.
![]()
Table 3. Classification report of lexicon-based model.
average has a precision of 0.84, a recall of 0.98, and an F1-Score of 0.86.
Table 3 shows the categorization report for the lexicon-based technique. Precision, recall, and F1-Score for negative and positive classes are similar to the deep learning model’s classification report.
The total accuracy of the deep learning model is stated to be 0.95. Furthermore, the macro average (across both classes) shows an average precision of 0.84, recall of 0.91, and F1-Score of 0.88. The weighted average accounts for class imbalance and yields precision of 0.82, recall of 0.90, and F1-Score of 0.84.
Comparing the deep learning model with lexicon-based approach’s classification reports indicates their performance. The deep learning model had 0.85 precision, while the lexicon-based technique had 0.90. However, the deep learning model had stronger recall (0.93) than the lexicon-based approach (0.92). The deep learning model had a higher F1-Score (0.88) in the negative class than the lexicon-based strategy (0.85). Precision (0.93 vs. 0.88) and recall (0.94 vs. 0.90) were better for the positive class using the lexicon-based strategy. In the positive class, the deep learning (0.82) outperformed the lexicon-based technique (0.80). The deep learning model outperformed the deep learning model with 0.98 accuracy. Also, the deep learning model performed better across both classes in precision, recall, and F1-Score criteria.
5. Discussion
The discussion section seeks to go deeper into the sentiment analysis research using lexicon-based and machine-learning approaches, with a particular focus on Twitter data. The findings will be thoroughly analyzed in this section, along with the study’s significance, the results’ ramifications, and prospective future research directions. The sentiment140 Twitter dataset was utilized in the study to compare two widely used approaches: the lexicon-based method and the deep learning method. With more than 1.6 million data points in the dataset, there is a significant amount of data for analysis and inference.
The study’s findings showed that the Long Short-Term Memory (LSTM) model, used especially in the deep learning approach, produced an amazing accuracy score of almost 98%. On the other hand, the lexicon-based strategy produced an accuracy score that was 95%. These results demonstrate the superiority of the deep learning strategy over the lexicon-based approach in sentiment analysis tasks, demonstrating its capacity to collect and understand sentiment accurately. The lexicon-based method’s intrinsic constraints can be blamed for the considerable accuracy gap between the two approaches. Lexicons are dictionaries or databases that include word or phrase sentiment scores that have already been determined. Although lexicons are a useful tool for sentiment analysis, they frequently miss the subtleties and context-specific meanings of words, producing less accurate results. Deep learning models, like LSTM, on the other hand, have the advantage of learning directly from the data and identifying intricate patterns and connections in the text. With more training data and this ability, they can adapt to various situations and gain accuracy over time.
The results of this study have several ramifications for sentiment analysis. They emphasize the necessity of using deep learning techniques, especially deep learning models like LSTM, for more precise sentiment classification. These algorithms are capable of accurately capturing the fine details and situation-specific moods expressed in social media messages. Second, the study emphasizes the importance of using expansive datasets for training and assessing sentiment analysis models, such as sentiment140. The sheer amount of data makes it possible to train robust models and conduct thorough evaluations, producing outcomes that are more trustworthy and generalizable.
The study also supports the idea that sentiment analysis on social media sites like Twitter is essential for determining the attitudes and sentiments of the general population towards certain issues. For companies, marketers, and decision-makers to make wise choices, comprehend client preferences, and raise customer happiness, accurate sentiment research can offer useful information. Despite the noteworthy results, this study has certain restrictions. The attitudes expressed in other domains or on other social media platforms may not be fully reflective of the analysis because it was specifically focused on the sentiment140 dataset and Twitter data. Furthermore, the study mainly concentrated on categorizing sentiment as either positive or negative; however, future research may explore more nuanced sentiment analysis.
Future studies should look into the possibility of merging lexicon-based and machine-learning methodologies to maximize the benefits of each approach. Furthermore, investigating additional robust deep learning architectures and combining domain-specific expertise could improve the precision and efficacy of sentiment analysis models.
6. Conclusion
We emphasized the lexicon-based technique and deep learning methods in this research. Our main contribution is to the study of popular sentiment-related posts from Twitter. We have explored many topics that are mostly very usable and that people want to discuss with each other. People want to share their opinions on some of the specific topics. Mostly they spend very busy time in their life but there are some specific criteria because people usually love to enjoy spending time on social media. For this work, we need a huge amount of data which might help us to get the desired accuracy rate. For that, we started to find the dataset from Kaggle. Likely we found a dataset made during the COVID-19 situation when most people have used social media a lot. This dataset carries almost 1.5 million data. So, this was the actual key point for our work. We collect the dataset which already has been in labeled condition. After that, we ran the clean process and made it ready to apply the model and the different approaches. For applying the model, we needed to split the dataset into train and test data. We split the dataset into 0.8 and 0.2 percent. To train our model we have used the train part data. After that, we used the test data to measure the output accuracy for the deep learning model. On the other hand, we have used the same test part data for measuring the output for the lexicon approach. Finally, we have compared the two different parts’ outputs. We have analyzed many related works and discovered that many people currently using the lexicon approach for sentiment analysis works. But based on our work we can find out that the deep learning approach can give a better accuracy for the sentiment analysis work. So, if the researcher works on a deep learning approach with various models, then with sentiment analysis, he will be able to get a better accuracy than other approaches. For this reason, our work will get better responses as proof of a better ac-curacy method.
Author Contributions
Conceptualization, T.I., M.A.S, M.S.T and M.M.R.F; methodology, T.I., M.R.S. and M.M.R.F; visualization and data collection, M.S.T., M.A.S.; validation, T.I. and M.A.S; investigation, M.R.S. and M.M.R.F; resources, M.A.S; data curation, T.I. and M.B; writing—original draft preparation, T.I, M.M.R.F and M.A.S; editing, M.S.T, T.I and M.A.S.; supervision, J.F and M.B.; project administration, T.I. All authors have read and agreed to the published version of the manuscript.