The Overlooked Repetitive Lengthening Form in Sentiment Analysis

A subtle but common way people express emotion online has been largely ignored by artificial intelligence systems, potentially limiting their ability to understand human sentiment accurately. When individuals write phrases like 'loooove' or 'amazing!!!!' on social media, they are using what linguists call the Repetitive Lengthening Form (RLF), a style where extra characters are added to words for emphasis. Despite its prevalence in informal communication, with an average of 5.8% of documents across various datasets containing RLF, this expressive feature has not been systematically studied in the context of AI-driven sentiment analysis. This oversight raises critical questions about whether AI can truly grasp the nuances of human emotion in digital conversations, where such informal styles are key to conveying feelings.

Researchers from Temple University have discovered that sentences containing RLF are significantly more effective at indicating the overall sentiment of a document compared to sentences without RLF. In a comprehensive study, they found that when predicting document-level sentiment using just a single sentence, those with RLF consistently outperformed those without across multiple AI models. For example, in zero-shot tests, accuracy and F1 scores were higher for RLF sentences, with models like RoBERTa achieving 85.94% accuracy for RLF versus 84.40% for non-RLF sentences. This pattern held across diverse domains, including books, electronics, restaurants, and social media, demonstrating that RLF sentences can serve as reliable signatures for sentiment, especially in short texts common on platforms like Twitter.

To investigate this phenomenon, the team created Lengthening, the first large-scale dataset focused on RLF for sentiment analysis, comprising 850,000 samples extracted from four public datasets. They designed a pipeline to identify RLF documents and sentences using regular expressions that match repeated letters and punctuation, such as 'loooove' or 'book!!!!!'. For each RLF sentence, they paired it with a non-RLF sentence from the same document to enable comparative analysis. The dataset spans five domains, with RLF ratios ranging from 4.32% in electronics reviews to 13.36% in Twitter posts, highlighting its widespread use. This resource allowed the researchers to evaluate and improve AI models' performance and explainability in handling RLF, addressing a gap in existing research.

Revealed that fine-tuned pre-trained language models (PLMs) like RoBERTa could surpass zero-shot GPT-4 in performance metrics, achieving up to 91.59% accuracy after training on Lengthening, compared to GPT-4's 86.26%. However, these models lagged in explainability, as measured by a novel unified approach that quantifies how much attention models pay to RLF words. For instance, fine-tuned RoBERTa had an explainability score of 0.22, while zero-shot GPT-4 scored 0.38, indicating that PLMs might be making correct predictions without fully understanding the expressive value of RLF. To bridge this gap, the researchers introduced Explainable Instruction Tuning (ExpInstruct), a two-stage framework that uses limited samples to instruct-tune open-source models like LLaMA2, enabling them to match GPT-4's performance and explainability, with scores reaching 0.35 and 87.20% accuracy.

This work has important for real-world applications, particularly in analyzing social media content where short, informal expressions are prevalent. The study showed that RLF sentences are especially effective for sentiment analysis when text length is under 80 characters, covering over 70% of such sentences, which aligns with the brevity of tweets and online comments. By improving AI's ability to interpret RLF, tools for content moderation, customer feedback analysis, and public opinion tracking could become more accurate and nuanced. Additionally, the transferability of gains from sentence-level to document-level analysis was confirmed, with fine-tuned models showing average improvements of 7.6% in accuracy and 4.5% in F1 score for full documents, suggesting broader benefits for natural language processing tasks.

Despite these advancements, the study acknowledges limitations that point to areas for future research. The dataset and s are currently focused on English, though RLF is also common in other languages like Spanish and Romanian, where similar expressive patterns exist. Expanding this work to multilingual contexts could enhance its global applicability. Furthermore, human evaluation revealed that while data quality was high, with an inter-rater agreement score of 0.86, there is room for improvement in explanation reliability, particularly for fine-tuned models. The researchers note that incorporating human correction into the instruction-tuning process could further boost model interpretability, and future studies might explore applying ExpInstruct to other model architectures or informal linguistic styles beyond RLF.