Sentiment analysis, the process of automatically detecting and classifying emotions from textual and visual data, has gained significant attention in recent years due to the explosive growth of user-generated content on various online platforms. While traditional sentiment analysis focused primarily on text data, the emergence of social media and the prevalence of multimedia content have necessitated the development of sentiment analysis systems capable of analyzing images and text. We conducted a study on various CNN models for image sentiment analysis. This method helps us understand people's emotions and moods by analyzing their facial expressions. Our project aims to find the best set of CNN models for image sentiment analysis. Next, we proposed a late fusion-based sentiment analysis system for images to extract and categorize sentiments accurately. Experimental work using the well-known and challenging “Face Expression Recognition” dataset showed that the late fusion-based model achieved 81%, 80%, 81%, and 80% accuracy, precision, recall, and F1 score, with an average improvement of 5.2% to 12.5% compared to the performance of the baseline models.