My areas of research are machine learning and multimedia forensics. Specifically, I work under the DejáVù Project on the development of an automated method to determine whether a piece of social media data is related or not to a given forensic event.
When a forensic event of large scale happens, immediately visual content related to it is shared on social networks. These data might be potentially useful for a posterior forensic inspection, given that they can depict different views in different moments of the event. However, an analysis of social media imagery related to an event might be harmed by the abundant number of irrelevant items that are retrieved by a collection procedure, such as memes and images from previous events. Manually sanitizing the dataset at hand is unfeasible, since it might contain thousands of items. To tackle this problem, we study the employment of machine learning techniques to speed up the procedure and reduce the required human force. In detail, our work follows four paths. The first one aims to provide a good representation for images, exploring different pre-trained convolutional neural networks and feature fusion; the second one targets including humans to the loop of the machine learning pipeline, by employing instance selection and active learning techniques; the goal of the third one is to perform classification with few samples, using semi-supervised techniques that vary from graph-based methods to graph neural networks; and the last one aims to incorporate knowledge from previous events to the machine learning pipeline, using domain adaptation and available datasets from previous events. These four paths are promising, and can improve the performance of methods for this task.
@inproceedings{nascimento2022,title={Few-shot Learning for Multi-modal Social Media Event Filtering},author={Nascimento, J. and Cardenuto, J. P. and Yang, J. and Rocha, A.},booktitle={2022 IEEE International Workshop on Information Forensics and Security (WIFS)},year={2022},organization={IEEE},}
