Featured AI Research

Recurrent Semantic Instance Segmentation

Salvador A, Bellver M, Baradad M, Campos V, Marqués F, Torres J, et al.. From Pixels to Object Sequences: Recurrent Semantic Instance Segmentation. In Submitted.

We present a recurrent model for semantic instance segmentation that sequentially generates pairs of masks and their associated class probabilities for every object in an image. Our proposed system is trainable end-to-end, does not require post-processing steps on its output and is conceptually simpler than current methods relying on object proposals. We observe that our model learns to follow a consistent pattern to generate object sequences, which correlates with the activations learned in the encoder part of our network. We achieve competitive results on three different instance segmentation benchmarks (Pascal VOC 2012, Cityscapes and CVPPP Plant Leaf Segmentation).

Learning to Skip State Updates in Recurrent Neural Networks

Campos V, Jou B, Giró-i-Nieto X, Torres J, Chang S-F. Skip RNN: Learning to Skip State Updates in Recurrent Neural Networks. In International Conference on Learning Representations (ICLR). 2018.

Recurrent Neural Networks (RNNs) continue to show outstanding performance in sequence modeling tasks. However, training RNNs on long sequences often face challenges like slow inference, vanishing gradients and difficulty in capturing long term dependencies. In backpropagation through time settings, these issues are tightly coupled with the large, sequential computational graph resulting from unfolding the RNN in time. We introduce the Skip RNN model which extends existing RNN models by learning to skip state updates and shortens the effective size of the computational graph. This model can also be encouraged to perform fewer state updates through a budget constraint. We evaluate the proposed model on various tasks and show how it can reduce the number of required RNN updates while preserving, and sometimes even improving, the performance of the baseline RNN models.

Detection-aided Liver Lesion Segmentation Using Deep Learning

Bellver M, Maninis K-K, Pont-Tuset J, Torres J, Giró-i-Nieto X, van Gool L. Detection-aided liver lesion segmentation using deep learning. In ML4H: Machine Learning for Health Workshop at NIPS 2017. Long Beach, CA, USA; 2017. Google Scholar BibTex

A fully automatic technique for segmenting the liver and localizing its unhealthy tissues is a convenient tool in order to diagnose hepatic diseases and assess the response to the according treatments. In this work we propose a method to segment the liver and its lesions from Computed Tomography (CT) scans using Convolutional Neural Networks (CNNs), that have proven good results in a variety of computer vision tasks, including medical imaging. The network (Fig \ref{arch}) consists of a cascaded architecture, which first focuses on the region of the liver in order to segment the lesion on it. Moreover, we train a detector to localize the lesions, and mask the results of the segmentation network with the positive detections. The segmentation architecture is based on DRIU~\cite{maninis2016deep}, a Fully Convolutional Network (FCN) with side outputs that work on feature maps of different resolutions to benefit from the multi-scale information learned by different stages of the network. The core of our network for lesion and liver segmentation consists in using the strength of a segmentation network plus a detection network to localize the lesions. As the output of the segmentation network is pixel-wise, it tends to trigger false positive pixels, since no constraints for more global decisions are imposed. A lesion detector decides if a complete patch is healthy or not, without constraints on the exact shape of the lesion. Having both techniques analyzing the input image yields a better overall result for our own validation set and the test set of the Liver Tumor Segmentation (LiTS) Challenge. Regarding the LiTS test set, before using detection we achieve a 0.54 of dice score for the lesion segmentation. After adding the detector to the pipeline, the result increases to 0.57. As a post-processing, we apply a 3D-Conditional Random Field, achieving a 0.59 of dice score as final result. An example of the liver and its lesions prediction and ground truth is in Fig. \ref{result}. We believe that using segmentation together with detection is an interesting direction for medical image segmentation pipelines, which typically deal with very small structures, being a detection-aided segmentation beneficial to localize the target region.

List with all the papers: Research Publications