Current R&D projects
The DIGISTORM project (2016 - 2020)
Saliency models for video protection applications (2015 - ) - PhD Thesis Pierre Marighetto
Saliency is, in computer science, the way to model visual attention, which is the ability to selectively focus on an external stimulus and take into account the most important information.
Concerning video protection, many tools can be used. These aim a specific goal, as abandoned luggage detection, people tracking, abnormal events detection or crowd analysis. These methods need a ground-truth to learn normal and abnormal situations. However, these mechanisms usually fail in unlearned conditions. This issue could be resolved using saliency mechanism.
By merging one of these approaches with saliency models, we could catch abnormal events, events defined as salient and redefine the normality.
The SonixTrip project (2013 - 2016)
The PREDATTOR project (2012 - 2015)
Based on neuro-psychological and human attention modelling research, the Predattor prototype is able to automatically compute an attention map of any image. This map shows where YOU will gaze when you see an image providing results very close to eye-tracking data. Predattor focuses on neuromarketing and helps you optimize your website or online ads. Cheap and fast, you can use Predattor during the creative process until your key message is visible enough and until you stand out from the competition.
Thèse François Rocca (2011 - ) - PhD Thesis François Rocca
Thèse Radhwan Ben Madhkour (2009 - ) - PhD Thesis Radhwan Ben Madhkour
The RECITE project (2007 - 2009)
RECITE aims at extending OCR application for machine vision for objects with different surfaces (metal and so on) and with very various characters. Close to natural scene text understanding, this project focuses on interactively configurable recognition software in order to give access to non-experts people (in SMEs for instance). Hence, the main goal is to enable the creation of dedicated recognizers for particular applications. Based on smart dialogues between the computer and the end-user, particularities of the application, degradations embedded into images will be semi-automatically defined in order to build an efficient recognizer. Additionally, some challenges are met such as extraction and recognition of engraved/embossed characters, which are limitations of systems dealing with natural scene text. In that context, one example is first taken in order to make further the model more versatile: the recognition of engraved characters into metallic and reflective surfaces in uncontrolled environment.
The TRANSLOGISTIC project (2007 - 2011)
TransLogisTIC is an ambitious research project financed by the Walloon Region (2.5 years - 14 m. euros) which is built around a longterm strategy aimed at developing a complete and efficient multimodal transport system in Wallonia as well as high quality logistics services with high added value. Supported by internationnaly recognized Walloon actors (10 enterprises and 5 universities), the project will result in the creation of innovative and efficient products and services.
The HCR-NN project (1998 - 2002)
Off-Line Handwritten Character Recognition using Neural Networks
Past R&D projects
VISION: Video and Image Saliency Detection (2010 - 2016) - PhD Thesis Nicolas Riche
Human visual system receives 80% of our daily lives information but this amount of visual data physically exceeds the capacity of our brain. The mechanism that overcomes this fundamental issue and determines what part of the incoming ocular information is interesting and must be processed first is called visual attention. Since the early 2000s, modeling visual attention has been a very active research area.
This thesis adds a brick to this crucial endeavor of understanding and modeling human attention: after presenting several state of the art models, it proposes new methods illustrated through several practical applications. Besides, these algorithms need to be fairly evaluated which is the scope of the second part of the thesis by developing a new framework to assess saliency models.
Thèse Julien Leroy (2010 - 2016) - PhD Thesis Julien Leroy
Designing interaction for browsing media collections (by similarity) (2010 - 2015) - PhD Thesis Christian Frisson
Sound designers source sounds in massive and heavily tagged collections. When searching for media content, once queries are filtered by keywords, hundreds of items need to be reviewed. How can we present these results efficiently? This doctoral work aims at improving the usability of browsers of media collections by blending techniques from multimedia information retrieval (MIR) and human-computer interaction (HCI). We produced an in-depth state-of-the-art on media browsers. We overviewed HCI and MIR techniques that support our work: organization by content-based similarity (MIR), information visualization and gestural interaction (HCI). We developed the MediaCycle framework for organization by content-based similarity and the DeviceCycle toolbox for rapid prototyping of gestural interaction, both facilitated the design of several media browsers. We evaluated the usability of some of our media browsers.
Our main contribution is AudioMetro, an interactive visualization of sound collections. Sounds are represented by content-based glyphs, mapping perceptual sharpness (audio) to brightness and contour (visual). These glyphs are positioned in a starfield display using Student t-distributed Stochastic Neighbor Embedding (t-SNE) for dimension reduction, then a proximity grid optimized for preserving direct neighbors. Known-item search evaluation shows that our technique significantly outperforms a grid of sounds represented by dots and ordered by filename.
Simulation de la réponse de la rétine en conditions lumineuses mésopiques (2009 - 2013) - PhD Thesis Justine Decuypere
Simulation de la réponse de la rétine en conditions lumineuses mésopiques
The MediaTIC project (2008 - 2015)
The MediaTIC portfolio was submitted in September 2007 in response to the first call for proposals of the ERDF and started on 1st July, 2008. This ambitious project falls within the scope of measure 2.2 dedicated to the exploitation of the potential of research centres. More concretely, the project's objective is to increase the competitiveness of innovating technological SMEs in Wallonia through collective projects dictated by concrete industrial requests. It works as a cross-action for the innovation in the NTIC component of each strategic line defined by the Walloon Marshall Plan.
To reach that goal, Multitel, as a project leader, has gathered a consortium composed of academic entities and research centres split all over the Walloon territory. Actually, MediaTIC has been submitted in both objectives of the period for 2007-2013 of the European structural funds programme, namely "Convergence" and "Regional Competitiveness and employment". The project counts on the know-how of laboratories such as the SEMI, TCTS and Telecommunications units of the Faculté polytechnique de Mons, the TELE laboratory from the Catholic University of Louvain-la-Neuve, of the research units in microelectronics (Microsys) and signal & image processing (Intelsig) from the University of Liege, of the Centexbel and SIRRIS research centres and finally, of the GIE MUWAC. By calling upon complementary partners, Multitel aimed at providing MediaTIC with the typical action leverages of a collaborative research and allowing the projects focusing towards common objectives.
MediaTIC is a portfolio of six integrated projects oriented towards specific industrial needs. Each one is run by a specialist from Multitel in the targeted field. These thematic platforms are Transmedia, Envimedia, Tracemedia, Intermedia, 3Dmedia and Optimedia.
The NUMEDIART project (2007 - 2012)
Numediart is a long-term research programme centered on Digital Media Arts, funded by Région Wallonne, Belgium (grant N°716631). Its main goal is to foster the development of new media technologies through digital performances and installations, in connection with local companies and artists.
It is organized around three major R&D themes: HyFORGE - hypermedia navigation, COMEDIA - body and media, COPI - digital instrument making. It is performed as a series of short (3-months) projects, typically 3 or 4 of them in parallel, which are concluded by a 1-week "hands on" workshop.
Numediart is the result of collaboration between Polytech.Mons (Information Technology R&D Department) and UCL (TELE Lab), with a center of gravity in Mons, the cultural capital of Wallonia. It also benefits from the expertise of the Multitel research center on multimedia and telecommunications. As such, it is the R&D component of MONS2015, a broader effort towards making Mons the cultural capital of Europe in 2015.
The MAIS project (2004 - 2007)
The objective of MAIS is to develop a low-cost, low-consumption, secure smart card that will be readable from a distance.
The main applications of the project will be freight train tractability and inclusion in windshields. For this last application, the project partners work in close collaboration with Glaverbel.
The iMed project (2003 - 2006)
The iMed project is about the design of a method to automatically detect emboli in the vessel tree of the pulmonary artery, from HCT (helicoidal computed tomography) millimeter slices.
The MERCATOR project (2003 - 2007)
In the context of preoperative images visualization and computer-assisted surgical planning, the Mercator project aims at updating the plannings made
before the operation by integrating real-time information resulting from intra-operative events in order to readjust the plans and the initial data on
the real evolution during the operation or the radiotherapy.
The SYPOLE project (2003 - 2006)
The blind or partially sighted people represent 17.5 million people in Europe and
about 75.000 in Wallonia. For most of these people, much information, which
exists in written or imaged forms, is not easily accessible for them. The main
aim of Sypole project is to remedy all these needs by the realization of a
prototype device, which will be portable, autonomous, small-size and easy to use
for blind or partially sighted people. This kind of device will be able to
recognize text and coloured forms, such as logos, and to auto-generate a speech
The SIMILAR project (2003 - 2007)
The SIMILAR European Network of Excellence will create an integrated task force on multimodal interfaces that respond intelligently to speech, gestures, vision, haptics and direct brain connections by merging into a single research group excellent European laboratories in Human-Computer Interaction (HCI) and in Signal Processing.
SIMILAR will develop a common theoretical framework for fusion and fission of multimodal information using the most advanced Signal Processing tools constrained by Human Computer Interaction rules.
SIMILAR will develop a network of usability test facilities and will establish an assessment methodology.
SIMILAR will develop a common distributed software platform available for researchers and the public at large through www.openinterface.org
SIMILAR will address Grand Challenges in the field of edutainment, interfaces for disabled people and interfaces for medical applications.
SIMILAR will establish a top-level foundation which will manage an International Journal, Special Sessions in existing conferences, organize summer schools, interact with key European industrial partners and promote new research activities at the European level.
TCTS Lab's contibution will be on Grand Challenges related to TTS and ASR technologies, and their integration into a multimodal framework. We will also work on enhancing Brain Computer Interfaces. SIMILAR is considered a central project for the evolution of our lab.
ATTENTION (2003 - 2007) - PhD Thesis Matei Mancas
Attention is a simplification or filtering process which transforms a huge acquired unstructured data set into a smaller structured one while preserving the main information. All cognitive processes need attention; humans pay attention (consciously or unconsciously) from their birth to their death in every single moment. Attention is even used during the dreams and the R.E.M. (Rapid Eye Movements) sleep phase.
Nevertheless, attention is not specifically a human process but it is simply used by any living being from humans to insects.
Attention is the beginning of intelligence: there is no intelligence without attention!
Similarly to the fact that attention is the beginning of intelligence in biology, computational attention may be the starting point of artificial intelligence in engineering applications.
Computational attention provides machines with human-like reactions and behaviours and let them free to make decisions even in unexpected situations:
- A computer which pays attention is able to be surprised and interested in novel data.
- A computer which pays attention is able to understand novel situations and to choose the important data it will learn.
UNDERSTAND (2003 - 2006) - PhD Thesis Céline Thillou
With the drastic expansion of low-priced cameras, text recognition is nowadays a fast changing field; in particular, natural scene text understanding which aims at extracting text from daily images. From text extraction to correction of recognition errors, each sub-step is deeply studied to enhance versatility for handling most complex images. Either in color camera-based images or in low resolution thumbnails, inherent degradations, such as complex backgrounds, artistic fonts, uneven lighting or unsatisfactory resolution, must be taken into account. In order to circumvent or correct them, studies of image formation and degradation sources challengingly led to overcome too constrained definitions of color spaces. Hence our selective metric text extraction attempts to combine magnitude and directional processing of colors in an unsupervised framework. Text extraction from background is simultaneously linked to subsequent steps of character segmentation and recognition. This intermingled chain mainly aims at combining color, intensity and spatial information of pixels for robustness and accuracy. Each of these features addresses different issues; the first one for text extraction and the two latter ones for recovering initial separation between characters through log-Gabor filtering. In order to reach higher quality results, pre- and post-processing of natural scene text understanding are necessary and deal with Teager-based super-resolution, assuming a simple affine motion between frames with our SURETEXT proposition for the first one and with association of recognition outputs and linguistic information through lightweight finite state machines for the second one.
The CAPA project (2002 - 2004)
The CAPA (Automatic Classification of Agricultural Products) project implies 4 labs., from 3 Universities, which combine their respective skills in order to develop an automatic classification system of agricultural products, such as apples, according to the current quality norms applied in practice. The quality will be estimated from the possible marks, the color, or the shape of the products. The aim is to obtain a concrete prototype allowing to show the algorithmical and the mechanical possibilities of an automatic selection of fruits or vegetables.
APPLE (2002 - 2006) - PhD Thesis Devrim Unay
Quality inspection of apple fruits, traditionally performed by human experts, has to be automated by machine vision to reduce error, variation, fatigue and cost due to humans as well as to increase speed...
A typical apple inspection system should employ image processing and pattern recognition techniques to precisely segment defected skin by minimal confusion with stem/calyx areas and classify fruit into correct quality category. In this thesis, we present a work performed for quality inspection of bi-colored apples using multispectral images by tackling each of these sub-problems (namely, stem/calyx recognition, defect detection and fruit grading) individually. Stem and calyx are natural parts of apples that are confused with some defects in machine vision systems. A precise inspection system requires their discrimination, which is achieved by a highly accurate support vector machines-based approach. Defect detection of apples by machine vision is very problematic due to numerous defect types present as well as high natural variability of skin color. This task is accomplished by multi-layer perceptrons (an artificial neural network), which outperformed several other methods in accuracy and speed. Final grading of fruit is obtained by binary and multi-category classification with different classifiers, where results achieved are very encouraging.
EMBOLI (2002 - 2007) - PhD Thesis Raphael Sebbe
Pulmonary embolism (PE) is an extremely common and highly lethal
condition that is a leading cause of death in all age groups. Over the
past 10 years, computed tomography (CT) scanners have gained
acceptance as a minimally invasive method for diagnosing PE. In this
book, a framework for computer-aided diagnosis of PE in contrast-
enhanced CT images is presented. It consists of a combination of a
method for segmenting the pulmonary arteries (PA), emboli detection
methods as well as a scheme for evaluating their performances. The
segmentation of the PA serves one of the clot detection methods, and
is carried out through a region growing method that makes use of a
priori knowledge of vessel topology. Two different approaches for clot
detection are proposed: the first one performs clot detection by
analyzing the concavities in the segmentation of the pulmonary
arterial tree. It works in a semi-automatic way and it enables the
detection of thrombi in the larger sections of the PA. The second
method does not make use of PA segmentation and is thus fully
automatic, enabling detection of clots farther in the vessels. The
combination of these methods provides a robust detection technique
that can be used as a safeguard by radiologists, or even as
preliminary computer-aided diagnosis (CAD) tool. The evaluation of the
method is also discussed, and a scheme for measuring its performance
is proposed, including a practical approach to making reference
detection data, or ground truths, by radiologists.