Deploying and processing neural representations of signals

Neural representations (NR) have emerged in the last few years as a powerful tool to represent signals from several domains, such as images, 3D shapes, or audio. Indeed, deep neural networks have been shown capable of approximating continuous functions that describe a given signal with theoretical infinite resolution. This finding allows obtaining representations whose memory footprint is fixed and decoupled from the resolution at which the underlying signal can be sampled, something that is not possible with traditional discrete representations, e.g., grids of pixels for images or voxels for 3D shapes. During the last two years, many techniques have been proposed to improve the capability of NR to approximate high-frequency details and to make the optimization procedures required to obtain NR less demanding both in terms of time and data requirements, motivating many researchers to deploy NR as the main form of data representation for complex pipelines. Following this line of research, we first show that NR can approximate precisely Unsigned Distance Functions, providing an effective way to represent garments that feature open 3D surfaces and unknown topology. Then, we present a pipeline to obtain in a few minutes a compact Neural Twin® for a given object, by exploiting the recent advances in modeling neural radiance fields. Furthermore, we move a step in the direction of adopting NR as a standalone representation, by considering the possibility of performing downstream tasks by processing directly the NR weights. We first show that deep neural networks can be compressed into compact latent codes. Then, we show how this technique can be exploited to perform deep learning on implicit neural representations (INR) of 3D shapes, by only looking at the weights of the networks.

Plasticity in body and peripersonal space representations

A successful interaction with objects in the environment requires integrating information concerning object-location with the shape, dimension and position of body parts in space. The former information is coded in a multisensory representation of the space around the body, i.e. peripersonal space (PPS), whereas the latter is enabled by an online, constantly updated, action-orientated multisensory representation of the body (BR) that is critical for action. One of the critical features of these representations is that both PPS and BR are not fixed, but they dynamically change depending on different types of experience. In a series of experiment, I studied plastic properties of PPS and BR in humans. I have developed a series of methods to measure the boundaries of PPS representation (Chapter 4), to study its neural correlates (Chapter 3) and to assess BRs. These tasks have been used to study changes in PPS and BR following tool-use (Chapter 5), multisensory stimulation (Chapter 6), amputation and prosthesis implantation (Chapter 7) or social interaction (Chapter 8). I found that changes in the function (tool-use) and the structure (amputation and prosthesis implantation) of the physical body elongate or shrink both PPS and BR. Social context and social interaction also shape PPS representation. Such high degree of plasticity suggests that our sense of body in space is not given at once, but it is constantly constructed and adapted through experience.