Latent space analysis and alignment for cross-Language code translation

Abstract

The motivation for this work is to better understand the latent representations learned by neural networks and how these representations align with human-perceivable concepts. Neural networks often operate as black-box systems, making it challenging to interpret the meaning of their internal activations. This study investigates the organization of these latent representations in the encoder and decoder modules of a language model. Using a code translation task between Java and C#, layer activations are extracted and grouped using K-means clustering. Metrics are applied to evaluate the semantic alignment and bidirectional consistency of the clusters, as well as their structural similarities between source and target language representations. This approach aims to provide insights into the organization of neural representations, offering a basis for further analysis of their alignment with meaningful, interpretable patterns.