Latent space exploration
as an alternative way of looking at digital objects beyond parametric modeling
WHEN? Fall 2021
WHO? Tomás Cabezón
WHY? 62-706: Generative systems in design
Science
WHERE? CMU
This project is my final assignment for the course generative systems. The idea behind this project is to try to better understand how Machine Learning (ML) models infer the statistical distributions of the data during training. More concretely, study the feature extraction that the ML model does from the dataset. In this case, a parametric design algorithm is used to generate a dataset and then this parametric space is compared with the latent space learned by the model. The final output of the project is a 2D arrangement of the elements on the dataset based on their features, this is, a 2D map of the elements of the dataset where similar data is more closely positioned, something interesting for recommendation systems, for example.
Project description
Parametric modeling has gained general appreciation among creative practitioners as it enables the
synthesis of multiple design alternatives and solutions in the design space.
Although creative practitioners prefer to visually or tangibly experiment with alternatives and
variations of design representations, design exploration is challenging in the parametric design space
as this iterative process focuses on the variation of these individual parameters, rather than on the
relationship among them.
In this project, we propose a generative deep learning (DL) model, a variational AutoEncoder (VAE), to
capture the morphological features of the output 3D shapes of the parametric model and use the extracted
latent space—reduced-dimensionality vector space embeddings of data where similar points are closer to
each other— for the clustering and visualization of the outputs. Latent spaces have been shown to
capture interesting semantic properties and support data analysis and synthesis within a domain. We
present here a set of computational methods that center around projecting object features into low
dimensional latent representational spaces that are directly learned from data. This research aims to
implement a system that reduces the effort involved in undestanding the parametric space by providing
the user with a new way to look at data based on the exploration of the latent space.
Dataset Generation
To conduct this experiment a simple parametric model was designed, a parametric algorithm for pot generation. Similar to the handcraft of pottery wheel throwing, a simple bezier curve with three control points was turned around an axis to generate each 3D digital pots. This parametric algorithm was used to randomly generate the dataset that was later used to train the VAE, a total of 15.000 different 3D pots were generated.
Generated .stl files and the used parameters and the corresponding voxels.
The dataset is divided in two groups, for training and testing and posterior analysis.
Training
The following image shows the internal structure of the used VAE (Encoder and Decoder).
The model was trained for 240 epochs. For training, apart from the KLD loss, the MSE Loss was used as the reconstruction loss.
Training results
On the top, the ground truth (input of the VAE) and on the bottom the reconstructed element.
Once the VAE is trained, the encoder is used to embed each of the pots into the latent space and analyze the captured features by the DL model. For a better visualization of the extracted features, a t-Distributed Stochasitc Neighbor Embedding (t-SNE) is used, a popular dimensionality-reduction algorithm for visualizing high-dimensional data sets, creating a plot of the objects into a 2D space.
At this point it is possible to evaluate that the DL model has not only learned the parameters, but the relationship between these parameters and their influence on the output morphological shape.
Finally, the data points were later clustered using a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) finding core samples of high density and expanding clusters with them. This shows that the DL model was able to extract the features of each of the pots and understand the relationship. This latent space clustering is compared with the parametric space showing that the first better captures the final shapes of the objects.
Bellow the final website of this project can be seen (note that this won't show properly on small screens). The final website can also be accessed here