“The world cannot stabilize what it does not watch.” Concerned about the drastic and inevitable effects of climate change, my team and I centered our project at a hackathon around the same, with an intent of making the common masses aware of the ongoing climatic situation. To achieve this, I thought of visualizing the consequences, by translating the stats and figures into an intuitive and interactive visualization that everyone can relate to on a personal level and gain true insights.
The base idea of the project was aimed at generating images that depict accurate, vivid and personalized outcomes of climate change using Cycle-Consistent Adversarial Networks (CycleGANs). The idea was to train the CycleGAN model on street-view images of houses before and after extreme weather events followed by learning a mapping that can be applied to images of locations that have not yet experienced these events. Researching on related work provided information about model architectures and training methods but the most challenging aspect of generating these images using GANs was finding the training data needed in order to extract the mapping function.
CycleGAN training assumes that there is some underlying relationship between the domain (for instance a change of seasons in a landscape) and hence would perform well only when there are as few extraneous objects (such as vehicles or people) as possible. Hence, finding the correct data would involve manual searching and tweaking, which under the given time constraints wasn’t an option. Therefore, to stick to the original idea of providing an intuitive visualization while keeping the above idea as a roadmap, efforts were diverted to a new idea involving global average temperatures and greenhouse gases data, which was plotted on an interactive 3D globe in the form of spikes, depicting temperature anomalies using varying lengths and color gradients.
This was followed by running regression-based machine learning algorithms, that helped in predicting the future rates of emissions and corresponding rises in temperature as well as providing the end user a personalized message involving actions that can be taken to improve the situation. Coming back to the CycleGAN approach, we restricted our idea (for testing purposes) to collecting a small amount of data of houses before and after flooding and training our model on the same.
However, another challenge that we encountered was the fact that flooding is not truly a one-to-one mapping(such as the one assumed by the CycleGAN approach, but in fact a many-to-one mapping, i.e. roads, grass, fences are all mapped to water. For this reason, our data collection was constrained to houses surrounded by lawns, which were then mapped to water by the model. The model was able to learn an adequate mapping between grass and water, and this mapping could be applied to generate fairly realistic images of flooded houses.
Abhishek's blog 