Skip to content

satellogic/sssgan

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

15 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SSSGAN:Satellite Style and StructureGenerative Adversarial Networks

This repository contains an end-to-end pipeline for training and using SSSGAN:Satellite Style and StructureGenerative Adversarial Networks presented in the Master Thesis link(also included in folder TFM_document of this repository). The network was trained using INRIA building dataset and OSM. SSSGAN is a deep learning GAN netowrk specially designed for generating synthetic satellite imagery. The network leverages semantic imformation specifically procesed from OSM in semantic global vectors. This vecotors provides prior knowledge that helps the network to distangle the style latent space, helping to generate more region specific imagery. More information of the network can be find in link. This repository contains docker environments where all the code is executed. Morover, this reposotry contains all the training pipelines, networks, pre-porcessing and post-procesing piplines, besides the code that generates the global semantic vectors. This code is an extension of SPADE.

CRESI

The code is divided into two:

  • SSSGAN: explains how to train and make inferences with the network. It includes sample dataset (images , masks and semantic global vecotrs) and pre-trained weights to make inferences.

  • Semantic global vector geneartion: This section explains how to generate semantic global vecotrs


SSSGAN

Install

  1. Download this repository

  2. Download models and weights from link

  3. Locate the folder checkoiint in the root directory and place data in any place of your local machine

  4. Edit file in config_paths/paths.config:

     ROOT_CODE=/home-local/etylson/ws/sssgan_public/sssgan # Path of the root code folder
     ROOT_PREP=/home-local/etylson/ws/sssgan_public/sssgan/generate_ds # Path to the generate_ds subfolder 
     INRIA_ORIG=/data-local/data1-hdd/etylson/INRIA_original # Path to the original INRIA dataset 
     GT_PATH=/home-local/etylson/ws/sssgan_demo_data/train/gt # Path to the ground truth folder
     IMAGES_PATH=/home-local/etylson/ws/sssgan_demo_data/train/images # Path to the images  folder
     VAL_GT_PATH=/home-local/etylson/ws/sssgan_demo_data/val/gt
     VAL_IMAGES_PATH=/home-local/etylson/ws/sssgan_demo_data/val/images
     SEMANTIC_VECTOR=/home-local/etylson/ws/sssgan_demo_data/global_descriptor_vec # Path to the semantic global vector folder
    
  5. Build docker image

     sh docker/build_environment.sh 
    
  6. Run docker environment container (all commands should be run in this container)

     sh docker/start_environment.sh
    

Train

  • Training script description

    • SSGAN is trained with train_satellite.py script. Use --dataset_mode with the satellite option that uses ouer custo DataSet for satellite imagery. Specify semantic global vector location with --label_dir. Disbale any pre-process since the images are already preprocessed, --preprocess_mode none. Original spade gives as input the segmentation map, so it is needed to disable it with --no_initial_structure. As we are using only binary segmenataion maps use --label_nc 2 and --contain_dontcare_label (we are not using ignore index) --no_instance (we dont use instance map). Using no_vgg_loss is optional, if not specified perceptual loss is used. The option --satellite_generator_mode controls the architecture of the network. Options for this parameter are global_area_vector for feeding the network the complete global semantic vector (area one hot-encoding vector and semantic classes information), area_vector_only to feed only area one-fot encoding information and global_vector to feed only semantic vector (without area information). More training parameters please refer to option files in options. Finally --residual_mode enable residual connections of the netwrok.
  • Training information are stored ina folder with name spacified in the --name attribute under folder ./checkpoints

  • Train demo

      sh train_demo.sh
    
  • Train full ablation study

      sh train_ablation.sh
    

Test & Visualize

  • Test script description

    • SSGAN is trained with test_satellite.py script. Use --dataset_mode with the satellite option that uses ouer custo DataSet for satellite imagery. Specify semantic global vector location with --label_dir. Disbale any pre-process since the images are already preprocessed, --preprocess_mode none. Original spade gives as input the segmentation map, so it is needed to disable it with --no_initial_structure. As we are using only binary segmenataion maps use --label_nc 2 and --contain_dontcare_label (we are not using ignore index) --no_instance (we dont use instance map). Using no_vgg_loss is optional, if not specified perceptual loss is used. The option --satellite_generator_mode controls the architecture of the network. Options for this parameter are global_area_vector for feeding the network the complete global semantic vector (area one hot-encoding vector and semantic classes information), area_vector_only to feed only area one-fot encoding information and global_vector to feed only semantic vector (without area information). More training parameters please refer to option files in options.
  • Visualize comparison between networks

    1. Edit configuration file for comparison ./predict_and_visualize/vis_config/vis_config.json

       {
           "vis_name": "baseline", # Visualization title
           "model_name": "baseline", # name of the model
           "version": "latest", # version of the model weights
           "script": "python test.py --name {} --dataset_mode custom --label_dir {} --image_dir {} --label_nc 2  --contain_dontcare_label --no_instance --gpu_ids 2 --batchSize 1", #Scripts to be used, use {} for the script ot later fill that parameters
           "position": [1, 1] # Row q column 1 of the visualization
       },
       {
           "vis_name": "semantic", 
           "model_name": "semantic", 
           "version": "latest",
           "script": "python test_satellite.py --name {} --dataset_mode satellite --label_dir {} --image_dir {} --global_descriptor_dir /datasets/INRIA/global_descriptor_vec --label_nc 2  --contain_dontcare_label --no_instance --gpu_ids 2 --batchSize 1 --preprocess_mode none --no_initial_structure --satellite_generator_mode global_area_vector",
           "position": [1, 2]
       },
      
       {
           "vis_name": "semantic+dense", 
           "model_name": "semantic#dense", 
           "version": "latest",
           "script": "python test_satellite.py --name {} --dataset_mode satellite --label_dir {} --image_dir {} --global_descriptor_dir /datasets/INRIA/global_descriptor_vec --label_nc 2  --contain_dontcare_label --no_instance --gpu_ids 2 --batchSize 1 --preprocess_mode none --no_initial_structure --satellite_generator_mode global_area_vector --residual_mode",
           "position": [1, 3]
       }
      
    2. Run python ./predict_and_visualize/predict_and_visualize.py

    3. Results are stored in ./vis folder

  • Visuallize collage of semantic vector modifications

      sh semantic_manipulation_collage.sh
    

Semantic global vector generation

Install

  1. Download this repository

  2. Download original INRIA building dataset

  3. Edit file in config_paths/paths.config:

     ROOT_CODE=/home-local/etylson/ws/sssgan_public/sssgan # Path of the root code folder
     ROOT_PREP=/home-local/etylson/ws/sssgan_public/sssgan/generate_ds # Path to the generate_ds subfolder 
     INRIA_ORIG=/data-local/data1-hdd/etylson/INRIA_original # Path to the original INRIA dataset 
     GT_PATH=/home-local/etylson/ws/sssgan_demo_data/train/gt # Path to the ground truth folder
     IMAGES_PATH=/home-local/etylson/ws/sssgan_demo_data/train/images # Path to the images  folder
     VAL_GT_PATH=/home-local/etylson/ws/sssgan_demo_data/val/gt
     VAL_IMAGES_PATH=/home-local/etylson/ws/sssgan_demo_data/val/images
     SEMANTIC_VECTOR=/home-local/etylson/ws/sssgan_demo_data/global_descriptor_vec # Path to the semantic global vector folder
    
  4. Build image

     sh ./generate_ds/docker/build_environment.sh
    
  5. Run docke encironment

     sh ./generate_ds/docker/run_environment_gdal.sh
    

Prepare dataset

  1. Create patches of size 256 pixels and 128 stride

     python generate_patches.py --o /datasets/INRIA_ORIG \
                        --d /datasets/INRIA/dataset \
                        --w 256 \
                        --s 128 \
    
  2. Download renders from OSM using images geo-location

     python download_render.py --o /datasets/INRIA_ORIG
    
  3. Generate global semantic vectors vectors

     python generate_vectors.py --o ./osm_renders_0.3 \
                        --r /datasets/INRIA/dataset \
                        --d /datasets/INRIA/global_descriptor_vec \
                        --w 256 \
                        --s 128 
    
  4. Remove Kitsap images

     python remove_all_kitsp.py --ref /datasets/INRIA/dataset
    

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published