Using Multi-layer Perceptron and Convolutional Neural Networks for Satellite image classification - 2022.
Antonio Fonseca
Packages to be installed:
conda install -c conda-forge umap-learn
pip install phate
conda install -c conda-forge imageio
[1]:
import numpy as np
import codecs
import copy
import json
import scipy.io
from scipy.spatial.distance import cdist, pdist, squareform
from scipy.linalg import eigh
import matplotlib.pyplot as plt
from sklearn.cluster import KMeans
import random
from sklearn import manifold
# import phate
# import umap
import pandas as pd
# import scprep
from torch.nn import functional as F
import pandas as pd
from sklearn.metrics import r2_score
from sklearn.preprocessing import MinMaxScaler
# import seaborn as sns
import torch
from torch.utils.data import Dataset, DataLoader
from torch.utils.data.sampler import SubsetRandomSampler,RandomSampler
from torchvision import datasets, transforms
from torch.nn.functional import softmax
from torch import optim, nn
import torchvision
import torchvision.transforms as transforms
import torchvision.datasets as datasets
import time
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
/Users/antonio/opt/anaconda3/envs/geo_comp/lib/python3.10/site-packages/tqdm/auto.py:22: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
from .autonotebook import tqdm as notebook_tqdm
cpu
Now that we have an autoencoder working on MNIST, let’s use this model to visualize some geodata. For the next section we will use the SAT-6 (https://csc.lsu.edu/~saikat/deepsat/)
SAT-6 consists of a total of 405,000 image patches each of size 28x28 and covering 6 landcover classes - barren land, trees, grassland, roads, buildings and water bodies. 324,000 images (comprising of four-fifths of the total dataset) were chosen as the training dataset and 81,000 (one fifths) were chosen as the testing dataset. Similar to SAT-4, the training and test sets were selected from disjoint NAIP tiles. Once generated, the images in the dataset were randomized in the same way as that for SAT-4. The specifications for the various landcover classes of SAT-4 and SAT-6 were adopted from those used in the National Land Cover Data (NLCD) algorithm.
The datasets are encoded as MATLAB .mat files that can be read using the standard load command in MATLAB. Each sample image is 28x28 pixels and consists of 4 bands - red, green, blue and near infrared. The training and test labels are 1x4 and 1x6 vectors for SAT-4 and SAT-6 respectively having a single 1 indexing a particular class from 0 through 4 or 6 and 0 values at all other indices.
The MAT file for the SAT-6 dataset contains the following variables:
train_x 28x28x4x324000 uint8 (containing 324000 training samples of 28x28 images each with 4 channels)
train_y 324000x6 uint8 (containing 6x1 vectors having labels for the 324000 training samples)
test_x 28x28x4x81000 uint8 (containing 81000 test samples of 28x28 images each with 4 channels)
test_y 81000x6 uint8 (containing 6x1 vectors having labels for the 81000 test samples)
Labels: - Building = 0 - Barren_land = 1 - Tree=2 - Grassland=3 - Road = 4 - Water = 5
[2]:
import numpy as np
import scipy.io
import matplotlib.pyplot as plt
import torch
from torch import optim, nn
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(device)
cpu
[3]:
# Using the satelite images dataset
###############################################################################
#load the data
data = scipy.io.loadmat("./SAT-4_and_SAT-6_datasets/sat-6-full.mat")
train_images = data['train_x']
train_labels = data['train_y']
test_images = data['test_x']
test_labels = data['test_y']
[4]:
####################################################################
#Checkout the data
print('Training data shape : ', train_images.shape, train_labels.shape)
print('Testing data shape : ', test_images.shape, test_labels.shape)
Training data shape : (28, 28, 4, 324000) (6, 324000)
Testing data shape : (28, 28, 4, 81000) (6, 81000)
[5]:
#Change the dimension to fit into the model
x_train = train_images.transpose(3,0,1,2)
t_train = train_labels.transpose()
# x_test = test_images.transpose(3,0,1,2)
# t_test = test_labels.transpose()
print('Training data shape : ', x_train.shape, t_train.shape)
# print('Testing data shape : ', x_test.shape, t_test.shape)
Training data shape : (324000, 28, 28, 4) (324000, 6)
[6]:
#Check what is in each channel
fig,ax = plt.subplots(4,4, figsize=(10,10))
ax = ax.ravel()
list_idx = np.linspace(0,100,num=16,dtype=np.int64)
for count, idx in enumerate(list_idx):
# print(idx)
print('count, t_train[count,:]: {}, {}'.format(count, t_train[count,:]))
# print(x_train[idx,:,:,0:3])
ax[count].imshow(x_train[count,:,:,0:3])
ax[count].set_title(str(np.argmax(t_train[count,:])))
count, t_train[count,:]: 0, [0 0 1 0 0 0]
count, t_train[count,:]: 1, [0 1 0 0 0 0]
count, t_train[count,:]: 2, [0 0 0 0 0 1]
count, t_train[count,:]: 3, [0 0 0 0 0 1]
count, t_train[count,:]: 4, [0 0 0 0 0 1]
count, t_train[count,:]: 5, [1 0 0 0 0 0]
count, t_train[count,:]: 6, [1 0 0 0 0 0]
count, t_train[count,:]: 7, [0 0 0 0 0 1]
count, t_train[count,:]: 8, [0 1 0 0 0 0]
count, t_train[count,:]: 9, [0 0 1 0 0 0]
count, t_train[count,:]: 10, [0 0 0 0 0 1]
count, t_train[count,:]: 11, [0 1 0 0 0 0]
count, t_train[count,:]: 12, [0 1 0 0 0 0]
count, t_train[count,:]: 13, [0 0 0 0 1 0]
count, t_train[count,:]: 14, [0 0 0 0 0 1]
count, t_train[count,:]: 15, [0 0 1 0 0 0]
[7]:
# split in training and testing
from torch.utils.data import Dataset, DataLoader
from torch.utils.data.sampler import SubsetRandomSampler
import torchvision.transforms as transforms
from scipy.ndimage import zoom
class MyDataset(Dataset):
def __init__(self, data, target):
print('data.dtype: {}'.format(data.dtype))
print('target.dtype: {}'.format(target.dtype))
self.data = torch.from_numpy(data).float()
self.target = torch.from_numpy(target).float()
def __getitem__(self, index):
x = self.data[index]
y = self.target[index]
return x, y
def __len__(self):
return len(self.data)
print('x_train.shape: {}'.format(x_train.shape))
n_samples = 50000
dataset = MyDataset(x_train[:n_samples,:,:,:], np.argmax(t_train[:n_samples],axis=1))
del x_train, t_train
dataset_size = len(dataset)
print('dataset_size: {}'.format(dataset_size))
test_split=0.2
batch_size=1024
# -- split dataset
indices = list(range(dataset_size))
split = int(np.floor(test_split*dataset_size))
print('split: {}'.format(split))
# np.random.shuffle(indices) # Randomizing the indices is not a good idea if you want to model the sequence
train_indices, val_indices = indices[split:], indices[:split]
# -- create dataloaders
# #Original
train_sampler = SubsetRandomSampler(train_indices)
valid_sampler = SubsetRandomSampler(val_indices)
dataloaders = {
'train': torch.utils.data.DataLoader(dataset, batch_size=batch_size, sampler=train_sampler),
'test': torch.utils.data.DataLoader(dataset, batch_size=batch_size, sampler=valid_sampler),
'all': torch.utils.data.DataLoader(dataset, batch_size=5000, shuffle=False),
}
x_train.shape: (324000, 28, 28, 4)
data.dtype: uint8
target.dtype: int64
dataset_size: 50000
split: 10000
[8]:
class FFnet(nn.Module):
'''
Linear activation in the middle (instead of an activation function)
'''
def __init__(self):
super(FFnet, self).__init__()
self.enc_lin1 = nn.Linear(3136, 1000)
self.enc_lin2 = nn.Linear(1000, 500)
self.enc_lin3 = nn.Linear(500, 250)
self.enc_lin4 = nn.Linear(250, 6)
self.relu = nn.ReLU()
self.tanh = nn.Tanh()
def encode(self, x):
x = self.enc_lin1(x)
x = self.relu(x)
x = self.enc_lin2(x)
x = self.relu(x)
x = self.enc_lin3(x)
x = self.relu(x)
output = self.enc_lin4(x)
return output
def forward(self, x):
z = self.encode(x)
return z
[9]:
## Second routine for training and evaluation (using the )
# Training and Evaluation routines
import time
def train(model,loss_fn, optimizer, train_loader, test_loader, num_epochs=None, verbose=False):
"""
This is a standard training loop, which leaves some parts to be filled in.
INPUT:
:param model: an untrained pytorch model
:param loss_fn: e.g. Cross Entropy loss or Mean Squared Error.
:param optimizer: the model optimizer, initialized with a learning rate.
:param training_set: The training data, in a dataloader for easy iteration.
:param test_loader: The testing data, in a dataloader for easy iteration.
"""
print('optimizer: {}'.format(optimizer))
if num_epochs is None:
num_epochs = 100
print('n. of epochs: {}'.format(num_epochs))
for epoch in range(num_epochs+1):
start = time.time()
# loop through each data point in the training set
for data, targets in train_loader:
# run the model on the data
model_input = data.view(data.size(0),-1).to(device)# TODO: Turn the 28 by 28 image tensors into a 784 dimensional tensor.
if verbose: print('model_input.shape: {}'.format(model_input.shape))
# Clear gradients w.r.t. parameters
optimizer.zero_grad()
out = model(model_input) # The second output is the latent representation
if verbose:
print('targets.shape: {}'.format(targets.shape))
print('targets: {}'.format(targets))
print('out.shape: {}'.format(out.shape))
print('out: {}'.format(out))
# Calculate the loss
targets = targets.type(torch.LongTensor) # add an extra dimension to keep CrossEntropy happy.
if verbose: print('targets.shape: {}'.format(targets.shape))
loss = loss_fn(out,targets)
if verbose: print('loss: {}'.format(loss))
# Find the gradients of our loss via backpropogation
loss.backward()
# Adjust accordingly with the optimizer
optimizer.step()
# Give status reports every 100 epochs
if epoch % 10==0:
print(f" EPOCH {epoch}. Progress: {epoch/num_epochs*100}%. ")
loss_train, acc_train = evaluate(model,train_loader,verbose)
loss_test, acc_test = evaluate(model,test_loader,verbose)
print(" Train loss: {:.4f}. Train Acc: {:.4f}, Test loss: {:.4f}. Test Acc: {:.4f}. Time/epoch: {:.4f}".format(loss_train, acc_train, loss_test, acc_test, (time.time() - start))) #TODO: implement the evaluate function to provide performance statistics during training.
def evaluate(model, evaluation_set, verbose=False):
"""
Evaluates the given model on the given dataset.
Returns the percentage of correct classifications out of total classifications.
"""
with torch.no_grad(): # this disables backpropogation, which makes the model run much more quickly.
correct = 0
total = 0
loss_all=0
for data, targets in evaluation_set:
# run the model on the data
model_input = data.view(data.size(0),-1).to(device)# TODO: Turn the 28 by 28 image tensors into a 784 dimensional tensor.
if verbose:
print('model_input.shape: {}'.format(model_input.shape))
print('targets.shape: {}'.format(targets.shape))
out = model(model_input)
targets = targets.type(torch.LongTensor)
loss = loss_fn(out,targets)
if verbose: print('out[:5]: {}'.format(out[:5]))
loss_all+=loss.item()
# the class with the highest energy is what we choose as prediction
_, predicted = torch.max(out.data, 1)
total += targets.size(0)
correct += (predicted == targets).sum().item()
acc = 100 * correct / total
loss = loss_all/total
return loss, acc
[10]:
lr_range = [0.01,0.005,0.001]
for lr in lr_range:
if 'model' in globals():
print('Deleting previous model')
del model, loss_fn, optimizer
model = FFnet().to(device)
optimizer = torch.optim.Adam(model.parameters(), lr = lr)
loss_fn = nn.CrossEntropyLoss().to(device)
train(model,loss_fn, optimizer, dataloaders['train'], dataloaders['test'],verbose=False)
Autoencoder - with linear activation in middle layer and non-linearity (tanh) everywhere else
optimizer: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.01
maximize: False
weight_decay: 0
)
n. of epochs: 100
EPOCH 0. Progress: 0.0%.
Train loss: 0.0009. Train Acc: 74.3500, Test loss: 0.0010. Test Acc: 74.3200. Time/epoch: 24.0663
EPOCH 10. Progress: 10.0%.
Train loss: 0.0002. Train Acc: 90.5100, Test loss: 0.0002. Test Acc: 90.1700. Time/epoch: 21.5834
EPOCH 20. Progress: 20.0%.
Train loss: 0.0007. Train Acc: 71.4850, Test loss: 0.0007. Test Acc: 72.3100. Time/epoch: 31.5746
EPOCH 30. Progress: 30.0%.
Train loss: 0.0002. Train Acc: 92.1275, Test loss: 0.0002. Test Acc: 91.4800. Time/epoch: 31.7279
EPOCH 40. Progress: 40.0%.
Train loss: 0.0002. Train Acc: 92.4675, Test loss: 0.0002. Test Acc: 92.1500. Time/epoch: 34.2578
EPOCH 50. Progress: 50.0%.
Train loss: 0.0002. Train Acc: 92.1400, Test loss: 0.0002. Test Acc: 91.5600. Time/epoch: 27.1172
EPOCH 60. Progress: 60.0%.
Train loss: 0.0002. Train Acc: 93.0550, Test loss: 0.0002. Test Acc: 92.5800. Time/epoch: 27.6910
EPOCH 70. Progress: 70.0%.
Train loss: 0.0002. Train Acc: 90.7450, Test loss: 0.0002. Test Acc: 90.1900. Time/epoch: 19.6197
EPOCH 80. Progress: 80.0%.
Train loss: 0.0002. Train Acc: 94.0375, Test loss: 0.0002. Test Acc: 93.3700. Time/epoch: 15.0691
EPOCH 90. Progress: 90.0%.
Train loss: 0.0002. Train Acc: 89.7775, Test loss: 0.0002. Test Acc: 89.7000. Time/epoch: 14.5441
EPOCH 100. Progress: 100.0%.
Train loss: 0.0002. Train Acc: 93.4900, Test loss: 0.0002. Test Acc: 92.8800. Time/epoch: 14.6357
Deleting previous model
optimizer: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.005
maximize: False
weight_decay: 0
)
n. of epochs: 100
EPOCH 0. Progress: 0.0%.
Train loss: 0.0006. Train Acc: 69.3350, Test loss: 0.0006. Test Acc: 69.8200. Time/epoch: 14.9185
EPOCH 10. Progress: 10.0%.
Train loss: 0.0004. Train Acc: 82.0250, Test loss: 0.0004. Test Acc: 81.9600. Time/epoch: 17.5713
EPOCH 20. Progress: 20.0%.
Train loss: 0.0002. Train Acc: 90.4775, Test loss: 0.0003. Test Acc: 90.0500. Time/epoch: 15.3081
EPOCH 30. Progress: 30.0%.
Train loss: 0.0003. Train Acc: 86.3750, Test loss: 0.0003. Test Acc: 86.3700. Time/epoch: 14.9161
EPOCH 40. Progress: 40.0%.
Train loss: 0.0004. Train Acc: 83.8150, Test loss: 0.0004. Test Acc: 84.0500. Time/epoch: 15.6279
EPOCH 50. Progress: 50.0%.
Train loss: 0.0002. Train Acc: 92.5800, Test loss: 0.0002. Test Acc: 92.1200. Time/epoch: 14.3379
EPOCH 60. Progress: 60.0%.
Train loss: 0.0002. Train Acc: 92.0525, Test loss: 0.0002. Test Acc: 91.6400. Time/epoch: 14.5943
EPOCH 70. Progress: 70.0%.
Train loss: 0.0002. Train Acc: 91.4350, Test loss: 0.0002. Test Acc: 90.9600. Time/epoch: 14.1033
EPOCH 80. Progress: 80.0%.
Train loss: 0.0004. Train Acc: 86.1225, Test loss: 0.0004. Test Acc: 85.3700. Time/epoch: 14.5240
EPOCH 90. Progress: 90.0%.
Train loss: 0.0002. Train Acc: 93.7375, Test loss: 0.0002. Test Acc: 93.1100. Time/epoch: 14.4524
EPOCH 100. Progress: 100.0%.
Train loss: 0.0001. Train Acc: 94.7475, Test loss: 0.0002. Test Acc: 94.0500. Time/epoch: 14.3222
Deleting previous model
optimizer: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.001
maximize: False
weight_decay: 0
)
n. of epochs: 100
EPOCH 0. Progress: 0.0%.
Train loss: 0.0006. Train Acc: 70.6150, Test loss: 0.0006. Test Acc: 71.1900. Time/epoch: 15.7802
EPOCH 10. Progress: 10.0%.
Train loss: 0.0002. Train Acc: 90.8450, Test loss: 0.0002. Test Acc: 90.7700. Time/epoch: 13.9836
EPOCH 20. Progress: 20.0%.
Train loss: 0.0002. Train Acc: 92.1425, Test loss: 0.0002. Test Acc: 91.5800. Time/epoch: 14.6497
EPOCH 30. Progress: 30.0%.
Train loss: 0.0002. Train Acc: 93.2475, Test loss: 0.0002. Test Acc: 92.5600. Time/epoch: 14.4071
EPOCH 40. Progress: 40.0%.
Train loss: 0.0003. Train Acc: 87.3000, Test loss: 0.0003. Test Acc: 87.2900. Time/epoch: 14.0998
EPOCH 50. Progress: 50.0%.
Train loss: 0.0001. Train Acc: 94.7875, Test loss: 0.0002. Test Acc: 94.2900. Time/epoch: 14.3237
EPOCH 60. Progress: 60.0%.
Train loss: 0.0001. Train Acc: 93.9550, Test loss: 0.0002. Test Acc: 93.4400. Time/epoch: 14.2208
EPOCH 70. Progress: 70.0%.
Train loss: 0.0001. Train Acc: 94.7900, Test loss: 0.0001. Test Acc: 94.1900. Time/epoch: 14.2017
EPOCH 80. Progress: 80.0%.
Train loss: 0.0003. Train Acc: 90.2175, Test loss: 0.0003. Test Acc: 90.1400. Time/epoch: 14.0847
EPOCH 90. Progress: 90.0%.
Train loss: 0.0001. Train Acc: 94.3650, Test loss: 0.0002. Test Acc: 93.3800. Time/epoch: 14.1263
EPOCH 100. Progress: 100.0%.
Train loss: 0.0003. Train Acc: 89.7800, Test loss: 0.0003. Test Acc: 89.0400. Time/epoch: 14.9697
[35]:
class CNNet(nn.Module):
def __init__(self):
super().__init__()
self.conv1 = nn.Conv2d(4, 6, 5)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16 * 4 * 4, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 6)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = torch.flatten(x, 1) # flatten all dimensions except batch
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
model = CNNet()
[38]:
# Training and Evaluation routines
import time
def train(model,loss_fn, optimizer, train_loader, test_loader, num_epochs=None, verbose=False):
"""
This is a standard training loop, which leaves some parts to be filled in.
INPUT:
:param model: an untrained pytorch model
:param loss_fn: e.g. Cross Entropy loss of Mean Squared Error.
:param optimizer: the model optimizer, initialized with a learning rate.
:param training_set: The training data, in a dataloader for easy iteration.
:param test_loader: The testing data, in a dataloader for easy iteration.
"""
print('optimizer: {}'.format(optimizer))
if num_epochs is None:
num_epochs = 100
print('n. of epochs: {}'.format(num_epochs))
for epoch in range(num_epochs+1):
start = time.time()
# loop through each data point in the training set
for data, targets in train_loader:
# run the model on the data
model_input = data.permute(0, 3, 2, 1).to(device)
if verbose: print('model_input.shape: {}'.format(model_input.shape))
# Clear gradients w.r.t. parameters
optimizer.zero_grad()
out = model(model_input) # The second output is the latent representation
if verbose:
print('targets.shape: {}'.format(targets.shape))
print('targets: {}'.format(targets))
print('out.shape: {}'.format(out.shape))
print('out: {}'.format(out))
# Calculate the loss
targets = targets.type(torch.LongTensor) # add an extra dimension to keep CrossEntropy happy.
if verbose: print('targets.shape: {}'.format(targets.shape))
loss = loss_fn(out,targets)
if verbose: print('loss: {}'.format(loss))
# Find the gradients of our loss via backpropogation
loss.backward()
# Adjust accordingly with the optimizer
optimizer.step()
# Give status reports every 100 epochs
if epoch % 10==0:
print(f" EPOCH {epoch}. Progress: {epoch/num_epochs*100}%. ")
loss_train, acc_train = evaluate(model,train_loader,verbose)
loss_test, acc_test = evaluate(model,test_loader,verbose)
print(" Train loss: {:.4f}. Train Acc: {:.4f}, Test loss: {:.4f}. Test Acc: {:.4f}. Time/epoch: {:.4f}".format(loss_train, acc_train, loss_test, acc_test, (time.time() - start))) #TODO: implement the evaluate function to provide performance statistics during training.
def evaluate(model, evaluation_set, verbose=False):
"""
Evaluates the given model on the given dataset.
Returns the percentage of correct classifications out of total classifications.
"""
with torch.no_grad(): # this disables backpropogation, which makes the model run much more quickly.
correct = 0
total = 0
loss_all=0
for data, targets in evaluation_set:
# run the model on the data
model_input = data.permute(0, 3, 2, 1).to(device)
if verbose:
print('model_input.shape: {}'.format(model_input.shape))
print('targets.shape: {}'.format(targets.shape))
out = model(model_input)
targets = targets.type(torch.LongTensor)
loss = loss_fn(out,targets)
if verbose: print('out[:5]: {}'.format(out[:5]))
loss_all+=loss.item()
# the class with the highest energy is what we choose as prediction
_, predicted = torch.max(out.data, 1)
total += targets.size(0)
correct += (predicted == targets).sum().item()
acc = 100 * correct / total
loss = loss_all/total
return loss, acc
[39]:
lr_range = [0.01,0.005,0.001]
for lr in lr_range:
if 'model' in globals():
print('Deleting previous model')
del model, loss_fn, optimizer
model = CNNet().to(device)
optimizer = torch.optim.Adam(model.parameters(), lr = lr) # This is absurdly high.
loss_fn = nn.CrossEntropyLoss().to(device)
train(model,loss_fn, optimizer, dataloaders['train'], dataloaders['test'],verbose=False)
Autoencoder - with linear activation in middle layer and non-linearity (tanh) everywhere else
Deleting previous model
optimizer: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.01
maximize: False
weight_decay: 0
)
n. of epochs: 100
EPOCH 0. Progress: 0.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.6766
EPOCH 10. Progress: 10.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.8254
EPOCH 20. Progress: 20.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.5843
EPOCH 30. Progress: 30.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.4217
EPOCH 40. Progress: 40.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.4936
EPOCH 50. Progress: 50.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.3445
EPOCH 60. Progress: 60.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.3718
EPOCH 70. Progress: 70.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.3719
EPOCH 80. Progress: 80.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.6060
EPOCH 90. Progress: 90.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.8346
EPOCH 100. Progress: 100.0%.
Train loss: 0.0015. Train Acc: 36.8250, Test loss: 0.0015. Test Acc: 37.5400. Time/epoch: 9.3048
Deleting previous model
optimizer: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.005
maximize: False
weight_decay: 0
)
n. of epochs: 100
EPOCH 0. Progress: 0.0%.
Train loss: 0.0006. Train Acc: 77.1475, Test loss: 0.0006. Test Acc: 77.3100. Time/epoch: 9.1976
EPOCH 10. Progress: 10.0%.
Train loss: 0.0003. Train Acc: 90.6200, Test loss: 0.0003. Test Acc: 90.0200. Time/epoch: 9.4351
EPOCH 20. Progress: 20.0%.
Train loss: 0.0002. Train Acc: 93.8775, Test loss: 0.0002. Test Acc: 92.8900. Time/epoch: 9.1924
EPOCH 30. Progress: 30.0%.
Train loss: 0.0002. Train Acc: 93.8075, Test loss: 0.0002. Test Acc: 92.9200. Time/epoch: 9.2729
EPOCH 40. Progress: 40.0%.
Train loss: 0.0001. Train Acc: 95.7125, Test loss: 0.0002. Test Acc: 94.4300. Time/epoch: 9.4951
EPOCH 50. Progress: 50.0%.
Train loss: 0.0001. Train Acc: 96.7625, Test loss: 0.0002. Test Acc: 94.3100. Time/epoch: 9.5598
EPOCH 60. Progress: 60.0%.
Train loss: 0.0002. Train Acc: 92.7925, Test loss: 0.0002. Test Acc: 91.6700. Time/epoch: 9.7035
EPOCH 70. Progress: 70.0%.
Train loss: 0.0001. Train Acc: 95.0525, Test loss: 0.0002. Test Acc: 92.7000. Time/epoch: 9.5682
EPOCH 80. Progress: 80.0%.
Train loss: 0.0003. Train Acc: 90.9500, Test loss: 0.0004. Test Acc: 89.5500. Time/epoch: 9.7392
EPOCH 90. Progress: 90.0%.
Train loss: 0.0001. Train Acc: 96.1700, Test loss: 0.0002. Test Acc: 93.1400. Time/epoch: 9.6254
EPOCH 100. Progress: 100.0%.
Train loss: 0.0000. Train Acc: 98.5050, Test loss: 0.0002. Test Acc: 94.2700. Time/epoch: 9.7619
Deleting previous model
optimizer: Adam (
Parameter Group 0
amsgrad: False
betas: (0.9, 0.999)
eps: 1e-08
lr: 0.001
maximize: False
weight_decay: 0
)
n. of epochs: 100
EPOCH 0. Progress: 0.0%.
Train loss: 0.0003. Train Acc: 88.8525, Test loss: 0.0003. Test Acc: 88.6600. Time/epoch: 9.9031
EPOCH 10. Progress: 10.0%.
Train loss: 0.0001. Train Acc: 95.2300, Test loss: 0.0001. Test Acc: 95.0900. Time/epoch: 9.9000
EPOCH 20. Progress: 20.0%.
Train loss: 0.0001. Train Acc: 94.5075, Test loss: 0.0001. Test Acc: 94.4700. Time/epoch: 10.7499
EPOCH 30. Progress: 30.0%.
Train loss: 0.0001. Train Acc: 95.5375, Test loss: 0.0001. Test Acc: 95.2300. Time/epoch: 9.8978
EPOCH 40. Progress: 40.0%.
Train loss: 0.0001. Train Acc: 94.6200, Test loss: 0.0001. Test Acc: 94.1000. Time/epoch: 10.0642
EPOCH 50. Progress: 50.0%.
Train loss: 0.0001. Train Acc: 94.4925, Test loss: 0.0001. Test Acc: 94.3600. Time/epoch: 10.3240
EPOCH 60. Progress: 60.0%.
Train loss: 0.0001. Train Acc: 96.1050, Test loss: 0.0001. Test Acc: 95.7600. Time/epoch: 10.2794
EPOCH 70. Progress: 70.0%.
Train loss: 0.0001. Train Acc: 96.4900, Test loss: 0.0001. Test Acc: 96.0200. Time/epoch: 10.4305
EPOCH 80. Progress: 80.0%.
Train loss: 0.0001. Train Acc: 96.5425, Test loss: 0.0001. Test Acc: 96.0300. Time/epoch: 9.8782
EPOCH 90. Progress: 90.0%.
Train loss: 0.0001. Train Acc: 97.4400, Test loss: 0.0001. Test Acc: 96.9900. Time/epoch: 9.8171
EPOCH 100. Progress: 100.0%.
Train loss: 0.0001. Train Acc: 97.9350, Test loss: 0.0001. Test Acc: 97.2100. Time/epoch: 11.4006
[106]:
#Change the dimension to fit into the model
x_train = train_images.transpose(3,0,1,2)
t_train = train_labels.transpose()
model.eval()
# with torch.no_grad():
# Retrieve output from the image
idx_to_plot = 0
image = x_train[idx_to_plot,:,:,:]
image_orig = image.copy()
print(image.shape)
image = torch.FloatTensor(image).permute(2, 1, 0).to(device)
# Make input tensor require gradient
# X.requires_grad_()
image = image[None,:].requires_grad_()
print(image.shape)
output = model(image)
# Catch the output
output_idx = output.argmax()
output_max = output[0, output_idx]
# Do backpropagation to get the derivative of the output based on the image
output_max.backward()
# Retireve the saliency map and also pick the maximum value from channels on each pixel.
# In this case, we look at dim=1. Recall the shape (batch_size, channel, width, height)
saliency, _ = torch.max(image.grad.data.abs(), dim=1)
saliency = saliency.reshape(28, 28)
# # Reshape the image
# image = image.reshape(-1, 28, 28)
# Visualize the image and the saliency map
fig, ax = plt.subplots(1, 2)
# x_train[count,:,:,0:3]
# print(image.shape)
# image = image.permute(3, 2, 1,0).squeeze()
# print(image.shape)
ax[0].imshow(image_orig[:,:,0:3])
ax[0].axis('off')
ax[1].imshow(saliency.cpu(), cmap='hot')
ax[1].axis('off')
ax[0].set_title('Label: {}'.format(t_train[idx_to_plot].argmax()))
plt.tight_layout()
fig.suptitle('The Image and Its Saliency Map')
plt.show()
(28, 28, 4)
torch.Size([1, 4, 28, 28])
