Part 3 – Analysing Data, Preparation, Dimension Reduction and Visual Exploration

In the last part a data frame was created.

This data has many dimensions and if we wanted to plot it in its current form it could prove difficult.

I have not previously worked with this kind of data so I had to look things up. Fortunately I have found a blog post about working with hyperspectral data here. It had a great deal of information and code for someone that begins to learn about HSI but most of the content did not explain how the used methods work or why use them. I did some additional research on those and I will include the list of interesting sources at the end of this post….

Continue reading →

Part 2 – Hyperspectral Cube – Getting familiar with data exploration.

Loading the data and check shape.

The Pavia University provided dataset includes the samples as well as the ground truth.

def load_data():
    '''
    X - input: 3D
    y - output: 2D
    '''
    X = loadmat('data/PaviaU.mat')['paviaU']
    y = loadmat('data/PaviaU_gt.mat')['paviaU_gt']
    print("X shape: ", X.shape)
    print("y shape: ", y.shape)
    return X, y

X, y = load_data()

To analyse HSI data I needed to remember that this image data has some differences from the regular cat, dog, human classification problem. HSI is high-dimensional and it took me a short while to wrap my head around this….

Continue reading →

Part 1 – Getting familiar with hyperspectral imaging data analysis – loading the data.

A friend of mine was recently hired as a graduate software developer in a company that specializes in hyperspectral imaging (HSI). This is when I first heard about this technology.

While I did not give much thought initially (I didn’t really know what it was), the more he spoke about it the more interested I became. From our conversations I was beginning to learn about its primary functions and how widely applicable the HSI is. It blew my mind.

Hyperspectral imaging is technique of collecting and processing large number of images of the same spatial area at different wavelengths. HSI obtains the spectrum for each pixel in the image of an area.

The collected data forms what is called a hyperspectral cube, where two dimensions represent the area while third dimension represents the spectral content.

I went on a quest of looking for more information about the subject in a more programming related context. I have found that the availability of sample data is somehow limited. There was also not many programming related training materials and the ones that I have found did not provide much context about the methods used.

Let’s get started.

Continue reading →