Create clusters using correlation matrix in Python

python matrix block correlation
Jasper C. picture Jasper C. · Oct 12, 2018 · Viewed 10.6k times · Source

all, I have a correlation matrix of 21 industry sectors. Now I want to split these 21 sectors into 4 or 5 groups, with sectors of similar behaviors grouped together.

Can experts shed me some lights on how to do this in Python please? Thanks much in advance!

Answer

Wes Doyle picture Wes Doyle · Oct 13, 2018

You might explore the use of Pandas DataFrame.corr and the scipy.cluster Hierarchical Clustering package

import pandas as pd
import scipy.cluster.hierarchy as spc


df = pd.DataFrame(my_data)
corr = df.corr().values

pdist = spc.distance.pdist(corr)
linkage = spc.linkage(pdist, method='complete')
idx = spc.fcluster(linkage, 0.5 * pdist.max(), 'distance')

Related questions

How to define a two-dimensional array?

I want to define a two-dimensional array without an initialized length like this: Matrix = [][] but it does not work... I've tried the code below, but it is wrong too: Matrix = [5][5] Error: Traceback ... IndexError: list index out of range What is …

Read More
How do I find the length (or dimensions, size) of a numpy matrix in python?

For a numpy matrix in python from numpy import matrix A = matrix([[1,2],[3,4]]) How can I find the length of a row (or column) of this matrix? Equivalently, how can I know the number of rows or columns? So far, the …

Read More
numpy matrix vector multiplication

When I multiply two numpy arrays of sizes (n x n)*(n x 1), I get a matrix of size (n x n). Following normal matrix multiplication rules, a (n x 1) vector is expected, but I simply cannot find any information …

Read More