Alternating least squares for Tucker model
The function tucker_als computes the best rank(R1,R2,..,Rn) approximation of tensor X, according to the specified dimensions in vector R. The input X can be a tensor, sptensor, ktensor, or ttensor. The result returned in T is a ttensor.
The method is originally from Tucker (1966) and later revisited in De Lathauwer et al. (2000).
- L. R. Tucker, Some mathematical notes on three-mode factor analysis, Psychometrika, 31:279-311, 1966, http://dx.doi.org/10.1007/BF02289464
- L. De Lathauwer, B. De Moor, J. Vandewalle, On the best rank-1 and rank-(R_1, R_2, R_N) approximation of higher-order tensors, SIAM J. Matrix Analysis and Applications, 21:1324-1342, 2000, http://doi.org/10.1137/S0895479898346995
Note: Oftentimes it's better to use hosvd instead.
Contents
Create a data tensor of size [5 4 3]
rng(0); %<-- Set seed for reproducibility
X = sptenrand([5 4 3], 10)
X is a sparse tensor of size 5 x 4 x 3 with 10 nonzeros (1,2,3) 0.0759 (1,3,2) 0.0540 (2,2,2) 0.5308 (2,2,3) 0.7792 (3,1,3) 0.9340 (3,4,2) 0.1299 (4,1,2) 0.5688 (4,4,2) 0.4694 (5,2,1) 0.0119 (5,4,3) 0.3371
Create a [2 2 2] approximation
T = tucker_als(X,2) %<-- best rank(2,2,2) approximation
Tucker Alternating Least-Squares:
Iter 1: fit = 3.266855e-01 fitdelta = 3.3e-01
Iter 2: fit = 4.285677e-01 fitdelta = 1.0e-01
Iter 3: fit = 4.707375e-01 fitdelta = 4.2e-02
Iter 4: fit = 4.728036e-01 fitdelta = 2.1e-03
Iter 5: fit = 4.728492e-01 fitdelta = 4.6e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
0.9045 0.0007
-0.0007 0.8920
T.core(:,:,2) =
0.2732 0.0006
0.0006 -0.2771
T.U{1} =
0.0666 0.0001
0.9978 0.0008
-0.0008 1.0000
-0.0001 0.0007
-0.0001 0.0018
T.U{2} =
-0.0015 1.0000
1.0000 0.0015
0.0021 0.0000
-0.0001 0.0007
T.U{3} =
-0.0000 -0.0000
0.2971 0.9548
0.9548 -0.2971
Create a [2 2 1] approximation
T = tucker_als(X,[2 2 1]) %<-- best rank(2,2,1) approximation
Tucker Alternating Least-Squares:
Iter 1: fit = 2.941961e-01 fitdelta = 2.9e-01
Iter 2: fit = 3.996958e-01 fitdelta = 1.1e-01
Iter 3: fit = 4.229258e-01 fitdelta = 2.3e-02
Iter 4: fit = 4.304139e-01 fitdelta = 7.5e-03
Iter 5: fit = 4.325472e-01 fitdelta = 2.1e-03
Iter 6: fit = 4.330654e-01 fitdelta = 5.2e-04
Iter 7: fit = 4.331799e-01 fitdelta = 1.1e-04
Iter 8: fit = 4.332039e-01 fitdelta = 2.4e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 1
T.core(:,:,1) =
0.9293 0.0000
-0.0000 0.8920
T.U{1} =
0.0747 -0.0000
0.9972 -0.0000
0.0000 0.9575
-0.0000 0.2844
-0.0000 0.0475
T.U{2} =
0.0000 0.9911
1.0000 -0.0000
0.0017 -0.0000
-0.0000 0.1333
T.U{3} =
-0.0000
0.4011
0.9160
Use a different ordering of the dimensions
T = tucker_als(X,2,struct('dimorder',[3 2 1]))
Tucker Alternating Least-Squares:
Iter 1: fit = 3.515073e-01 fitdelta = 3.5e-01
Iter 2: fit = 3.548900e-01 fitdelta = 3.4e-03
Iter 3: fit = 3.560941e-01 fitdelta = 1.2e-03
Iter 4: fit = 3.565373e-01 fitdelta = 4.4e-04
Iter 5: fit = 3.566994e-01 fitdelta = 1.6e-04
Iter 6: fit = 3.567586e-01 fitdelta = 5.9e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
0.8911 -0.2312
-0.1522 0.0400
T.core(:,:,2) =
0.1445 0.1767
0.6535 0.2855
T.U{1} =
0.0011 -0.0003
0.0124 0.0067
0.9855 -0.1691
0.1690 0.9856
-0.0000 0.0000
T.U{2} =
0.9678 -0.2514
0.0146 0.0015
-0.0000 0.0001
0.2513 0.9679
T.U{3} =
-0.0000 0.0000
0.0008 1.0000
1.0000 -0.0008
Use the n-vecs initialization method
This initialization is more expensive but generally works very well.
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init','eigs'))
Computing 2 leading e-vectors for factor 2.
Computing 2 leading e-vectors for factor 1.
Tucker Alternating Least-Squares:
Iter 1: fit = 4.726805e-01 fitdelta = 4.7e-01
Iter 2: fit = 4.728466e-01 fitdelta = 1.7e-04
Iter 3: fit = 4.728501e-01 fitdelta = 3.5e-06
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
0.9045 -0.0000
0.0000 0.8918
T.core(:,:,2) =
0.2731 -0.0000
-0.0000 -0.2775
T.U{1} =
0.0666 -0.0000
0.9978 -0.0000
0.0000 1.0000
0.0000 0.0001
0.0000 0.0002
T.U{2} =
0.0000 1.0000
1.0000 -0.0000
0.0021 -0.0000
0.0000 0.0005
T.U{3} =
0.0000 0.0000
0.2973 0.9548
0.9548 -0.2973
Specify the initial guess manually
U0 = {rand(5,2),rand(4,2),[]}; %<-- Initial guess for factors of T
T = tucker_als(X,2,struct('dimorder',[3 2 1],'init',{U0}))
Tucker Alternating Least-Squares:
Iter 1: fit = 3.843537e-01 fitdelta = 3.8e-01
Iter 2: fit = 4.493600e-01 fitdelta = 6.5e-02
Iter 3: fit = 4.721643e-01 fitdelta = 2.3e-02
Iter 4: fit = 4.728355e-01 fitdelta = 6.7e-04
Iter 5: fit = 4.728499e-01 fitdelta = 1.4e-05
T is a ttensor of size 5 x 4 x 3
T.core is a tensor of size 2 x 2 x 2
T.core(:,:,1) =
0.9047 -0.0006
0.0006 0.8917
T.core(:,:,2) =
0.2727 -0.0002
-0.0002 -0.2778
T.U{1} =
0.0666 -0.0000
0.9978 -0.0001
0.0001 1.0000
0.0000 0.0001
0.0000 0.0004
T.U{2} =
0.0008 1.0000
1.0000 -0.0008
0.0021 -0.0000
0.0000 0.0010
T.U{3} =
0.0000 0.0000
0.2976 0.9547
0.9547 -0.2976