Anal1a, in terms of clumping, doesn’t look very good, at least not anymore. That was not previously the case, but I had revised my algorithm to make it symmetrical as per the insructions of a computing statistician here at the University of Sussex. He claimed that the Procrustes Rotation needed symmetric data and my nonsymmetric data, where Doc1 vs Doc2 didn’t have the same similarity as Doc2 vs Doc1, was not going to work. That change has, I believe, altered the efficacy of the algorithm and things are no longer clumped together as promisingly as they were previously. The clumps should be a two- or three-letter short code followed by a digit. Therefore, ac1 and ac2 belong together. Pl1, pl2, and pl3 belong together, and so on. The clumping is significantly better in the already symmetric cosine normalization algorithm (anal2a). The two speech processing documents are clumped together (sp1 and sp2), all of the Power PC and G4 documents are together (pp1, pp2, g4c), and the three Pine Lake tornado stories are clumped far away from everything else (which is all computer-related) and together on their own. Excellent clumping, in fact. So the hypernym hypothesis looks like, on these short documents, it is working well with cosine normalization.

Here’s the final bit of loveliness: comparing one MDS cluster diagram against another. MDS output is mapped to the vector space independently. That is, the same data will produce the same visualization or mapping, but different data is mapped to a different vector space, so you cannot just compare one MDS matrix to another directly. That is where Procrustes Rotation comes in. It applies a series of intelligent matrix transformations, trying to map the second vector matrix onto the source vector matrix. As a side benefit, essential in my case, it always provides a fitness measure to tell you how close the two were. on a scale of 0 to 1. So these two, as you can see (see above image), even after the transformations, were not that close together. As it happens, though, this is not particularly useful information to know. I am currently more interested in assessing how close the two algorithms are to human classifiers.

This recent success gave us plenty to discuss, particularly with respect to metric and non-metric data. The MDS community calls source data metric when the similarity or dissimilarity data is symmetric. That is, the value at row 2, column 1 is the same as the value at row 1, column 2. Classical multi-dimensional scaling (MDS) is designed to only work with metric data. SPSS includes the ALSCAL and PROXSCAL MDS algorithms which can work with non-metric data, but MATLAB’s classical MDS does not because it treats things as Eucledean distances–another reason why I had to alter the Anal1a algorithm. The primary reason I now had metric data for everything, however, was because the computing statistician had told me I needed it for the Procrustes. Hawever, as we were examining my output, it occurred to me that Procrustes did not really care if the data was symmetric, so long as the dimensions of the data were the same (the same number of rows and columns). Which leads us to question whether the application of the method is statistically sensible or not. To that end, I need to track down a new computing statistician and perhaps a mathematician and discuss the process with them. My original computing statistician has retired.

Earlier I said that comparing one machine to another, to see how they fit is not useful information, but what would be interesting is to prepare a matrix of all the possible combinations of human judgements, cosine normalization, and weird formula:

cosine   wrd form.   human
cosine (anal2a)		x
weird formula (anal1a)           x
human                                        x

So that is my task for my next meeting (on the 16th of June). Before then, I need to figure out how to get MATLAB to take multiple tables as data. In SPSS, I could paste in several tables (representing all of the people’s individual data, for example) and it would work with that. That is necessary in order to aggregate the peopel to do the comparison. Onward ho, then! Progress at last!

Share/Save

David L. Jones had a series of MATLAB pointers which included links to toolboxes for non-metric multidimensional scaling. The latter toolkit, developed by Mark Steyvers, doesn’t come with any documentation and includes some DLLs, so I wonder if only works in Windows somehow? I couldn’t find any other reference to it on the web.
I was waiting for the Mac support person to come install a new version of MATLAB for me. The demo installation and toolkits I installed last fall have long since expired. I’m also still waiting to hear back from the UNIX software support people in the department about acquiring one of the pool licenses for use with a copy of MatLab on my Macintosh off campus. Latish on in the day, I found the Mac support person and acquired a valid license file. It didn’t work right off the bat. I had to edit the file and change the linefeeds from Macintosh ones to UNIX ones. After that, it worked great and it looks fantastic. So I should be able to start doing something with that soon. It also works from home, surprisingly enough, as long as I have an Internet connection, so that will be quite convenient. Hurrah! I am moving ahead.

Share/Save

This is my current revised program:
% Import anal1ahyp.txt into anal1a matrix variable. Do this first! It’s not done
% automatically here. Need first to do something like anal1a = abs(100 – anal1a) to
% get proper dissimilarity values to work with. The default ones in the files don’t
% work as is.
docs = {‘g4c’, ‘pp1′, ‘pp2′, ‘msc’, ‘pl1′, ‘pl2′, ‘pl3′, ‘sp1′, ‘sp2′, ‘ac1′, ‘ac2′, ‘bws’};
[Anal1aMDS, eigvals] = cmdscale(Anal1aRaw);
plot(1:length(eigvals),eigvals,’bo-’);
graph2d.constantline(0,’LineStyle’,':’,'Color’,[.7 .7 .7]);
axis([1,length(eigvals),min(eigvals),max(eigvals)*1.1]);
xlabel(‘Eigenvalue number’);
ylabel(‘Eigenvalue’);
plot(Anal1aMDS(:,1),Anal1aMDS(:,2),’bx’);
axis(max(max(abs(Anal1aMDS))) * [-1.1,1.1,-1.1,1.1]); axis(‘square’);
text(Anal1aMDS(:,1),Anal1aMDS(:,2),docs,’HorizontalAlignment’,'left’);
hx = graph2d.constantline(0,’LineStyle’,'-’,'Color’,[.7 .7 .7]);
hx = changedependvar(hx,’x');
hy = graph2d.constantline(0,’LineStyle’,'-’,'Color’,[.7 .7 .7]);
hy = changedependvar(hy,’y');
% Import anal2ahyp.txt into anal2a matrix variable. Do this first! It’s not done
% automatically here.
Anal2aRaw = abs(100 – Anal2aRaw);
[Anal2aMDS, eigvals] = cmdscale(Anal2aRaw);
% do procustes
[D,Z] = procrustes(Anal1aMDS, Anal2aMDS(:,1:2));
plot(Anal1aMDS(:,1), Anal1aMDS(:,2), ‘bo’, Z(:,1), Z(:,2), ‘rd’);
text(Anal1aMDS(:,1)+0.5,Anal1aMDS(:,2), docs, ‘Color’, ‘b’);
text(Z(:,1)+0.5,Z(:,2),docs, ‘Color’, ‘r’);
xlabel(‘East of the Sun’);
ylabel(‘West of the Moon’);
legend({‘Anal1a’, ‘Anal2a’}, 4);

Share/Save