³ò ŒB_Kc@s’ddkZddkZddkZddkZddkTdefd„ƒYZd„Zd„Zde fd„ƒYZ d e fd „ƒYZ dS( iÿÿÿÿN(t*tVectorSpaceClusterercBskeZdZeed„Zeed„Zd„Zd„Zd„Z d„Z d„Z d„Z d „Z RS( s© Abstract clusterer which takes tokens and maps them into a vector space. Optionally performs singular value decomposition to reduce the dimensionality. cCsd|_||_||_dS(s* @param normalise: should vectors be normalised to length 1 @type normalise: boolean @param svd_dimensions: number of dimensions to use in reducing vector dimensionsionality with SVD @type svd_dimensions: int N(tNonet_Ttt_should_normaliset_svd_dimensions(tselft normalisetsvd_dimensions((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyt__init__s  c Csmt|ƒdjpt‚|iot|i|ƒ}n|ioÖ|it|dƒjo¼tiiti t |ƒƒƒ\}}}||i ti |iti ƒ}|dd…d|i…f}|d|i…dd…f} ti ti || ƒƒ}ti |ƒ|_n|i||ƒ|o8|iG|GHg} |D]} | |i| ƒqH~ SndS(Ni(tlentAssertionErrorRtmapt _normaliseRtnumpytlinalgtsvdt transposetarraytidentitytFloat64tmatrixmultiplyRtcluster_vectorspacetclassify( Rtvectorstassign_clustersttracetutdtvttStTtDtt_[1]tvector((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pytcluster!s $*  cCs tƒ‚dS(sD Finds the clusters using the given set of vectors. N(R (RRR((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR:scCsb|io|i|ƒ}n|idjoti|i|ƒ}n|i|ƒ}|i|ƒS(N(RR RRRRtclassify_vectorspacet cluster_name(RR"R#((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR@s  cCs tƒ‚dS(sN Returns the index of the appropriate cluster for the vector. N(R (RR"((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR$HscCsV|io|i|ƒ}n|idjoti|i|ƒ}n|i||ƒS(N(RR RRRRtlikelihood_vectorspace(RR"tlabel((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyt likelihoodNs  cCs,|i|ƒ}||jodSndSdS(sP Returns the likelihood of the vector belonging to the cluster. gð?gN(R$(RR"R#t predicted((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR&UscCsJ|io|i|ƒ}n|itjoti|i|ƒ}n|S(sU Returns the vector after normalisation and dimensionality reduction (RR RRRR(RR"((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR"]s  cCs|titi||ƒƒS(s7 Normalises the vector to unit length. (tmathtsqrtRtdot(RR"((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR gs(t__name__t __module__t__doc__tFalseRR R#RRR$R(R&R"R (((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyRs       cCs#||}titi||ƒƒS(s} Returns the euclidean distance between vectors u and v. This is equivalent to the length of the vector (u - v). (R*R+RR,(Rtvtdiff((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyteuclidean_distancems cCsBti||ƒtiti||ƒƒtiti||ƒƒS(se Returns the cosine of the angle between vectors v and u. This is equal to u.v / |u||v|. (RR,R*R+(RR1((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pytcosine_distanceust_DendrogramNodecBs,eZdZd„Zed„Zd„ZRS(s Tree node of a dendrogram. cGs||_||_dS(N(t_valuet _children(Rtvaluetchildren((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR s cCsb|io8g}x'|iD]}|i|i|ƒƒqW|Sn|o|igSn|gSdS(N(R7textendtleavesR6(RtvaluesR;tchild((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR;ƒs  cCsê|i|fg}x¡t|ƒ|jo|iƒ\}}|ip|i||fƒPnxH|iD]=}|io|i|i|fƒqf|id|fƒqfW|iƒqWg}x'|D]\}}|i|iƒƒqÃW|S(Ni(R6R tpopR7tpushtappendtsortR;(RtntqueuetprioritytnodeR=tgroups((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyRFŽs"    (R-R.R/R tTrueR;RF(((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR5|s  t DendrogramcBs>eZdZgd„Zd„Zd„Zd„Zd„ZRS(s Represents a dendrogram, a tree with a specified branching order. This must be initialised with the leaf items, then iteratively call merge for each branch. This class constructs a tree representing the order of calls to the merge function. cCsLg}|D]}|t|ƒq ~|_ti|iƒ|_d|_dS(ss @param items: the items at the leaves of the dendrogram @type items: sequence of (any) iN(R5t_itemstcopyt_original_itemst_merge(RtitemsR!titem((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyR «s*cGs‘t|ƒdjpt‚t|ig}|D]}||i|q.~Œ}|id7_||i|d||jo| | ||dƒƒn| | |||ƒƒ| |ƒ| |ðss tc3s"x|]}|iˆƒVqWdS(N(tcenter(t.0RN(twidth(s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pys sN(RSRTRU(R RIR5RLRKR`R6tmaxR R>R7tindextmintrangeR@RAtjoin(RtJOINtHLINKtVLINKRRR;R!tleaftlast_rowR\RaRCt_[2]t verticalsRDREtchild_left_leafROtmin_idxtmax_idxRPR=tvertical((RZRgR[s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pytshowÐsT  *  '  ''    #cCsYt|iƒdjot|i|iŒ}n|id}|itƒ}dt|ƒS(Niis(R RIR5RLR;R0(RRRR;((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyt__repr__ s  (R-R.R/R RQRFRxRy(((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyRH£s   =( RJR]R*RtapitClusterIRR3R4tobjectR5RH(((s'/p/zhu/06/nlp/nltk/nltk/cluster/util.pyss     ^  '