Uses of the T"-Statistic, 5. Some Optimal Properties or the T 1 -Test, 5. The Problem of Classification, 6. Pro ;eOureJ. Probabilities of Misc1assification, 6. Classification into One of Several Populations, 6. Introduction, 7. The Wishart Distribution, 7. Some Properties of the Wishart Distribution, 7.
Cochran's Theorem, 7. The Generalized Variance, 7. Improved Estimation of the Covariance Matrix, 7. Introduction, 8. Estimators of Parameters in Multivariate Linear Regl'ession, 8. Other Criteria for Testing the Linear Hypothesis, 8. Multivariate Analysis of Variance, 8. Some Optimal Properties of Tests, 8. I ntroductiom, 9. Other Criteria, 9. Step-Down Procedures, 9. An Example, 9. The Case of Two Sets of Variates, 9. Monotonicity of Power Functions of Tests of Independence of 9.
Introduction, Asymptotic EXpansions of the Distributions of the Criteria, The Case of Two Populations, Admissibility of Tests, Definition of Principal Components in the Populat 1 0n, An Example, Statistical Inference, Canonical Correlations and Variates in the Population, Estimation of Canonical Correlations and Variates, Reduced Rank Regression, The Case of Two Wishart Matrices, Canonical Correlations, Asymptotic Distribution in a Regression Model, The Model, Estimation for Fixed Factors, Factor Interpretation and Transformation, Estimation for Identification by Specified Zeros, Undirected Graphs, Directed Graphs, Chain Graphs, Definition of a Matrix and Operations on Matrices, A.
Partitioned Vectors and Matrices, A. Some Miscellaneous Results, A. Tables of Significance Points for the. Since the second edition was published, multivariate analysis has been developed and extended in many directions. Rather than attempting to cover, or even survey, the enlarged scope, I have elected to elucidate several aspects that are particularly interesting and useful for methodology and comprehen- sion. Earlier editions included some methods that could be carried out on an adding machine!
In the twenty-first century, however, computational tech- niques have become so highly developed and improvements come so rapidly that it is impossible to include all of the relevant methods in a volume on the general mathematical theory.
Some aspects of statistics exploit computational power such as the resampling technologies; these are not covered here. The definition of multivariate statistics implies the treatment of variables that are interrelated.
Several chapters are devoted to measures of correlation and tests of independence. A new chapter, "Patterns of Dependence; Graph- ical Models" has been added. A so-called graphical model is a set of vertices Or nodes identifying observed variables together with a new set of edges suggesting dependences between variables.
The algebra of such graphs is an outgrowth and development of path analysis and the study of causal chains. A graph may represent a sequence in time or logic and may suggest causation of one set of variables by another set. Another new topic systematically presented in the third edition is that of elliptically contoured distributions. The multivariate normal distribution, which is characterized by the mean vector and covariance matrix, has a limitation that the fourth-order moments of the variables are determined by the first- and second-order moments.
The class. A density in this class has contours of equal density which are ellipsoids as does a normal density, but the set of fourth-order moments has one further degree of freedom. This topic is expounded by the addition of sections to appropriate chapters.
Reduced rank regression developed in Chapters 12 and 13 provides a method of reducing the number of regression coefficients to be estimated in the regression of one set of variables to another. This approach includes the limited-information maximum-likelihood estimator of an equation in a simul- taneous equations model.
The preparation of the third edition has been benefited by advice and comments of readers of the first and second editions as well as by reviewers of the current revision. In addition to readers of the earlier editions listed in those prefaces I want to thank Michael Perlman and Kathy Richards for their assistance in getting this manuscript ready.
Stanford, California February T. This new edition purports to bring the original edition up to date by substantial revision, rewriting, and additions. The basic approach has been maintained, llamely, a mathematically rigorous development of statistical methods for observations consisting of several measurements or characteristics of each sUbject and a study of their properties. The general outline of topics has been retained.
The method of maximum likelihood has been augmented by other consid- erations. In point estimation of the mf"an vectOr and covariance matrix alternatives to the maximum likelihood estimators that are better with respect to certain loss functions, such as Stein and Bayes estimators, have been introduced. In testing hypotheses likelihood ratio tests have been supplemented by other invariant procedures.
New results on distributions and asymptotic distributions are given; some significant points are tabulated. Properties of these procedures, such as power functions, admissibility, unbi- asedness, and monotonicity of power functions, are studied. Simultaneous confidence intervals for means and covariances are developed. A chapter on factor analysis replaces the chapter sketching miscellaneous results in the first edition. Some new topics, including simultaneous equations models and linear functional relationships, are introduced.
Additional problems present further results. FOr a comprehensive listing of papers until and books until the reader is referred to A Bibliography of Multivariate Statistical Analysis by Anderson, Das Gupta, and Styan I am in debt to many students, colleagues, and friends for their suggestions and assistance; they include Yasuo Amemiya, James Berger, Byoung-Seon Choi. Special thanks go to Johanne Thiffault and George P. H, Styan for their precise attention. Seven tables of significance points are given in Appendix B to facilitate carrying out test procedures.
Tables 1, 5, and 7 are Tables 47, 50, and 53, respectively, of Biometrika Tables for Statisticians, Vol. Pearson and H. Table 2 is made up from three tables prepared by A.
Simulation and Computa- tion Tables 3 and 4 are Tables 6. Nagarscnkcr and K. Pillai, Aerospacc Research Laboratorics The author is indebted to the authors and publishers listed above for permission to reproduce these tables. California June T. It is hoped that the book will also serve as an introduction to many topics in this area to statisticians who are not students and will be used as a reference by other statisticians.
For several years the book in the form of dittoed notes has been used in a two-semester sequence of graduate courses at Columbia University; the first six chapters constituted the text for the first semester, emphasizing correla- tion theory. It is assumed that the reader is familiar with the usual theory of univariate statistics, particularly methods based on the univariate normal distribution.
A knowledge of matrix algebra is also a prerequisite; however, an appendix on this topic has been included. It is hoped that the more basic and important topics are treated here, though to some extent the coverage is a matter of taste. Some 0f the mOre recent and advanced developments are only briefly touched on in the late chapter.
The method of maximum likelihood is used to a large extent. This leads to reasonable procedures; in some cases it can be proved that they are optimal. In many situations, however, the theory of desirable or optimum procedures is lacking. Over the years this manuscript has been developed, a number of students and colleagues have been of considerable assistance. The preparation of this manuscript was sup- ported in part by the Office of Naval Research.
The measurements made on a single individual can be assembled into a column vector. We think of the entire vector as an observation from a multivariate population or distribution. When the individual is drawn ran- domly, we consider the vector as a random vector with a distribution or probability law describing that population. The set of observations on all individuals in a sample constitutes a sample of vectors, and the vectors set side by side make up the matrix of observations.
We shall see that it is helpful in visualizing the data and understanding the methods to think of each observation vector as constituting a point in a Euclidean space, each coordinate corresponding to a measurement or vari- able. Indeed, an early step in the statistical analysis is plotting the data; since tWhen data are listed on paper by individual, it is natural to print the measurements on one individual as a row of the table; then one individual corresponds to a row vector.
Since we prefer to operate algebraically with column vectors, we have chosen to treat observations in terms of column vectors.
In practice, the basic data set may weD be on cards, tapes, or di. Characteristics of a univariate distribution of essential interest are the mean as a measure of location and the standard deviation as a measure of variability; similarly the mean and standard deviation of a univariate sample are important summary measures. In multivariate analysis, the means and variances of the separate measurements-for distributions and for samples -have corresponding relevance.
An essential aspect, however, of multivari- ate analysis is the dependence between the different variables. The depen- dence between two variables may involve the covariance between them, that is, the average products of their deviations from their respective means.
The covariance standardized by the corresponding standard deviations is the correlation coefficient; it serves as a measure of degree of A set of summary statistics is the mean vector consisting of the univariate means and the covariance matrix consisting of the univariate variances and bivari- ate covariances. An alternative set of summary statistics with the same information is the mean vector, the set of' standard deviations, and the correlation matrix. Similar parameter quantities describe location, variability, and dependence in the population or for a probability distribution.
The multivariate nonnal distribution is completely determined by its mean vector and covariance and the sample mean vector and covariance matrix constitute a sufficient set of statistics. The measurement and analysis of dependence between between sets of variables, and between variables and sets of variables are fundamental to multivariate analysis. The multiple correlation coefficient is an extension of the notion of correlation to the relationship of one variable to a set of variables.
The partial correlation coefficient is a measure of dependence between two variables when the effects of other correlated variables have been removed. The various correlation coefficients computed from samples are used to estimate corresponding correlation coefficientS of distributions. In this hook tests or or independence are developed. The proper- ties of the estimators and test proredures are studied for sampling from the multivariate normal distribution.
A number of statistical problems arising in multivariate populations are straightforward analogs of problems arising in univariate populations; the suitable methods for handling these problems are similarly related. For example, ill the univariate case we may wish to test the hypothesis that the mean of a variable is zero; in the multivariate case we may wish to test the hypothesis that the vector of the means of several variables is the zero vector.
The analog of the Student t-test for the first hypOthesis is the generalized T 2 -test. The analysis of variance of a single variable is adapted to vector 1. A comparison of variances is generalized into a comparison of covariance matrices.
The test procedures of univariate statistics are generalized to the multi- variate case in such ways that the dependence between variables is taken into account. These methods may not depend on the coordinate system; that is, the procedures may be invariant with respect to linear transformations that leave the nUll. In some problems there may be families of tests that are invariant; then choices must be made. Optimal properties of the tests are considered. For some other purposes, however, it may be important to select a coordinate system so that the variates have desired statistical properties.
One might say that they involve characterizations of inherent properties of normal distributions and of samples. These are closely related to the algebraic problems of canonical forms of matrices. An example is finding the normal- ized linear combination of variables with maximum or minimum variance finding principal components ; this amounts to finding a rotation of axes that carries the covariance matrix to diagonal form.
Another example is characterizing the dependence between two sets of variates finding canoni- cal correlations. These problems involve the characteristic roots and vectors of various matrices.
The statistical properties of the corresponding sample quantities are treated. Some statistical problems arise in models in which means and covariances are restricted. Factor analysis may be based on a model with a population covariance matrix that is the sum of a positive definite diagonal matrix and a positive semidefinite matrix of low rank; linear str Jctural relationships may have a Similar formulation.
The simultaneous equations system of economet- rics is another example of a special model. A major reason for basing statistical analysis on the normal distribu- tion is that this probabilistic model approximates well the distribution of continuous measurements in many sampled popUlations. In fact, most of the methods and theory have been developed to serve statistical analysis of data.
Francis Galton, th.! Karl Pearson and others carried on the development of the theory and use of differe'lt kinds of correlation coefficients t for studying proble. Fisher further developed methods for agriculture, botany, and anthropology, including the discriminant function for classification problems. In another direction, analysis of scores 01 mental tests led to a theory, including factor analysis, the sampling theory of which is based on the normal distribution.
In these cases, as well as in agricultural experiments, in engineering problems, in certain economic problems, and in other fields, the multivariate normal distributions have been found to be sufficiently close approximations to the populations so that statistical analy- ses based on these models are justified. The univariate normal distribution arises frequently because the effect studied is the sum of many independent random effects.
Similarly, the multivariate normal distribution often occurs because the multiple meaSUre- ments are sums of small independent effects. Statistical theory based on the normal distribution has the advantage that the multivariate methods based on it are extensively developed and can be studied in an organized and systematic way.
This is due not only to the need for such methods because they are of practical US,! The 'suitable methods of analysis are mainly based on standard operations of matrix. The point of view in this book is to state problems of inference in terms of the multivariate normal distributions, develop efficient and often optimum methods in this context, and evaluate significance and confidence levels in these terms.
The procedures are appropriate to many nonnormal distributions, f For a detailed study of the development of the ideas of correlation, see Walker , 1. This inflexibility of normal methods with respect to moments of order greater than two can be reduced by including a larger class of elliptically contoured distributions. In the univariate case the normal distribution is determined by the mean and variance; higher-order moments and properties such as peakedness and long tails are functions of the mean and variance.
Similarly, in the multivariate case the means and covariances or the means, variances, and correlations determine all of the properties of the distribution. That limitation is alleviated in one respect by consideration of a broad class of elliptically contoured distributions. That class maintains the dependence structure, but permits more general peakedness and long tails. This study leads to more robust methods. The development of computer technology has revolutionized multivariate statistics in several respects.
As in univariate statistics, modern computers permit the evaluation of obsetved variability and significance of results by resampling methods, such as the bootstrap and cross-validation. Such methodology reduces the reliance on tables of significance points as well as eliminates some restrictions of the normal distribution.
Nonparametric techniques are available when nothing is known about the underlying distributions. Jransformations of variables to approximate normality and homoscedas- tIClty.
The availability of modem computer facilities makes possible the analysis of large data sets and that ability permits the application of multivariate methods to new areas, such as image analysis, and more effective a.
Moreover, new problems of statistical analysis arise, such as sparseness of parameter Or data matrices. Because hardware and software development is so explosive and programs require specialized knowledge, we are content to make a few remarks here and there about computation.
Packages of statistical programs are available for most of the methods. In Section 2. The variances, covariances, and correlations-called partial correlations-are constants. The multiple correlation coefficient is the maximum correlation between a scalar random variable and linear combination of other random variables; it is a measure of association be- tween one variable and a set of others.
The fact that marginal and condi- tional distributions of normal distributions are normal makes the treatment of this family of coherent. Joint Distributions In this section we shall consider the notions of joint distributions of several derived marginal distributions of subsets of variables, and derived conditional distributions.
First consider the case of two real random variables t X and Y. Probabilities of events defined in terms of these variables can be obtained by operations involving the cumulative distribution function abbrevialed as cdf , 1 F x,y defined for every pair of real numbers x, y.
The pair of random variables ex, Y defines a random point in a plane. The probability of the random point X, Y falling in any set E for which the following int.! In later chapters we may be unable to hold to this convention because of other complications of notation. Since j u, [ is continuous, 6 is approximately j x, y tlx tl y. Now we consider the case of p random variables Xl' X 2 , , Xp' The cdf is 8 defined for every set of real numbers XI"'" xp' The density function, if F Xh'''' x p is absolutely continuous, is 9 almost everywhere , and.
Let this be F x. Now we turn to the general case. The marginal density of Xl. The joint moments of a subset of variates can be computed from the marginal distribution; for example, 2. If the cc'f of XI"'" Xp is F x l ,. The set Xl".. One result of independence is that joint moments factor. For example, if Xl".. The probability that Xl"'" Xp falls in a region R is given by 11 ; the probability that Y 1 , , Yp falls in a region S is If S is the transform of R, that is, if each point of R transforms by 33 into a point of S and if each point of S transforms into R by 34 , then 11 is equal to 3U by the usual theory of transformation of multiple integrals.
From this follows the assertion that 36 is the density of Y 1 , , Yp' 2. The density function of a multivariate normal distribution of XI"'" Xp has an analogous form. Written in matrix notation, the similarity of the multivariate normal density 6 to the univariate density 1 is clear. Throughout this book we shall use matrix notation and operations. Some characterizations of the multivariate t distribution. Huntington, NY: Robert E. The Multivariate Normal Distribution. Lukacs A multivariate t vector X is represented in two different forms, one associated with a normal vector and an independent chi-squared variable, and the other with a normal.
Use the link below to share a full-text version of this article with your friends and colleagues. That s right, all we need is the price of a paperback book to sustain a non-profit website the whole world depends. Perfected over three editions. The application of multivariate statistics is multivariate analysis. Theodore Wilbur Anderson - Wikipedia. Working with R or SAS. Professor of Statistics and Economics. The sample data may be heights and weights of some individuals drawn randomly from a population of school children in a given city, or the statistical treatment.
Adds two new chapters, along with a number of new sections. Summary For more than four decades An Introduction to Multivariate Statistical Analysis has been an invaluable text for students and a resource for professionals wishing to acquire a basic knowledge of multivariate statistical analysis. Since the previous edition, the field has grown significantly.
Images Donate icon An illustration of a heart shape Donate Ellipses icon An illustration of text ellipses. An introduction to multivariate statistical analysis Item Preview. EMBED for wordpress. Want more?