Archive for the ‘Statistical Analysis’ Category

ASA Election Results are In!

Monday, May 13th, 2013

I won the election for the American Statistical Association Section on Statistics in Marketing Chair Elect!
I am honored to have won and thank all of you across the nation and the world who voted for me.
My goal is to end with the section healthier than when I took office.

Attended the presentation: “An excursion through flatland: Braiding interactions of anyons” by Gavin K. Brennen, Dept. of Physics, Macquarie University, Sydney.

Friday, April 12th, 2013

Attended the presentation “An excursion through flatland: Braiding interactions of anyons,” by Gavin K. Brennen, Dept. of Physics, Macquarie University, Sydney.  Anyons are hypothetical particles.  They interact as a function of braiding, knotting or topological relationship with other particles rather than with distance (as with other matter).  Majorana fermions are an example of fermionic anyons.

Just finished reading “Transportability across studies: A formal Approach” by Judea Pearl and Elias Bareinboim, March 2012, UCLA, Technical Report R-372.

Tuesday, April 9th, 2013

Another gem by Professor Pearl and Elias Bareinboim.  It identifies the external validity practice of making threats versus license.  The causal mechanisms in an experimental group and the observational group influence license for external validity.   There is even help  for moving from an observational situation to another observational population.   This is a must reading for any serious statistician or practitioner that creates statistical models and then applies them.

Just finished reading “Commentator: A Front-End User-Interface Module for Graphical and Structural Equation Modeling” by Trent Mamoru Kyono, May 2010, UCLA, Technical Report R-364.

Monday, April 8th, 2013

Requires the compilation of the included C code for the module using Microsoft Visual Studio 2008.  Uses the Structural Equation Modeling software EQS software version 6.1 2006.  A first step in implementing Pearl’s causality interpretation of SEM (Structural Equation Models) using graphical diagrams to encode the causality and provides a means to determine structural causality zeros and constraints.  I have the trial version of the EQS software and plan on compiling the code to replicate some of the examples int he paper.

Wrote an excel script to compute a probability for one binomial variable exceeding another.

Monday, April 8th, 2013

Check out the problem and solution at:
http://www.data4analysis.com/3.html
It is number 22013 on the webpage.
An excel file is included to dynamically change model parameters and recompute the solution.

Just finished reading “The Causal Foundations of Structural Equation Modeling” by Judea Pearl, February 2012, UCLA, Technical Report R-370.

Saturday, April 6th, 2013

This paper has many very useful ideas.
One it highlights and resolves is the symmetry/asymmetry problem or paradox that exists in the fact that the linear model
in statistics has a symmetry between the independent and dependent variables which are then consequently labeled and treated differently.
Dr. Pearl traces the cause back to the equality sign in the linear model.
In one sense it is treated as an assignment equality sign as in computer science and in another sense it is treated as the equality sign in mathematics.
The solution to the paradox is found in the use of structural equation modeling and the treatment of causality.
The methodology requires the specification of causal assumptions, quries of interest and data.
The structural equation methodology results in: logical implications of the assumptions, causal inference, testable implications, statistical inference, model testing (goodness of fit) and conditional claims.
If you ever noticed and were concerned about the symmetry in the linear model and its apparent assymetry in modeling then this is a great paper to read!

I enjoyed Dr. Pearl’s talk at the Joint Statistical meetings and this paper is a great followup to that presentation.

http://www.data4analysis.com for data analyis

Saturday, April 6th, 2013

Do you have data that you want analyzed?
Check out the tiny data set analysis offer on
http://www.data4analysis.com

My Proficiency Levels for Statistical Analysis in Business and Industry

Tuesday, February 2nd, 2010

Definition:

Use the Company process to provide Statistical Analysis services for an internal or external client. 

           

Proficiency Basic

·         Define basic descriptive statistics (mean, median, mode, range, covariance, population standard deviation) and terms, such as: Euclidean Distance, Mahalanobis Distance, Missing Values versus Zeros, Product Popularity Computations, Quantiles and Rank.

·         Explain basic descriptive statistics and terms to clients.

·         Recognize when ones own basic statistical knowledge is inadequate to resolve a problem.  Be willing and know how to bring in appropriate resources.

·         Execute, with guidance, a simple statistical analysis software program.

·         Know how to create scatter plots, bar charts and histograms.

·         Collect or assemble data for statistical analysis.

 

Proficiency Basic Plus

·         Have knowledge and/or practical experience in introductory level statistical analysis including the following concepts: Assignable Causes and Actions, Avoiding Lying with Statistics and Maps, Chernoff Faces, Coded/Dummy Variables, Coefficient of Determination, Combinatorial Analysis, Computation of Expected, Confidence Interval for a datum, Confidence Intervals, Data Basics, Discrete Distributions, Elementary Probability Theory, Elementary Statistical Sampling, Gaussian/Normal Distribution, Geometric Mean, Graphical Display/Analysis, Harmonic Mean, Histograms, Interpolation, Introduction to trend cycle forecasting, Multivariate Star Plots, Lack of Fit in Regression Analysis, Method of Least Squares, Market Share Analysis, Odds Ratio, One-sample t-test, One-way tables, Original Data Plots, Over parameterized model, Paired t-test, Percentiles, Pie Chart Replacements, Quartiles, R – Pearson correlation coefficient, Randomness, Ranks, Raw Data Plots, Residuals, Run tests, Sign Test, Spearman R, Student’s t-test, Travel Distance or Travel Time, Trimmed Means, Types of Data, Uniform distribution, Univariate Regression Analysis and Weighted Mean.

·         Define intermediate descriptive statistics (correlation, r-squared, chi-squared, geometric mean, harmonic mean, t-statistic, F-statistic, sample standard deviation and those included in the introductory concepts above).

·         Know the following statistical areas at an introductory level.  Areas include: computation of expected, confidence intervals, data basics, discrete distributions, elementary probability theory, elementary statistical sampling, graphical display/analysis, introduction to trend-cycle forecasting, market share analysis, one sample t-test, one-way tables, paired t-test and sign test.

·         Know and be able to explain to a client univariate regression analysis.

·         Statistical consulting on a basic level – personal communication skills.

·         Understand and be able to explain a statistical hypothesis testing.

·         With guidance, create statistical analysis output using SAS, Excel and other statistical software.

Proficiency Advanced

·         Have knowledge and/or practical experience in intermediate level statistical analysis (equivalent to a Statistics Bachelor’s degree) including the following concepts: Analysis of Variance, Bayesian Inference, Chi-square Test, Cluster Analysis, Confidence Interval for the mean, Confidence Interval Theory, Confidence Level, Continuous Distributions, Contouring, Mallows Cp statistic, Data Smoothing, Exploratory Data Analysis (Data Mining), External Model Validation, Extrapolation, Frequentist Statistics, Gravity Model Application, Hypothesis Testing, Intermediate Probability Theory, Internal Model Validation, Kendall Tau, K-Means Algorithm, K-Nearest Neighbor algorithm, Kruskall-Wallis Test, Kurtosis, Lack of Fit Tests, Level of Significance (alpha), Matrix Algebra, Maximum Likelihood Method, Metric Spaces (Generalized Distances), Minimax, Multiple or Multivariate Regression, Multi-way tables, Normality Tests, Outliers, Overfitting, Parallel Coordinate Plots, p-level (statistical significance), PRESS Statistic, Quality Control, Residual Analysis, Robust Analysis, Scheffe’s test, Shapiro-Wilks test, Shewart Control Charts, Short run control charts, Signal to Noise ratio, Site Selection, Skewness, Standard error of a proportion, Standardized residuals, Statistical Graphics, Statistical Inference, Statistical Power, Statistical Significance (p-level), Stepwise regression, Studentized residuals, Univariate Time Series Analysis, Time Series Analysis – Seasonal Factors, Type I Error, Type II Error, Weighted Least Squares, Weighted Variance and Wilcoxin Test.

·         Define advanced descriptive statistics (auto-correlation, Kendall Tau, outlier test statistics, robustness statistics and others involved in the intermediate level concepts above).

·         Explain the sources of variation and their impact upon the product or process.

·         Know the intermediate level statistical analysis areas at an intermediate level.  Areas include: analysis of variance, Bayesian inference, chi-square test, confidence interval theory, continuous distributions, exploratory data analysis (data mining), frequentist statistics, gravity model application, hypothesis testing, intermediate probability theory, lack of fit tests, matrix algebra, maximum likelihood method, multi-way tables, normality tests, residual analysis, robust analysis, site selection, statistical graphics, statistical inference, stepwise regression, univariate time series analysis, time series analysis – seasonal factors, Type I error, Type II error and weighted least squares.

·         Know and be able to explain multivariate regression analysis to a client.

·         Statistical consulting at an intermediate level – problem identification.

·         Know the stages of statistical analysis methodology (refer to Company Report #15 – Statistical Modeling Process).

     

Proficiency Advanced Plus

·         Have knowledge and/or practical experience in advanced level of statistical analysis (equivalent to a Statistics Masters’ Degree) including the following concepts: Bayesian Analysis, Categorical Data Analysis, Categorical Trees, Census Data Analysis, Classification Trees, Cook’s Distance, Cross-Validation, Data Mining, Density Estimation, Discrete Multivariate Analysis, Discriminant Function Analysis, Distributions, Duncan’s Test, Dunnett’s Test, Experimental Design, Exploratory Data Analysis, Factor Analysis, Forecasting, General Linear Models (GLIM), Gravity Modeling Theory, Hazard Function, Inductive versus Deductive Reasoning, Local Minima/Maxima, Logit Regression, Model Criticism, Multidimensional Scaling, Multinomial Logit and Probit Regression, Multivariate Time Series Analysis, Network Analysis, Newman-Kuels Test, Nonlinear Regression Analysis, Panel Data, Parallel Coordinate Analysis, Predictive Modeling, Principal component analysis, Probability Theory, Probit Regression, Quality and Productivity, Random Variables, Regression Trees, Reliability, Response surface, Sample survey, Segmentation Theory and Analysis, Spectral plot, Spatial Interaction Models, Spline fitting, Statistical Consulting Skills (technical jargon, asking good questions, listening skills, negotiating fair exchanges, talking about statistics, and resolving difficult situations), Statistical Software (S-Plus, SAS, Excel, etc.), Survival Data Analysis, Variance components (In Mixed Model ANOVA), Visual Display of Quantitative Information, Wald Statistic, Weighted Regression Analysis and Yates Corrected Chi-Square.

·         Recognize when ones own statistical knowledge is inadequate to resolve a problem.  Be willing and know how to bring in appropriate resources.

·         Know the advanced level of statistical analysis concepts, listed above, at some level.

·         Statistical consulting at an advanced level – a level at which the statistical consulting services may be sold to the client.

·         Determine what statistical analysis packages will be useful for the project.

·         Know how to interpret output from all statistical analysis packages.

·         Lead a complex Statistical Analysis including establishing an overall strategy, providing on-going guidance, reviewing results and solving complex issues quickly.

·         Educate clients and other employees on the pros and cons of various Statistical Analysis approaches.

·         Manage the execution of a Statistical Analysis including delivery dates, quality and project costs.

·         Present Statistical Analysis results in a skilled and professional manner.

·         Influence client decisions regarding Statistical Analysis.

·         Sought out by clients and employees as an experienced, knowledgeable resource in Statistical Analysis.

·         Respond knowledgeably and professionally to requests or challenges under pressure.

·         Understand current changes in the industry and how they will impact Statistical Analysis.

·         Understand the various legal guidelines/statutes/precedences that impact Statistical Analysis.

·         Qualify and testify as an “expert” in issues of statistics and statistical analysis.

·         Produce two or more written documents, in the “Company Statistical Report” series, per year.

·         Attend at least one relevant professional meeting (such as, The Joint Statistical Meetings) per year.

     

Proficiency Expert

·         Have knowledge and/or practical experience in the expert level of statistical analysis (equivalent to a Statistics Ph.D. degree) including the following concepts: Approximation Theory, Artificial Intelligence, Bayesian Statistics, Bonferroni test, Censoring, Display of Statistical Equations and Technical Information (TeX, LaTeX, Mathtype, etc.), Econometric Models/Modeling, Entropy, Estimable Functions, Expert Systems, Fractals, Function Estimation, Function Smoothing, Functional Analysis, Genetic Algorithm, Group Theory, Graph Theory, Hilbert Space Theory, Interval Analysis, Latent Variable Models, Markov Chain Monte Carlo, Mathematical Statistics, Multivariate density Estimation, Network Flows, Network Optimization, Neural Networks, Non parametric methods, Nonlinear Estimation, Optimization Methods/Routines, Orthogonal Polynomials, Predictive Data Mining, Quality Theory, Signal detection theory, Simulations, Simplex algorithm, Spatial Data Analysis, Special Functions, Statistical Control Theory, Statistical Signal Processing, Statistics and the Law, Stochastic Processes, Structural Equation Modeling, The Scientific Method, Theoretical Statistics, Transmitted Variation and Wavelets.

·         Recognized by Company and/or clients as an “expert” in Statistical Analysis.

·         Know and be able to utilize the mathematical underpinnings of statistical methods.

·         Statistical Consulting Skills at an expert level including: problem identification, opportunity identification, application field technical jargon, asking good questions, interpretation of complex ideas into understandable examples, listening skills, negotiating fair exchanges, talking about statistics, and resolving difficult situations.

·         Respond knowledgeably and professionally to requests or challenges under significant pressure.

·         Train and coach new “experts” in advanced Statistical Analysis skills.

·         Keep abreast of research in the field of statistics at a professional level.

·         Attend and participate (present a paper, presentation, discussion or poster session) in at least two relevant professional meetings per Year (such as, The Joint Statistical Meetings).