Holmes And, C C Holmes, B K Mallick · 2007
We present a Bayesian analysis of a piecewise linear model constructed using a basis function approach with a multivariate generalization of the univariate linear spline. Prior distributions are adopted on both the number and locations of the splines. Conditioning on the data produces a Bayes local linear model with distributions on predictions and on local linear parameters that vary smoothly over the design space. The method is spatially adaptive and fairly automatic and the difficult problem of "bandwidth" selection does not arise. Furthermore, by using splines of lower dimension than the data the model automatically removes irrelevant covariates and determines the relative importance of those remaining. KEYWORDS: Bayesian linear regression; nonlinear regression; multivariate splines; piecewise linear; local linear regression. 1 Introduction Many methods exist for modelling the mean regression surface, E(Y jX), of a response variable Y , given a design matrix of covariates or pre...
Cited by 0
Richard Hollamby, I.F. Alcover · 2024
For civil engineering infrastructure, the objectives of monitoring are often to quantify current performance behavior and seek to detect changes in behavior or status that could be attributed to component damage or degradation. For structures, responses such as displacements and stress changes are often strongly influenced by the effects of wind loading, traffic and temperature, the latter presenting via correlated daily and seasonal variations between structural components, often of differing materials. Displacements due to temperature change can be significant, and loading imparted by traffic is often only indirectly measured. In contrast, comparatively small deformations of the structure can be expected due to component degradation or localized failure. Without a principled approach to data analysis there is a risk that such subtle changes will be go unnoticed due to the complex daily, seasonal, and random volatility of the variable actions. Through monitoring two long span bridges in Normandy, France, COWI have developed analysis methods that use measured data from several different sensor types to deliver insights with regards to the defined monitoring objectives. Bayesian multivariate linear regression has been found to be a versatile and scalable model class for predicting structural responses, owing to its compatibility with the linear system theory by which structures are typically analyzed. In this class of model, action-related measurement channels (e.g., temperatures, wind speeds) are typically used as regressors, with response measurements being the dependent variables. Response predictions obtained from deployed models serve to contextualize new measured data and identify anomalous responses. Sophisticated model variants, in which some of the parameters are time-varying, can serve to expose any long-term trends or changes that may be evident, yet otherwise masked, within the measured data. This approach constitutes an efficient automated methodology for processing large data quantities that scales well. Inferred model parameters deliver information that is relevant for digital twinning and asset management purposes. The practical application of this methodology is illustrated via a few case studies analyzing real data.
Cited by 1Open access
Cao, Xuan, Lee, Kyoungjae · ArXiv.org · 2025
We consider jointly estimating the coefficient matrix and the error precision matrix in high-dimensional multivariate linear regression models. Bayesian methods in this context often face computational challenges, leading to previous approaches that either utilize a generalized likelihood without ensuring the positive definiteness of the precision matrix or rely on maximization algorithms targeting only the posterior mode, thus failing to address uncertainty. In this work, we propose two Bayesian methods: an exact method and an approximate two-step method. We first propose an exact method based on spike and slab priors for the coefficient matrix and DAG-Wishart prior for the error precision matrix, whose computational complexity is comparable to the state-of-the-art generalized likelihood-based Bayesian method. To further enhance scalability, a two-step approach is developed by ignoring the dependency structure among response variables. This method estimates the coefficient matrix first, followed by the calculation of the posterior of the error precision matrix based on the estimated errors. We validate the two-step method by demonstrating (i) selection consistency and posterior convergence rates for the coefficient matrix and (ii) selection consistency for the directed acyclic graph (DAG) of errors. We demonstrate the practical performance of proposed methods through synthetic and real data analysis.
Cited by 0Open access
Bo Ning, Seonghyun Jeong, Subhashis Ghosal · arXiv (Cornell University) · 2018
We study frequentist properties of a Bayesian high-dimensional multivariate linear regression model with correlated responses. The predictors are separated into many groups and the group structure is pre-determined. Two features of the model are unique: (i) group sparsity is imposed on the predictors. (ii) the covariance matrix is unknown and its dimensions can also be high. We choose a product of independent spike-and-slab priors on the regression coefficients and a new prior on the covariance matrix based on its eigendecomposition. Each spike-and-slab prior is a mixture of a point mass at zero and a multivariate density involving a $\ell_{2,1}$-norm. We first obtain the posterior contraction rate, the bounds on the effective dimension of the model with high posterior probabilities. We then show that the multivariate regression coefficients can be recovered under certain compatibility conditions. Finally, we quantify the uncertainty for the regression coefficients with frequentist validity through a Bernstein-von Mises type theorem. The result leads to selection consistency for the Bayesian method. We derive the posterior contraction rate using the general theory by constructing a suitable test from the first principle using moment bounds for certain likelihood ratios. This leads to posterior concentration around the truth with respect to the average Rényi divergence of order 1/2. This technique of obtaining the required tests for posterior contraction rate could be useful in many other problems.
Cited by 4Open access
Ufuk Ekiz · DergiPark (Istanbul University) · 2001
In this study, a Bayesian method will be introduced to describe outlying observations in multivariate linear regression.This method was proposed by Chaloner and Brant [3].Later on, Varbanov [7] extended this method to apply to multivariate linear regression.According to Chaloner and Brant, an observation will be accepted as an outlier if the following condition is fulfilled: the posterior probability of the occurrence of the realized error (Arnold Zelner [8]) of an observation being greater than a critical value "k", is higher than the probability an error occurred in the model, with a critical value "k" over the assumed distribution.That is, if pr[( i/, y) > k] > pr( i > k) then the i th observation will be accepted as an outlier.In the second section, the method proposed by Varbanov [7] will be considered.In the application section the existence or non-existence of outlying observations over the posterior distribution of the square form of the realized error in multivariate linear regression data is discussed.
Cited by 4Open access
Sarmad Abdulkhaleq Salih, Emad Aboudi · IRAQI JOURNAL OF STATISTICAL SCIENCES · 2021
This research includes the Bayesian estimation of the parameters of the multivariate partial linear regression model when the random error follows the matrix-variate generalized modified Bessel distribution and found the statistical test of the model represented by finding the Bayes factor criterion, the predictive distribution under assumption that the shape parameters are known.The prior distribution about the model parameters is represented by non-informative information, as well as the simulate on the generated data from the model by a suggested way based on different values of the shape parameters, the kernel function used in the generation was a Gaussian kernel function, the bandwidth (Smoothing) parameter was according to the rule of thumb.It found that the posterior marginal probability distribution of the location matrix and the predictive probability distribution is a matrix-t distribution with different parameters, the posterior marginal probability distribution of the scale matrix is proper distribution but it does not belong to the conjugate family, Through the Bayes factor criterion, it was found that the sample that was used in the generation process was drawn from a population that does not belong to the generalized modified Bessel population.
Cited by 0Open access
Na Hu, Paul-Christian Bürkner, Amalia Arvaniti · PsyArXiv (OSF Preprints) · 2025
Purpose: When studying how factors influence multiple outcomes (e.g., acoustic measures in phonetics), researchers often analyse each outcome separately using a univariate approach. However, this approach ignores relationships between outcomes, which can reduce estimate accuracy and make it difficult to examine how effects are related across outcomes. A multivariate approach addresses these issues by modelling all outcomes jointly. This tutorial illustrates how to fit Bayesian multivariate linear mixed-effects models using the R package brms. Method: We present three example applications in phonetic research, using data of L+H* accents in Greek varying in accentual context and stress location. Results: The multivariate approach reduced uncertainty in population-level effect estimates compared to univariate models and provided a convenient way to examine correlations among effects across outcomes. Conclusions: This tutorial provides guidance on implementing Bayesian multivariate linear mixed-effects models and demonstrates their potential.
Cited by 0Open access
Ousmane Seidou, Jérôme Asselin, Taha B. M. J. Ouarda · Water Resources Research · 2007
Multivariate linear regression is one of the most popular modeling tools in hydrology and climate sciences for explaining the link between key variables. Piecewise linear regression is not always appropriate since the relationship may experiment sudden changes due to climatic, environmental, or anthropogenic perturbations. To address this issue, a practical and general approach to the Bayesian analysis of the multivariate regression model is presented. The approach allows simultaneous single change point detection in a multivariate sample and can account for missing data in the response variables and/or in the explicative variables. It also improves on recently published change point detection methodologies by allowing a more flexible and thus more realistic prior specification for the existence of a change and the date of change as well as for the regression parameters. The estimation of all unknown parameters is achieved by Monte Carlo Markov chain simulations. It is shown that the developed approach is able to reproduce the results of Rasmussen (2001) as well as those of Perreault et al. (2000a, 2000b). Furthermore, two of the examples provided in the paper show that the proposed methodology can readily be applied to some problems that cannot be addressed by any of the above‐mentioned approaches because of limiting model structure and/or restrictive prior assumptions. The first of these examples deals with single change point detection in the multivariate linear relationship between mean basin‐scale precipitation at different periods of the year and the summer–autumn flood peaks of the Broadback River located in northern Quebec, Canada. The second one addresses the problem of missing data estimation with uncertainty assessment in multisite streamflow records with a possible simultaneous shift in mean streamflow values that occurred at an unknown date.
Cited by 65Open access
Rajarshi Das Bhowmik, Seung Beom Seo, Saswata Sahoo · Water · 2018
Statistical models for hydrologic simulation are a common choice among researchers particularly when catchment information is limited. In this study, we adopt a new statistical approach, namely Bayesian regression with multivariate linear spline (BMLS) for long-term simulation of streamflow on a Hydroclimate Data Network (HCDN) site in the United States. The study aims to: (i) evaluate the performance of the BMLS model; (ii) compare the performance of climate model outputs as predictors in hydrologic simulation; and (iii) estimate the changes in streamflow caused by anthropogenic climate change which is defined as the projected change in precipitation and temperature under different greenhouse gas emission scenarios. Performance of the BMLS model is compared with climatology for the validation period. Results suggest that the BMLS model forced with observed monthly precipitation and average temperature exhibits information that is not presented in the climatology of the validation period. Later, we consider Coupled Model Intercomparison Project Phase 5 (CMIP5) historical and hindcast runs to simulate streamflow at the HCDN site. The study found that sea-surface temperature-initialized decadal hindcast runs are performing no better than 20th century historical runs regarding hydrologic simulation. Finally, the changes in mean and variability in streamflow at the HCDN site are estimated by forcing the model with CMIP5 future projections for the period 2000–2049.
Cited by 4Open access
Toussaint, U. · MPG.PuRe (Max Planck Society) · 2015
Cited by 0
Rare & Special e-Zone (The Hong Kong University of Science and Technology) · 2018
Cited by 0
James P. Hobert, Yeun Ji Jung, Kshitij Khare et al. · Scandinavian Journal of Statistics · 2018
Abstract When Gaussian errors are inappropriate in a multivariate linear regression setting, it is often assumed that the errors are iid from a distribution that is a scale mixture of multivariate normals. Combining this robust regression model with a default prior on the unknown parameters results in a highly intractable posterior density. Fortunately, there is a simple data augmentation (DA) algorithm and a corresponding Haar PX‐DA algorithm that can be used to explore this posterior. This paper provides conditions (on the mixing density) for geometric ergodicity of the Markov chains underlying these Markov chain Monte Carlo algorithms. Letting d denote the dimension of the response, the main result shows that the DA and Haar PX‐DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log‐normal, inverted Gamma (with shape parameter larger than d /2) or Fréchet (with shape parameter larger than d /2). The results also apply to certain subsets of the Gamma, F and Weibull families.
Cited by 9
Qian Qin, James P. Hobert · Journal of Multivariate Analysis · 2018
Cited by 7
Qian Qin, James P. Hobert · arXiv (Cornell University) · 2016
Let $π$ denote the intractable posterior density that results when the likelihood from a multivariate linear regression model with errors from a scale mixture of normals is combined with the standard non-informative prior. There is a simple data augmentation algorithm (based on latent data from the mixing density) that can be used to explore $π$. Let $h(\cdot)$ and $d$ denote the mixing density and the dimension of the regression model, respectively. Hobert et al. (2016) [arXiv:1506.03113v2] have recently shown that, if $h$ converges to 0 at the origin at an appropriate rate, and $\int_0^\infty u^{\frac{d}{2}} \, h(u) \, du < \infty$, then the Markov chains underlying the DA algorithm and an alternative Haar PX-DA algorithm are both geometrically ergodic. In fact, something much stronger than geometric ergodicity often holds. Indeed, it is shown in this paper that, under simple conditions on $h$, the Markov operators defined by the DA and Haar PX-DA Markov chains are trace-class, i.e., compact with summable eigenvalues. Many of the mixing densities that satisfy Hobert et al.'s (2016) conditions also satisfy the new conditions developed in this paper. Thus, for this set of mixing densities, the new results provide a substantial strengthening of Hobert et al.'s (2016) conclusion without any additional assumptions. For example, Hobert et al. (2016) showed that the DA and Haar PX-DA Markov chains are geometrically ergodic whenever the mixing density is generalized inverse Gaussian, log-normal, Fréchet (with shape parameter larger than $d/2$), or inverted gamma (with shape parameter larger than $d/2$). The results in this paper show that, in each of these cases, the DA and Haar PX-DA Markov operators are, in fact, trace-class.
Cited by 1Open access
Wiley series in probability and statistics · 2002
10.1002/9780470317105.ch12.abs The prelims comprise: Introduction Simple Linear Regression Multivariate Regression Model Multivariate Analysis of Variance Model Bayesian Inference in the Multivariate Mixed Model
Cited by 0
Zhiqiang Lyu, Wang Geng, Cao Tan · Microelectronics Reliability · 2022
Cited by 22
Peter Rossi · 2005
Covers many important models used in marketing and micro-econometrics applications. The package includes: Bayes Regression (univariate or multivariate dep var), Bayes Seemingly Unrelated Regression (SUR), Binary and Ordinal Probit, Multinomial Logit (MNL) and Multinomial Probit (MNP), Multivariate Probit, Negative Binomial (Poisson) Regression, Multivariate Mixtures of Normals (including clustering), Dirichlet Process Prior Density Estimation with normal base, Hierarchical Linear Models with normal prior and covariates, Hierarchical Linear Models with a mixture of normals prior and covariates, Hierarchical Multinomial Logits with a mixture of normals prior and covariates, Hierarchical Multinomial Logits with a Dirichlet Process prior and covariates, Hierarchical Negative Binomial Regression Models, Bayesian analysis of choice-based conjoint data, Bayesian treatment of linear instrumental variables models, Analysis of Multivariate Ordinal survey data with scale usage heterogeneity (as in Rossi et al, JASA (01)), Bayesian Analysis of Aggregate Random Coefficient Logit Models as in BLP (see Jiang, Manchanda, Rossi 2009) For further reference, consult our book, Bayesian Statistics and Marketing by Rossi, Allenby and McCulloch (Wiley second edition 2024) and Bayesian Non- and Semi-Parametric Methods and Applications (Princeton U Press 2014).
Cited by 29Open access
Solmaz Seifollahi, Kaniav Kamary, Hossein Bevrani · arXiv (Cornell University) · 2021
Univariate and multivariate general linear regression models, subject to linear inequality constraints, arise in many scientific applications. The linear inequality restrictions on model parameters are often available from phenomenological knowledge and motivated by machine learning applications of high-consequence engineering systems (Agrell, 2019; Veiga and Marrel, 2012). Some studies on the multiple linear models consider known linear combinations of the regression coefficient parameters restricted between upper and lower bounds. In the present paper, we consider both univariate and multivariate general linear models subjected to this kind of linear restrictions. So far, research on univariate cases based on Bayesian methods is all under the condition that the coefficient matrix of the linear restrictions is a square matrix of full rank. This condition is not, however, always feasible. Another difficulty arises at the estimation step by implementing the Gibbs algorithm, which exhibits, in most cases, slow convergence. This paper presents a Bayesian method to estimate the regression parameters when the matrix of the constraints providing the set of linear inequality restrictions undergoes no condition. For the multivariate case, our Bayesian method estimates the regression parameters when the number of the constrains is less than the number of the regression coefficients in each multiple linear models. We examine the efficiency of our Bayesian method through simulation studies for both univariate and multivariate regressions. After that, we illustrate that the convergence of our algorithm is relatively faster than the previous methods. Finally, we use our approach to analyze two real datasets.
Cited by 0Open access
Danny Arroyo, Mario Ordaz · Bulletin of the Seismological Society of America · 2010
An application of a linear multivariate Bayesian regression model to compute pseudoacceleration (SA) ground-motion prediction equations (GMPEs )i s presented. The model is able to include the correlation between observations for a given earthquake, the correlation between SA ordinates at different periods, and the correlation between regression coefficients of the ground-motion prediction model. We evaluate the advantages of the Bayesian approach over the traditional regression methods, and we discuss the differences between univariate and multivariate analyses. Because the application of the Bayesian method is in general complex and implies an increase in the numerical effort with respect to the traditional methods, our computer code to perform linear Bayesian analyses is freely available on request.
Cited by 28
C C Holmes, B K Mallick · 1999
A Bayesian method is presented for the nonparametric modelling of univariate and multivariate non-Gaussian response data. Data adaptive multivariate smoothing splines are employed where the number and location of the knot points are treated as random. The posterior model space is explored using a reversible jump Markov chain Monte Carlo sampler. Computational difficulties are partly alleviated by introducing a residual effect in the model that leaves many of the posterior distributions of the model parameters in standard form. The use of the latent residual effect provides a convenient vehicle for modelling correlation in multivariate response data and as such our method can be seen to generalize the seemingly unrelated regression model (Zellner, 1962) to non-Gaussian data. KEYWORDS: Bayesian nonlinear regression; multivariate splines; piecewise linear; local linear regression; SUR; multivariate nonlinear regression; generalised nonlinear regression. 1 1 Introduction R...
Cited by 0