我想知道glm
中如何指定默认起始值。
此post建议将默认值设置为零。 one表示其背后有一个算法,但是相关链接已断开。
我试图用算法跟踪拟合简单的逻辑回归模型:
set.seed(123)
x <- rnorm(100)
p <- 1/(1 + exp(-x))
y <- rbinom(100, size = 1, prob = p)
# to see parameter estimates in each step
trace(glm.fit, quote(print(coefold)), at = list(c(22, 4, 8, 4, 19, 3)))
首先,不指定初始值:
glm(y ~ x, family = "binomial")
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
NULL
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.386379 1.106234
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3991135 1.1653971
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3995188 1.1669508
第一步,初始值为NULL
。
第二,我将起始值设置为零:
glm(y ~ x, family = "binomial", start = c(0, 0))
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0 0
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3177530 0.9097521
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3909975 1.1397163
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3994147 1.1666173
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3995191 1.1669518
而且我们可以看到第一种方法和第二种方法之间的迭代是不同的。
[要查看由glm
指定的初始值,我尝试仅通过一次迭代就拟合模型:
glm(y ~ x, family = "binomial", control = list(maxit = 1))
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
NULL
Call: glm(formula = y ~ x, family = "binomial", control = list(maxit = 1))
Coefficients:
(Intercept) x
0.3864 1.1062
Degrees of Freedom: 99 Total (i.e. Null); 98 Residual
Null Deviance: 134.6
Residual Deviance: 115 AIC: 119
参数的估计(毫不奇怪)对应于第二次迭代中第一种方法的估计,即[1] 0.386379 1.106234
将这些值设置为初始值会导致与第一种方法相同的迭代序列:
glm(y ~ x, family = "binomial", start = c(0.386379, 1.106234))
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.386379 1.106234
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3991135 1.1653971
Tracing glm.fit(x = structure(c(1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, .... step 22,4,8,4,19,3
[1] 0.3995188 1.1669508
所以问题是,这些值如何计算?
start=c(b0,b1)
将eta初始化为b0+x*b1
(mu到1 /(1 + exp(-eta)))start=c(0,0)
将eta初始化为0(μ到0.5),与y或x值无关。start=NULL
如果y = 1,则初始化eta = 1.098612(mu = 0.75),与x值无关。[start=NULL
如果y = 0,则初始化eta = -1.098612(mu = 0.25),与x值无关。
w
的精神,计算出一次eta(因此计算了mu和var(mu)),计算了z
和qr.solve(cbind(1,x) * w, z*w)
并将其发送到QR解算器。
[建立Roland的注释:我创建了glm.fit.truncated()
,将glm.fit
移至C_Cdqrls
调用,然后将其注释掉了。 glm.fit.truncated
输出z
和w
值(以及用于计算z
和w
的数量的值),然后将其传递给C_Cdqrls
调用:
## call Fortran code via C wrapper
fit <- .Call(C_Cdqrls, x[good, , drop = FALSE] * w, z * w,
min(1e-7, control$epsilon/1000), check=FALSE)
可以阅读有关C_Cdqrls
here的更多信息。幸运的是,基数R中的函数qr.solve
直接进入glm.fit()
中调用的LINPACK版本。
因此我们针对不同的起始值规格运行glm.fit.truncated
,然后使用w和z值调用qr.solve
,然后我们将看到如何计算“起始值”(或第一个显示的迭代值) 。如Roland所指出的,在glm()中指定start=NULL
或start=c(0,0)
会影响w和z的计算,而start
的not会影响计算。
对于开始= NULL:z
是一个向量,其中元素的值为2.431946或-2.431946,w
是一个向量,其中所有元素的值为0.4330127:
start.is.null <- glm.fit.truncated(x,y,family=binomial(), start=NULL)
start.is.null
w <- start.is.null$w
z <- start.is.null$z
## if start is NULL, the first displayed values are:
qr.solve(cbind(1,x) * w, z*w)
# > qr.solve(cbind(1,x) * w, z*w)
# x
# 0.386379 1.106234
对于开始= c(0,0):z
是元素值为2或-2的向量,w
是所有元素均为0.5的向量:
## if start is c(0,0)
start.is.00 <- glm.fit.truncated(x,y,family=binomial(), start=0)
start.is.00
w <- start.is.00$w
z <- start.is.00$z
## if start is c(0,0), the first displayed values are:
qr.solve(cbind(1,x) * w, z*w)
# > qr.solve(cbind(1,x) * w, z*w)
# x
# 0.3177530 0.9097521
所以一切都很好,但是我们如何计算w
和z
?在glm.fit.truncated()
底部附近,我们看到
z <- (eta - offset)[good] + (y - mu)[good]/mu.eta.val[good]
w <- sqrt((weights[good] * mu.eta.val[good]^2)/variance(mu)[good])
查看用于计算z
和w
的数量的输出值之间的以下比较:
cbind(y, start.is.null$mu, start.is.00$mu)
cbind(y, start.is.null$eta, start.is.00$eta)
cbind(start.is.null$var_mu, start.is.00$var_mu)
cbind(start.is.null$mu.eta.val, start.is.00$mu.eta.val)
请注意,start.is.00
的矢量mu
的值仅为0.5,因为eta设置为0,并且mu(eta)= 1 /(1 + exp(-0))= 0.5。 start.is.null
将y = 1的那些设置为mu = 0.75(对应于eta = 1.098612),将y = 0的那些设置为mu = 0.25(对应于eta = -1.098612),因此var_mu
= 0.75 * 0.25 = 0.1875。
但是,有趣的是,我改变了种子并重新运行了所有内容,对于y = 1,mu = 0.75,对于y = 0,mu = 0.25(因此其他数量保持不变)。也就是说,无论w
和z
是什么,start = NULL都会产生相同的y
和x
,因为如果y = 1和eta =则它们初始化eta = 1.098612(mu = 0.75)。如果y = 0,则-1.098612(mu = 0.25)。
因此,似乎没有为start = NULL设置截距系数和X系数的起始值,而是根据y值和x值将初始值赋予eta。从那里计算w
和z
,然后与x
一起发送到qr.solver。
set.seed(123)
x <- rnorm(100)
p <- 1/(1 + exp(-x))
y <- rbinom(100, size = 1, prob = p)
glm.fit.truncated <- function(x, y, weights = rep.int(1, nobs),
start = 0,etastart = NULL, mustart = NULL,
offset = rep.int(0, nobs),
family = binomial(),
control = list(),
intercept = TRUE,
singular.ok = TRUE
){
control <- do.call("glm.control", control)
x <- as.matrix(x)
xnames <- dimnames(x)[[2L]]
ynames <- if(is.matrix(y)) rownames(y) else names(y)
conv <- FALSE
nobs <- NROW(y)
nvars <- ncol(x)
EMPTY <- nvars == 0
## define weights and offset if needed
if (is.null(weights))
weights <- rep.int(1, nobs)
if (is.null(offset))
offset <- rep.int(0, nobs)
## get family functions:
variance <- family$variance
linkinv <- family$linkinv
if (!is.function(variance) || !is.function(linkinv) )
stop("'family' argument seems not to be a valid family object", call. = FALSE)
dev.resids <- family$dev.resids
aic <- family$aic
mu.eta <- family$mu.eta
unless.null <- function(x, if.null) if(is.null(x)) if.null else x
valideta <- unless.null(family$valideta, function(eta) TRUE)
validmu <- unless.null(family$validmu, function(mu) TRUE)
if(is.null(mustart)) {
## calculates mustart and may change y and weights and set n (!)
eval(family$initialize)
} else {
mukeep <- mustart
eval(family$initialize)
mustart <- mukeep
}
if(EMPTY) {
eta <- rep.int(0, nobs) + offset
if (!valideta(eta))
stop("invalid linear predictor values in empty model", call. = FALSE)
mu <- linkinv(eta)
## calculate initial deviance and coefficient
if (!validmu(mu))
stop("invalid fitted means in empty model", call. = FALSE)
dev <- sum(dev.resids(y, mu, weights))
w <- sqrt((weights * mu.eta(eta)^2)/variance(mu))
residuals <- (y - mu)/mu.eta(eta)
good <- rep_len(TRUE, length(residuals))
boundary <- conv <- TRUE
coef <- numeric()
iter <- 0L
} else {
coefold <- NULL
eta <-
if(!is.null(etastart)) etastart
else if(!is.null(start))
if (length(start) != nvars)
stop(gettextf("length of 'start' should equal %d and correspond to initial coefs for %s", nvars, paste(deparse(xnames), collapse=", ")),
domain = NA)
else {
coefold <- start
offset + as.vector(if (NCOL(x) == 1L) x * start else x %*% start)
}
else family$linkfun(mustart)
mu <- linkinv(eta)
if (!(validmu(mu) && valideta(eta)))
stop("cannot find valid starting values: please specify some", call. = FALSE)
## calculate initial deviance and coefficient
devold <- sum(dev.resids(y, mu, weights))
boundary <- conv <- FALSE
##------------- THE Iteratively Reweighting L.S. iteration -----------
for (iter in 1L:control$maxit) {
good <- weights > 0
varmu <- variance(mu)[good]
if (anyNA(varmu))
stop("NAs in V(mu)")
if (any(varmu == 0))
stop("0s in V(mu)")
mu.eta.val <- mu.eta(eta)
if (any(is.na(mu.eta.val[good])))
stop("NAs in d(mu)/d(eta)")
## drop observations for which w will be zero
good <- (weights > 0) & (mu.eta.val != 0)
if (all(!good)) {
conv <- FALSE
warning(gettextf("no observations informative at iteration %d",
iter), domain = NA)
break
}
z <- (eta - offset)[good] + (y - mu)[good]/mu.eta.val[good]
w <- sqrt((weights[good] * mu.eta.val[good]^2)/variance(mu)[good])
# ## call Fortran code via C wrapper
# fit <- .Call(C_Cdqrls, x[good, , drop = FALSE] * w, z * w,
# min(1e-7, control$epsilon/1000), check=FALSE)
#
#print(iter)
#print(z)
#print(w)
}
}
return(list(z=z, w=w, mustart=mustart, etastart=etastart, eta=eta, offset=offset, mu=mu, mu.eta.val=mu.eta.val,
weight=weights, var_mu=variance(mu)))
}