2
1

I'm trying to find (alpha, beta) via maximum likelihood for a beta binomial model. Unfortunately, I am limited to Java for this project - does anyone know any high quality optimization routines for this? It would be ideal if someone here has had good luck with a certain package with the same (or similar) likelihood function, but any recommendations would be helpful. I've tried the Apache Commons Nelder Mead, but it is not very good (far worse than the implementation of same in R).

On a related note, has anyone ever noticed that this method overestimates the mean? I'm seeing this phenomenon and am having a hard time explaining it.

asked Jul 14 '10 at 00:28

george%20s's gravatar image

george s
517810


3 Answers:

Why not use the method of moments estimates? They are more stable in small samples anyways.

(cut and pasted from my R code, should be easy enough to port if you choose to go this way:

xb <- mean(na.omit(x))
s2 <- var(x)
a <- xb * (((xb*(1-xb))/s2)-1)
b <- (1-xb) * (((xb*(1-xb))/s2)-1)

If you must use the MLE, look into Simas (2010) for bias correction approaches to beta MLEs.

answered Jul 14 '10 at 01:50

Tim's gravatar image

Tim
12

edited Jul 14 '10 at 01:54

These estimates are for the beta distribution - the MOM for the beta binomial is more complex and less stable.

(Jul 14 '10 at 10:15) george s

What's your exact setup? Do you have n trials and for each you draw theta_i from Beta(alpha,beta) and have observed X_i? And you want to find point alpha, beta to maximize marginal likelihood of the X_is? i.e.

L(alpha,beta) = P(X_1:n | alpha, beta) = int_{theta_{1:n}} P(X_i | theta_i) P(theta_i | alpha,beta) d theta_{1:n}?

I believe what your looking for is described in a Kevin Murphy paper: Empirical Bayes for Beta-Binomial model (2007). If you have some variant of this problem (like a shared theta amongst trials), its not too difficult to get there from this paper?

Update: Actually he only does a method-of-moments estimate, I'm pretty sure its possible to numerically optimize the empirical bayes likelihood, I'll think about it.

answered Jul 14 '10 at 10:50

aria42's gravatar image

aria42
209972441

edited Jul 14 '10 at 10:56

1

Unfortunately, I am not observing X_i directly - I have to estimate it separately. This estimate is not an integer - I am wondering whether this has anything to do with the bias? I don't think it is theoretically incorrect - pseudo likelihood functions are fairly common in practice, I think.

(Jul 14 '10 at 16:52) george s

Can you be more specific about exactly what you are and aren't observing? Its a little difficult to help without some of those specifics.

If you don't observe X_i, what are you observing? You must be observing something related to the X_i otherwise your problem would be trivial.

(Jul 14 '10 at 18:20) aria42

Right. If I flip a coin, I don't observe whether it is heads or tails - all I know is that one of a small number of coins that were flipped was heads. I have a method to attribute heads to coins, so that coin 1 might receive 0.25 heads, coin 2 0.6 heads, and coin 3 0.15 heads.
So, for each coin, I know exactly how many times it was flipped, along with the sum of the "heads attributions."

(Jul 14 '10 at 19:26) george s

You only have two parameters to estimate and you can compute and store the gradient and full Hessian. So you can simply use Newton's method. You'll need a numerical library for evaluating the Gamma function and its derivatives. The GNU Scientific Library has everything you need. Its relatively easy to call it from Java via JNI.

answered Jul 16 '10 at 16:52

Glorblflx's gravatar image

Glorblflx
12124

Your answer
toggle preview

powered by OSQA

User submitted content is under Creative Commons: Attribution - Share Alike; Other things copyright (C) 2010, MetaOptimize LLC.