ADVI can write 1.#INF on Windows, which cannot be read

[bgoodri]

Summary:

It is possible for ADVI to obtain a realization that is infinite, and if it writes that to a CSV file, reading the CSV file breaks. We should coerce this to the nearest floating point number before writing.

Description:

CSV files should only have finite numerical values below the headings

Reproducible Steps:

In R 3.2.x for Windows, load rstanarm 2.15.x and do

fit <- stan_glm(mpg ~ ., data = mtcars, prior = hs_plus(4, 1, 2, 0.5), 
                       seed = 12345 algorithm = "meanfield", QR = TRUE)

Current Output:

scan() expected 'a real', got '1.#INF'

Expected Output:

Finite estimates

Additional Information:

Provide any additional information here.

Current Version:

v2.15.0

[bob-carpenter]

Is there something in this (stan-dev/stan) repo that should be fixed?

Would it be OK if that fix allowed infinite and NaN values to be read?

I added the next release as milestone and assigned to @mitzimorris but I'm still not sure what needs to be fixed (if anything) in this repo.

[syclik]

Hmm. This is in the writers. Currently, we use stringstream's operator<< to take a double and get it out to a string. The issue here is that Windows prints infinity as #1.INF and NaN as #1.NAN while Linux and Mac does something differently. Some reasonable options to fix this: 1. In Stan, update the writers so that it prints something not OS dependent. That's this file: src/stan/callbacks/stream_writer.hpp 2. Update RStan's writer to write something different when it sees NaN or Inf. We'd want to do the same for CmdStan and PyStan, I think.

…

On Wed, Apr 19, 2017 at 4:11 PM, Bob Carpenter ***@***.***> wrote: Is there something in this (stan-dev/stan) repo that should be fixed? Would it be OK if that fix allowed infinite and NaN values to be read? I added the next release as milestone and assigned to @mitzimorris <https://github.com/mitzimorris> but I'm still not sure what needs to be fixed (if anything) in this repo. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2283 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAZ_F5ZAUqUroVQUw04fq2cmtFc0IHgNks5rxmqAgaJpZM4NBBmK> .

[syclik]

@bgoodri, thoughts? Can we either close this or list what needs to be done?

[bgoodri]

We should fix ADVI in Stan so that it does not report values greater than the largest floating point number.

…

On Tue, May 9, 2017 at 5:22 PM, Daniel Lee ***@***.***> wrote: @bgoodri <https://github.com/bgoodri>, thoughts? Can we either close this or list what needs to be done? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2283 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADOrqjiq_rFELMQyRPjbTCwEuqqiTGEbks5r4NkJgaJpZM4NBBmK> .

[syclik]

That seems reasonable. Can you either update this issue to reflect that our close this and open a new one with that suggestion?

…

On May 9, 2017, at 5:30 PM, bgoodri ***@***.***> wrote: We should fix ADVI in Stan so that it does not report values greater than the largest floating point number. On Tue, May 9, 2017 at 5:22 PM, Daniel Lee ***@***.***> wrote: > @bgoodri <https://github.com/bgoodri>, thoughts? Can we either close this > or list what needs to be done? > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#2283 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/ADOrqjiq_rFELMQyRPjbTCwEuqqiTGEbks5r4NkJgaJpZM4NBBmK> > . > — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[bob-carpenter]

@bgoodri: is your proposal todo censoring and report the max double instead of +infinity? Is there no way to robustly read NaN or infinite values into R? Because I'd think the right fix would be fixing our writers to match whatever that format is.

[bgoodri]

Yes, and not that I know of.

…

On Wed, May 10, 2017 at 2:50 PM, Bob Carpenter ***@***.***> wrote: @bgoodri: is your proposal todo censoring and report the max double instead of +infinity? Is there no way to robustly read NaN or infinite values into R? Because I'd think the right fix would be fixing our writers to match whatever that format is. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2283 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADOrqj5zAPVfrv_SRJrQnEepBKptH3wEks5r4gbpgaJpZM4NBBmK> .

[jgabry]

I'm not aware of a super robust method for reading them into R either, unfortunately.

…

On Wed, May 10, 2017 at 5:38 PM bgoodri ***@***.***> wrote: Yes, and not that I know of. On Wed, May 10, 2017 at 2:50 PM, Bob Carpenter ***@***.***> wrote: > @bgoodri: is your proposal todo censoring and report the max double > instead of +infinity? > > Is there no way to robustly read NaN or infinite values into R? Because > I'd think the right fix would be fixing our writers to match whatever that > format is. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#2283 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/ADOrqj5zAPVfrv_SRJrQnEepBKptH3wEks5r4gbpgaJpZM4NBBmK > > . > — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2283 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AHb4QzrHgQYnoyx0Qut3lcXJcP6t8pOyks5r4i5SgaJpZM4NBBmK> .

[mitzimorris]

what is the current proposal? initial proposal was this:

In Stan, update the writers so that it prints something not OS dependent. That's this file: src/stan/callbacks/stream_writer.hpp

unassigned myself per subsequent discussion, but happy to do this if that's what we want.

[ahartikainen]

Are these not the common/preferred ways to write&read infinities / nans to and from CSV?

They should be platform independent. I mean reading with common R/Python(Pandas) tools.

inf
-inf
nan or " "