Using Replicate with Heroku #22
Conversation
|
I'd very much be interested in seeing a patch. If its not too invasive and is fairly small LOC-wise I'd be happy to merge it. |
Heroku can't handle non-UTF-8 encoded data being output from a 'heroku run' command. This change allows replicate to dump and load Base64 encoded objects so Heroku can handle it. Heroku also sends stderr output to stdout so the loading code attempts to be smart about removing non-base64 lines from the input before loading it.
|
I tried to stay out of the internals of replicate as much as possible. The new code just gets in between stdin and the Marshal code when the user passes in certain options. If you want anything done differently or anything let me know. |
|
I just learned that Heroku silently truncates the output of 'heroku run' jobs if they output a lot of data so this solution might be not be the way to go. I'm going to see what Heroku has to say about it and I'll get back to you. |
|
Heroku is at least aware of this bug now (heroku/legacy-cli#674) but doesn't seem to have a quick fix. Until that is fixed, this PR isn't worth merging in. |
|
Thanks @ejfinneran. Let me know when something materializes. |
2 participants
Just wanted to let people know about a workaround I found for working with Replicate and Heroku.
Basically Heroku only wants to pass UTF-8 friendly characters back to the client so you have to base64 encode the replicate output in order to keep Heroku happy.
Full details here:
http://finneran.us/blog/2012/12/17/using-replicate-gem-with-heroku/
I also figured I'd ask if there is any interest in including something like this in the gem itself to make it more Heroku friendly. I'd be happy to hack on that if there is.