DairyBox Next

[mikekasprzak]

Well it's been a few years and much of the software we using is out-of-date or unsupported (see #69).

This issue collects some notes and thoughts on what to do next, and what we need.

Post Scotch/Box

In the absence of a replacement for Scotch/Box, we could make a new setup script that starts from a stock Ubuntu vagrant box, and installs/configures everything as we need it. The downside is setup time would be much longer, but frankly we've added so much to Scotch/Box anyway that it might not be that much worse.

Vagrant works best with images that don't need to do much after they've been instanced. That's the opposite of what we've done.

It would be nice if DairyBox only contained Jammer Core setup specifics, and didn't need all the hacked-in software updates.

JuiceBox vs DairyBoxSource

We could create a monolithic script that not only does the system setup, but all the Jammer Core setup as well. This would solve a lot of our problems, and potentially make upgrading to newer Ubuntu versions far simpler (just change the base-box). The downside is every fresh build would be slow, and it would be proprietary. I call this the "DairyBoxSource" approach.

Alternatively we could create a more work for ourselves, isolate the system setup from the Jammer Core setup, and make a general purpose box. The box would be driven by our needs, but anyone could use it for their own projects (even us). Scotch.io did us a huge favour creating and sharing Scotch/Box, and it would be cool if we did the same.

I've dabbled with this in the past, taking Scotch/Box and upgrading it. I also intended to do this with the original JuiceBox, but in practice Scotch/Box was good enough. I still like the name JuiceBox, so I'll call this the "JuiceBox" approach. 😋

I'm conflicted. I think the JuiceBox approach is better, but it would be much easier to get the DairyBoxSource approach working. That said, gutting DairyBoxSource and splitting it into JuiceBox and DairyBox later could also work. It would be nice to just skip the DairyBoxSource approach, but deadlines are looming.

Needs

Here's a rough list of what the new "box" needs.

Server Needs

• Latest Ubuntu (20.04 today, 22.04 after April)
• Nginx (instead of Apache)
• PHP 8.x with APCu
• Redis (NOTE: only used as a better PHP session cache)
• MariaDB
• ImageMagick, FFMPEG, and other image optimizers

Tooling needs

• GNU Make
• Nodejs
• JavaScript linter and other build tools
• Less and other CSS build tools
• SVG build tools
• Typescript tools (future use)
• Rustup and Cargo (future use)
• [ ] Sass (future use)

Other needs

• Something for database hacking (PHPMyAdmin?)
• Something for outgoing email capturing and review
• Something capable of emulating Amazon S3 object storage (CEPH?)
• Search database (Sphinx?)
• Something to make PHP easier to debug

[mikekasprzak]

On Vagrant boxes: If I recall, creating a Vagrant box isn't actually that difficult, but there are some gotchas.

An Ubuntu installation becomes quite fat once you do something with it. I believe Scotch/Box was only a few hundred MB, but my first boxes were several GB (I have a much faster Internet connection today, so that might be less of a concern). I later found out there are recommendations for "files that can be deleted" from an Ubuntu installation, which greatly helps with file size.

I can't remember the specifics, but I remember there is something really tricky about getting the networking right. Something to do with fixed MAC addresses or interfaces. The problem is that you couldn't safely run 2 instances of the same box without properly changing certain setup files before imaging, and you need to remember to do this every time before you image.

I think there might be something about hosting the boxes, but I can't remember the specifics. I believe Hashicorp runs the servers that host free boxes, and it might have had something to do with them.

Last time I touched this was nearly 6 years ago, so it's possible these are non issues today. Either way there is some research ahead to be sure boxes are being packaged and pushed correctly.

[zwrawr]

DareBox

DareBox (dárɛ box) is a pre provisioned LAMP vagrant box in the vein of scotchbox, but up to date.
The name isn't great

This gets Dairybox running on ubuntu 20.04 and replaces Scotch Box. This repo is purely a test and doesn't meet the objectives set out above. The goal is to gain an understanding of how to build, provision, package and upload a Box. Currently, this can be done by changing the version number and running a single command ./darebox.sh build upload. It gets the website more or less running on ubuntu 20.04. The simulator (with my fixes PR) runs and generates test data. MailHog works, PHPMyAdmin works (but has some errors). I haven't tested everything yet, but it's probably not worth doing that till we have decided on a final approach.

So what you would need to look at this are these three branches.

• https://github.com/zwrawr/DareBox
• https://github.com/zwrawr/dairybox/tree/darebox (mainly removing our hacky upgrades to scotch box)
• https://github.com/zwrawr/ludumdare/tree/darebox (mainly changing the db name to darebox, catching an error with loading globals when they don't exist in db-create.

On the points above

• ScotchBox 3.5 is 1.2GB DareBox v0.0.8 is 1.68GB. There may be some more cleanup we can do but the reality is that newer Ubuntu releases are bigger than old ones. This is why ScotchBox went from 800MB to 1.2GB between 3.0 and 3.5. There are some cleanup steps I have turned off because they were uninstalling the software I just installed. We might be able to reduce this a bit.
• I believe the MAC address problem is solved by the cleanup scripts I am running
• This is more of the JuiceBox style
• app.vagrantup.com seems to allow free hosting of publicly available boxes
• Switching to php8.x would actually simplify the scripts a lot since it is the default in ubuntu 20.
• I'm using Hashicorps packer as well as vagrant. I don't think this is really what it is meant for. It does help in some ways but I might try not using it and just use the darebox.sh script.
• Installing other software should be easy
• Switching to Nginx shouldn't be too bad

Now I have an idea of how to create a base box it shouldn't be too hard to make something that meets a specific set of requirements.

https://app.vagrantup.com/zwrawr/boxes/DareBox

[winniehell]

Don't want to criticize your approach here, just out of curiosity: What's the main difference/advantage of a Vagrant box in that context over a docker-compose file? Does it make development on a Windows machine easier?

[mikekasprzak]

It did. Docker was still new in 2015-2016, and so was WSL. I don't think Docker worked on Windows back then, at least not as a Linux host on Windows. I'm not 100% sure of the specifics, but I also recall there some trouble with Docker until WSL2, which I think is Windows 10+. Vagrant serves a similar purpose as Dockers, but it manages VM's, not containers. Containers run on the native machine, where as VM's are fully virtualized. Before WSL this was the only option for Windows users. DairyBox is a legacy monolithic toolchain that roughly matches how the severs are deployed today, though it's across two machines instead of one. The JammerCore roadmap includes splitting the project up into its distinct parts (API, SDK, App, Templates, and Home). It's highly likely we will switch to a containerized configuration then, and update the tooling accordingly. It's highly likely there will be a Vagrant box as well, to keep setup simple, or isolated from your machine.

…

On Wed, Mar 30, 2022, 16:25 Winnie ***@***.***> wrote: Don't want to criticize your approach here, just out of curiosity: What's the main difference/advantage of a Vagrant box in that context over a docker-compose file? Does it make development on a Windows machine easier? — Reply to this email directly, view it on GitHub <#71 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAL7MCPKW5G4LW47WEZUVITVCS2CRANCNFSM5N6XM4VA> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[mikekasprzak]

So what you would need to look at this are these three branches.

@zwrawr Will do, and thank you for digging into this! I'll fork and send a PR once things calm down here.

FYI: I'm taking the rest of the month off after LD50 ends (April 21st). In early May, I'll get back to rebuilding the servers. Once that's done, we can start accepting PHP 8.x changes.