Always evaluate the context file in a separate process

[JamesWrigley]

The GUI used to have a special case for databases using the default environment where it would evaluate the context file in the same process. This is bad because then the context file might not be evaluated in the database directory, which would break local imports. Now we always evaluate it with get_context_file() in a separate process in the database directory.

(CC @turkot)

[JamesWrigley]

sigh the tests passed locally for me 🤦 I'll look into it tomorrow.

[takluyver]

LGTM, thanks!

Do we want to also remove the context_python is None in get_context_file, so that always uses a subprocess as well? I think it's becoming normal to use a separate Python for the context file, so the extra code for a fast path is less useful than it was.

[JamesWrigley]

(just checking what CI says after a rebase, there's a lingering segfault so it shouldn't be merged yet)

[JamesWrigley]

@tmichela, this is another place the context file is run outside of the sandbox. What additionally complicates matters is that the sandbox arguments are stored in the listener settings rather than the DAMNIT database settings so we can't even get the sandbox arguments... This also made me realize a glaring omission in the sandboxing: it isn't used at all when reprocessing manually 🙃

The reason for not storing the sandbox arguments in the DAMNIT database is because then anyone would be able to change them. But we somehow need to allow retrieving the sandbox arguments from the listener (a trusted source).

[tmichela]

wouldn't it be simpler to handle reprocessing as other triggers? i.e. we send a request to the central listener?

[JamesWrigley]

In some ways yes, but not having to go through the listener is very convenient for testing locally, and if we need the sandbox args anyway for the GUI then we might as well use it for reprocessing too.