Re: [ANN] Dynamic object interfaces in RADOS with Lua

[Noah Watkins <jayhawk@...>]

On Tue, Oct 29, 2013 at 12:21 PM, Javier Guerra Giraldez <javier@guerrag.com> wrote:

> Currently the scripts are sent along with each request, so the scope is a
> client and there is no persistence. I think this is going to be the focus of
> our next steps, namely to use a single VM instance (rather than
> one-per-request right now), as well as persisting the scripts, although I'm
> not sure what the scoping will look like in that case.

i guess that should be considered in context of the current C/C++
plugin deployment.  maybe the Lua plugin should have some 'startup
script(s)' in a predefined place.

Yes, providing persistence is particularly complex. As you mentioned, adding OSDs is problematic as ideally these scripts will be pre-loaded as part of the start-up process, and clients only have to 'register' their scripts once. Additional complexity arises as users modify scripts--now multiple versions may exist as propogation cannot be instant. Luckily, information with similar requirements in Ceph are already handled. It's just a matter of building a specialized version of these services.

 

about the VM per-request, remember that setting/destroying a VM can
take a significant time, so it's very convenient to keep it running.
of course, that puts a lot of responsibility on the script author,
since any undesired side-effect would also persist in the VM.

It should be relatively trivial to support configuration of the level of sandboxing (separate vm, single vm, ffi, etc...). For many users, having no safety is OK as scripts are by design only submitted by trusted applications.

 

another consideration is about concurrency.  a single VM could
represent a scarce resource, maybe a pool of them would be better.

I need to take a closer look at some of the concurrency assumptions at the level of the Lua VM in Ceph. I think these requests may be serialized, so concurrency may not be a problem.