...
This workaround it still constrained by the configurable system limitlimits, and it makes the action code more complex that than it needs to be.
...
Measuring the impact
The problem described above results in:
...
Measuring developer impact
Questions:
- Maintenance overhead
- Implementation time
- Additional code complexity
- How much is the maintenance overhead ?
- How is the implementation time affected ?
- How much complexity is added to the action ?
Proposal
The proposal is to transparently provide developers with a temporary way to store large assets in the OW cluster. This is probably the hardest problem to solve when compared to the other ones b/c it involves persistence, state, and possibly handling large amounts of data. Bellow are listed a few possible options:
Namespace volumes
(credits: Cosmin Stanciu for suggesting this simpler solution)
Namespaces can be provided with their own volume, limited in size, i.e 50GB. The programming model should handle this volume as short-lived. Its purpose is to make it efficient for sequences and compositions to access assets faster and easier. The volume is independent of an activation lifecycle. Developers have full control over it, including writing to it, but also cleaning it up.
OpenWhisk's implementation could configure a network file system like NFS, (i.e. AWS EFS), GlusterFS. This volume is mounted on each Invoker host. Each namespace has its own subfolder on that volume, and it can be mounted appropriately into a known path in the action containers that want to make use of it. To simplify the stem-cells management, each action requiring access to such volume may ignore stem-cells; instead, they may go through a cold-start, as if they're blackbox containers.
The diagram bellow shows 2 Invoker Hosts that mount a network file system on /mnt/actions
on the host. Action containers mount on /tmp
the folder from the host corresponding for the namespace /mnt/actions/<NAMESPACE>
draw.io Diagram | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
Action volumes
Allow developers to "attach" a persistent disk to an action. The programming model in this case assumes there's always a folder available on the disk on a well known path defined by the developer. OpenWhisk's implementation could leverage solutions particular to a deployment:
...