I want to build apps in Python and C and ship them in the browser. I can't.
I ran full steam into this problem when I built Dirpy, a web app that records MP3s from YouTube. I could have built a native app in Python and C, but to test, distribute, and maintain builds for every OS is a total nightmare. So I built Dirpy as a web app, did all the work on servers, and paid $10,000 every month in hosting costs.
This was painful, expensive, and inefficient. Clients are more than capable of transcoding video; the problem is browsers aren't. Browsers can't run Linux software like ffmpeg, can't run Python, can't reach native performance, and can't make cross-domain requests.
Why can't I distribute native apps like web apps that are cross-platform, always up-to-date, securely sandboxed, and don't need to be installed? Why can't I have the freedom to build such apps with the languages, libraries, and network protocols of my choice? It doesn't matter if I build for the web or build for native, I still have to settle for software that falls short of its potential.
I'm tired of settling. So I built Arc.
Arc
Arc gives websites a Linux virtual machine on every client.
It runs native software in independent, lightweight virtual machines and is accessible from websites with arc.js, a Javascript library that gives browsers access to Arc's API.
Arc apps have full access to the facilities of their Linux virtual machine. They can spawn threads and processes, read and write to an internal filesystem, and make full use of their virtual machine's graphics, audio, and network adapters. Cross-platform differences are virtualized away -- Arc apps behave identically on OS X, Linux, and Windows. And because Arc apps are securely isolated in their virtual machines, they can't corrupt the host, either through malice or accident.
Who is Arc for?
Arc is for developers who want both the power and flexibility of native apps and the distribution of web apps.
Developers can save thousands of dollars in hosting costs by packaging services in Arc apps and pushing them to clients. It's easy to migrate server work to clients because Arc apps are just Linux VMs; existing Linux binaries work as-is.
Web developers can augment existing web apps with native code in three lines. Opening an Arc app with arc.js to securely run C, Python, Node.js, PHP, Go, Ruby, Lua, or anything else on the client is as easy as
arc.openApp('http://peggo.co/app/ArcManifest.yaml', function(app) { // App opened successfully. Accomplish wonders. });
Peggo
To build a better YouTube recorder, dogfood Arc and arc.js, and put my money where my mouth is, I built Peggo.
Peggo is a web app with an Arc engine. The frontend is a website, HTML styled with CSS and a smidge of controlling Javascript, but the magic is the backend: an Arc app written in Python that does all the transcoding securely on clients.
For a sense of Arc's potency, Peggo is 1,000 lines of Python compared to Dirpy's 15,000. Many of those 15,000 lines were to scale Dirpy across two dozen servers in three datacenters -- scaling not required with Arc. Instead of scaling Peggo, I just built a little app that ran on my development machine and dropped it in Arc. Now it runs everywhere.
How does Arc work?
Arc is architecturally similar to a headless browser. Arc apps are defined by a manifest (think index.html) and opened from a URL, just like a web page. And just like a browser retrieves CSS and Javascript files to assemble a web page, Arc retrieves files used by Arc apps to assemble Linux virtual machines.
An Arc app's manifest is comprised of four simple things
- Metadata, like the app's name and icon.
- Ports to expose.
- Files to retrieve.
- Commands to run.
When an Arc app is opened, Arc
- Downloads the app's manifest.
- Creates a new virtual machine for that app.
- Retrieves all files specified in the manifest and places them in the virtual machine's filesystem.
- Runs all shell commands specified in the manifest.
When the last shell command completes, the app is ready to use.
Performance
Arc uses VirtualBox, so runtime performance is near-native.
Startup performance is also good. On my MacBook Air, Peggo's Arc app is created and assembled from scratch in 1.3 seconds. This 1.3 second startup is further hidden with arc.js, which loads the Arc app in background when any peggo.co page is opened, even if the app isn't needed immediately. The Arc app is then preserved and shared among all peggo.co tabs.
Current state
Arc works. It's in beta, so there are burrs, but it works. People use Peggo every day.
Arc is only packaged for OS X right now. It will be packaged and available for Ubuntu and Windows soon. The installer is a heavy 140MB, of which VirtualBox makes up 100MB, and will be slimmed down. (Update: OS X installer is now 60MB).
I've built lean versions of Python, Node.js, and Ruby for use in Arc apps. Of course any language works as-is, it's just probably bigger than it needs to be. Language runtimes in Arc apps don't need docs, tests, build tools, etc.
Documentation for Arc and arc.js is forthcoming. (Update: http://arc.io).
The opportunity
As developers, we've built amazing native apps and amazing web apps, but with Arc we can build something even more amazing: apps that are the best of both.
Arc is an opportunity to unshackle the latent power in clients -- the power we need to continue the transition from native to web -- and retain the web's distribution model.
Your input is helpful as Arc advances. What will you build with Arc? Let me know.