It's been a long time since I last wrote about webruby. 2014 is a really busy year for me, mostly busy in places other than programming :) Anyway, I'm back here, and I'm confident to say that webruby is not dead. In fact, I've recently upgraded webruby to the latest version of mruby with emscripten SDK. This solves a lot of existing bugs in mruby, and will also give us better experience in the better.
Okay, back to the topic of this thread. I will probably write about this upgrades, but let's leave that to a future thread. I've been wanting to write this for a while: someone posted a comment on my webruby tutorial asking:
why not using opal?
I've answered this question once when I was doing Q&A for RubyConf China 2013, but at that time, I know so little about Opal that I didn't even know Opal can do bootstrapping!
I've thought about this question for a while, and here's my conclusion: I think Opal and webruby are 2 quite different way of running Ruby on the web: while Opal translates Ruby directly to JS, webruby compiles the whole mruby VM(written in C) to JS, and runs mruby bytecode directly. Both implementations can parse Ruby code right in the browser now without a question, the targets used are different: JS for Opal, and bytecode for mruby.
I won't go that far to say this is definitely advantage for webruby, since I believe at one time the legendary Mike Pall recommended language creators target Lua directly, not luajit bytecode. But it's worth pointing out this is a difference. Depending on the exact application you are building, one might be more appealing than the other one.
Below is a few points I want to make about webruby, I won't classify them as benefits over Opal, I just like to call them as interesting behaviours of webruby:
Webruby shares the same parsing as mruby, so if there're any parsing changes at mruby side, all we need to do is updating mruby code. There's no need to implement the special parsing code again. Webruby will always(well, the exact answer is mostly, since the underlying C compiler might have different semantics) have the same syntax and semantics as the official mruby implementation.
In webruby, C-interoperability is possible via emscripten. Existing C code can be brought to the web, either due to performance reasons, or legacy reasons. Certain wrapper code is needed for interop between Ruby and C, of course, but it still saves a lot of efforts IMHO.
Portability is an area that I think webruby certainly has an advantage: if you have a mruby codebase already running in other environments such as embedded boards, Android or iOS, porting to the web via webruby might not be a hard task. See this for one example on this, those awesome developers use webruby to build a Web IDE for their mobile payment system. What's also worth mentioning is that by portability, I'm not talking about Ruby alone! For those embedded systems, I consider both Ruby part and C part as a single codebase. Code written in both Ruby and C can be ported to webruby with certain efforts.
It's been quite some time since I wrote my last post. I've been pretty busy with my work these days, I've also been to Japan for my RubyKaigi 2013 talk. But I managed to finish one feature for webruby, of which I've been dreaming for quite a long long time. Ladies and gentleman, from this time, you can install webruby using the following command:
No more boring git commands, it is just that simple, webruby is now a gem!
Well, the sad truth is: it is actually not that simple. We need python2, ruby, node.js for this, but I believe you awesome guys or ladies already have these installed, right? The only tricky part is that LLVM 3.2 is needed to run emscripten, with LLVM 3.3 already released, this can be a little troublesome. What I suggest is that you can go to here, download the binary files for your favourite OS, extract it and define an environment variable LLVM containing the bin folder of the extracted files. This will be enough to let emscripten find LLVM 3.2.
With LLVM and webruby installed, we can use the following command to create a new project:
Webruby will create a folder named MyFirstWebrubyProject with files you need:
In Rakefile, you can do all the customization for webruby: whether you want to add new mrbgem, whether you want to build as release mode or debug mode, or whether you want to change the entrypoint file. By default, the entrypoint will be app/app.rb, mrubymix is used here to solve the difficult require part. I know you guys don't like this, but this is the closest solution I could think of. I will keep an eye on thoughts on the Internet, and if I found a nicer way, I will bring your favourite require back.
Building is as simple as running rake. But keep in mind that if you are running emscripten for the first time on your machine, you may find this:
In most cases, emscripten will figure everything out, and you can simple type rake again to build your project. I only did this to keep you away from wired behaviours like one of you files are not built correctly.
I feel excited that we can finally avoid the annoying git staging areas, what do you guys think?
Happy birthday Ruby! It's 24th in the US now, while it is still 24th in Japan. So it is neither too early nor to late:) It is also a perfect chance(and a happy coincidence, I didn't plan to release on this day, it just happens) to show you some of my latest work. The result looks like this:
From a comparison of the two versions of source code you can see that a lot of things can be achieved purely in the Ruby side:
For new call, instead of new bar(), we need to use bar.invoke_new, since I haven't found a good way to implement new function. Maybe I need to check rspec for inspiration-_-
You can create a variable that caches a function: such as:
However, to invoke this function, you need to use following syntax:
This is now only a temporarily solution, I may find a way to work around this.
Suppose you want to call a set function on an object foo, you cannot directly do a foo.set(1), since set is already used in Ruby side, what you can do here is one of the following solutions:
When adding Ruby Procs as callback functions, you can either directly create a proc:
You can also use existing functions as callbacks, but do remember you need to use the following syntax:
While we still use a symbol to represent a function, to_proc must be invoked to tell webruby that we want to use the actual function, not the onResize string.
Do remember that in current implementation of webruby, you only have the precision of float, not double. So the range that can be expressed in a number is a little limited. I will check later if we can work this around.
And there's one more TODO task: currently you cannot pass arguments to a callback function using a Ruby proc. For example, if you add arguments to window.setInterval, they are not passed back to your Ruby Proc. I will work on this in the next few days(but not today, I have some stupid homework to finish today-_-).
I've been considering a lot about the "require" problem in Webruby. We know that mruby will not have an official require function. While it is not so hard to write one, we still may not be able to get a consistent API, since all the mrbgems are directly adding modules or classes without needing a require. Personally, I think this makes require useless. What's more, when targeting an embedded system, is it possible that we will always have a dynamic library loading mechanism? I just don't have the answer.
However, we still need a way to organize the source code. Nowadays we cannot assume that anyone is interested in putting all the source code in one file! One way of solving this is to use cat to concatenate all the source files just as what we would do when compiling mruby unit tests. However, there's still one problem: you need to maintain the order of concatanation. A simple cat *.rb > rbcode.rb will not solve the problem since the order may not be what you want, just imagine if you have a zlib library named zlib.rb and many other source files depend on it.
So what we really need here is a utility that can:
Combine multiple Ruby source files into one giant file.
Maintain a dependency or an order of different files.
Here I created a gem(Ruby gem, not mrbgem) called mrubymix. It reads a root source file, or an entrypoint file, then parses and includes all the other source files required by the entrypoint. mrubymix will automatically records the dependency of each file, and only include one source file once, even if it is required by many other file.
Here is an example, supports we have the following entrypoint file, named app.rb:
You may notice that the syntax here we use is the same as Rails asset pipeline. This app.rb file has two depencencies: aaa.rb and ./foo/bar.js. Two simple rules are used here:
The suffix .rb can be omitted.
The path of required file is relative to the path of current file
To demostrate how mrubymix resolves multiple dependencies of the same file, suppose aaa.rb contains the following content:
And this is the content of ./foo/bar.rb:
We can run the following command to process app.rb:
The first argument to mrubymix is the path of the entrypoint file, while the second argument is the path of output file. After running this command, the content of out.rb should be:
For each file, mrubymix will first write the full path of the file included as a Ruby comment, then the actual content of the file will be appended. Notice here that all the require lines are removed. What's more, even though the file foo/bar.rb has been required twice, only one is actually inserted, the inserted location is also the earliest one, which is ahead of all depending source files.
The build process of Webruby has also been changed to use this small library. Considering the simplicity of building process and the small size of this gem, I just added as a git submodule instead of requiring this gem to be available on your machine. This may provide another reason of maintaining an independent library instead of using sprockets.
I believe this gem is not only helpful for Webruby. Whenever you are using mruby, you need to solve the code organization problem, and this gem can provide an alternative to the regular require way:)
To enable this optimization, instead of running rake, you can simple run the following command:
My test shows that the file size has been further reduced to only 401k!
Notice that now the linking time is longer than the normal case, so I suggest you only use optimization mode only when releasing your code, and continue with the non-optimization option when developing.
There's also a small API changes from what I described in the tutorial. Now the HTMl skeleton for invoking webruby looks like following:
Now WEBRUBY serves as a function instead of an object. With each run of WEBRUBY(), you can have a separate webruby instance for running Ruby code. The mruby state, mrb, is now encapsulated in the returned object of WEBRUBY(). You do not need to specify it manually now.
Personally, I think this new API is a little simpler. However, if you have other opinions, please feel free to leave a comment here:)
Update: I've changed the API a little bit. Please check this post for the latest API. I decided to keep this post unchanged so you can see both versions. Feel free to leave a comment if you belive the old one is better:)
Lately some people have been asking me to write a tutorial on how to use Webruby. Sorry, guys. I was having a lot of fun writing this stuff but I forgot to bring these fun to you. Now the tutorial on Webruby is finally here:)
The final result of this tutorial looks like this:
First question: where do I put my Ruby source code?
Before everything starts, we need a place to keep and load our Ruby source code. Webruby now supports 3 kinds of source code loading methods:
All the source code in app folder of Webruby will be compiled and attached in the mruby.js file automatically. Then we can use WEBRUBY#run to load this part of source code.
And of course, you can use WEBRUBY#run_source to parse and execute Ruby code directly.
Webruby allows you to specify the loading modes you will use when compiling, this can help reduce the size of mruby.js file: if you do not need to parse Ruby source code on the fly, you wouldn't need all the parsing code in the generated mruby.js file, and if you only execute source code from app folder, modern optimizers may take advantage of that to further reduce the file size. Please refer to rakelib/functions.rb for how to specify supported loading modes. In this tutorial, we will use the default loading modes, which will support all 3 kinds of loading methods. And we will show how to load Ruby source code using WEBRUBY#run and WEBRUBY#run_source. The second method describes above is a little bit of complicated(since mruby has multiple versions of bytecode), I will describe it in another post(maybe with specialized bytecode generation tools) later.
Okay, I got that. But how to run this stuff exactly?
Now we've got all the backgrounds needed, let's walk through the developing process step by step.
And do remember one thing: webruby uses emscripten, and emscripten uses LLVM internally. So you may also need to install LLVM:
Note this is the Mac-with-homebrew way of installing LLVM. If you know how to install LLVM 3.2 on Linux/Windows, I will really appreciate it if you can comment below, I will update this post accordingly.
Update: Thanks to Reed for pointing out, actually you can just go to http://llvm.org/releases/download.html, download and extract the pre-compiled binary for your platform, change the value of LLVM_ROOT in your ~/.emscripten file to match the bin directory of your LLVM installation. You are then good to go!
This is almost it! I guess I may assume that most of you have Ruby and node.js installed already. But if you are not so convinced, you can run the mruby unit tests in webruby environment:
If you can see this, everything is fine:)
The latest mruby supports using mrbgem directly from a git repository, so all you need to do here is uncomment Line #32 in build_config.rb. And make it look like this:
Notice that if you have run mrbtest before uncommenting this line, you need to run rake clean to cleanup everything and do a full rebuild. This is due to that the gem settings have changed.
Writing Webruby code
Now we can actually start writing Ruby code. First we will try the first loading method: all the source code in app folder of webruby will be compiled and then attached in the mruby.js file. So feel free to add or change any file in the folder, but please do remember to add .rb suffix.
A special file named app.rb will be compiled at last, it serves as the entrypoint(or so-called "main" file). For our simplest demo, we will just code in this file. Now you can bring out your favourite code editor and change the content of this file to:
JsObject also comes with call_constructor function for making a new call, and get for getting a field value from an object. Note that mruby-js is still under development, function and array passing support is still lacking, and the APIs are subject to change. I will try my best to keep here updated, but please always use the source code as the answer if you find a disagreement.
Now you can compile webruby using rake, if everything works well(it should), you can find a mruby.js file in build folder.
Node.js is great for debugging, but I believe most of you want to use webruby in a browser enviroment. We also need an HTML skeleton for loading our generated JS file.
Here's a sample skeleton: note that this only serves as a sample, and I believe all of you can write a much better skeleton within seconds. But for now, please bear with this dumb one:
Put this html file in the same folder with mruby.js file and open it with a modern browser, you can see an initial result:)
Loading Ruby source code on the fly
By changing the value of environment variable LOADING_MODE, you can actual customize the loading methods you want to support. For example, the following line can be used to only allow for running attached Ruby code compiled from app folder:
If you have been following this post, then your mruby.js must have already contained parsing code. Otherwise you may want to use following command to rebuild the JS file:
Now it looks just like what is shown in the demo. You can also use Closure Compiler to strip the size of generated JS file if you like, but please keep in mind that only simple optimization works for now, the advanced option still needs a little tweaking. On my machine, with a simple optimization the size of the JS file can be reduced from 4.2MB to 1.5MB.
It really feels nice to finally have something for everyone to use:) I've had a lot of fun and I will be continue maintaining this. Of course this is not fit for everyone's project, even after optimization there's a 1.5MB JS file to load. For now I guess the most suitable use case will be Web games or single page apps. This is why I'm also working on an OpenGL ES 2.0 binding. Anyway, I wish all of you had fun playing with webruby, just like I had fun developing this:)
Merry Christmas everyone! Honestly Christmas is not so important for me but it may be very important for you guys:) Also wish everyone of you a Happy New Year!
Maybe my original plan on an OpenGL ES 2.0 API is right-_- Anyway, I will update here if I have more results.
Update: Well, I manage to succeed in building an interface using neither of the methods above! I just pass the pointer(array) of values to the JS side, and then let the JS code call "back" to the C code to process the arguments one-by-one. The cost should not be more than simply looping through the arguments. The internal structure of mruby calling stack helps a lot in this implementation! If you are interested, feel free to check out the code, specifically this commit.