It might make sense to put some compute intensive tasks in Go, or any other compiled language thats more efficient, but exec'ing and returning JSON isnt really a great idea.
If you were going to consider this seriously, then you should have the target app start once, listen for comands on some channel like a socket, and have it return information. JSON is probably a little too verbose for this kind of use-case as well, but it depends how efficient Ruby would be at unpacking alternative serialization formats.
> It might make sense to put some compute intensive tasks in Go, or any other compiled language thats more efficient, but exec'ing and returning JSON isnt really a great idea.
It's not a terrible one, though. You don't choose Ruby for its performance. If you find yourself creating and returning lots of JSON with this, that should be a clue that you should be pushing more functionality down to that lower level, and just returning the bulk results back up.
Using a library like Oj[1] it is vastly faster to unpack JSON in Ruby than it is even to use the built in marshaling. It's extremely portable, performs well and is easy to debug. There really aren't many practical drawbacks to using it for transport on the same system.
With regards to Go, it is currently not possible as the Go compiler only supports building static binary executables. For it to work with extconf, Go would need the ability to build dynamic libraries.
There has been talk of incorporating this ability and slim executables go (I think it might even be in the roadmap now).
This is true, but it's a point worth making every so often too. I created a system with Erlang and Perl that punched well above its weight (in terms of resources allocated to it) because I was able to cleanly separate the "concurrency" into the Erlang and the "reusing tons of existing code and working in a language the devs are familiar with" into the Perl. It's a powerful pattern, and if you are, like I was, stuck in a language with poor speed and in the case of perl no significant concurrency story [1], it's important to hear that there are options other than "throw it all out and start over again in a Cooler Language (TM)".
[1]: Perl has plenty of event loop choices, but in terms of trying to maintain thousands of live SSL connections at scale it's a terrible, terrible choice for that connection manager. But it was just fine as a service provider.
While this is true, the strategies for each are different. For example, the ways you include Rust or C into a Ruby program can be much more varied than the others in your list. This article, for example, chooses a 'shell out' option. While you can still do that with C or Rust, you can also compile them as an extension directly to your Ruby. There's upsides and downsides to both approaches.
If you want to extract parts of your ruby app to go in a simple way, may I suggest a look at http://www.goworker.org instead of following this articles advice.
I don't really know why this is an article. So, uh, the author has discovered that you can fork/exec a process? I'm not sure what is novel or interesting that is presented here.
This article could have been written in 1998, titled "Using C to improve your Perl application's performance".
I'll say this about this strategy. For most web apps (and ruby is mostly used in webapps), it's usually more robust to just write a go background worker listening over RabbitMQ or the like and just shove messages around as a service.
While I can sympathize with your critique, I wish it were a bit more elaborate so that the author has an idea about what should have been done better. What is bad about that? What are the alternative solutions, etc.
More than just the escaping. You should also clean your environment, providing the child process with a bare minimum of information it needs to do its job.
Otherwise you leave yourself open to future security issues, especially if you keep API keys or DB passwords in ENV.
Also, it's generally a good idea to write code where the original developer's intent can be easily figured out by looking at the code. If the code is too compact, does too much magic (or it's too verbose, which is the other extreme) it can be difficult to tell what was intentional, how it was meant to work, if something is a bug or not. The code should "tell a story", so to speak.
Grabbing the data from the exec response is a super clunky way of communicating between two applications, particularly since Go is really good at keeping long-lived TCP connections.
Even without connecting directly (which is just as trivial as reading a response from exec), you should be looking at a formal message passing system. This also allows you to easily distribute these dissonant applications, which is a good idea when you're talking about "utilizing all the CPUs available" and potentially high network usage.
Well it's a microservice as described in the OP, I'm just talking about formalizing a communication method. Reading CLI responses isn't a very smart way to do it. REST APIs aren't really ideal for slow processes, either, which is why my first thought would be something like RabbitMQ or just a simple TCP connection.
So instead of using a unix domain socket and amortizing the cost of startup he forks a process every time. This is ok for prototyping but definitely not ok for any kind of production use.
Anyone wanting to do this at scale should look at posix_spawn to avoid forking the process (ruby gem here: https://github.com/rtomayko/posix-spawn). Also probably ruby's ShellWords for escaping.
I guess that is a simple way to delegate heavy processing tasks to Go. Communication through stdout in JSON is an easy way to communicate between Ruby and anything else faster than Ruby (C, Rust, Nim, Go, etc.) in specific tasks.
I agree there are some pretty big issues with this tutorial. However it's still a pretty good article the "ideas" here are useful, I started thinking about how I could use Go in my ruby stack.
How so? The idea is not novel, in fact he links to a proper implementation (having a Go process listen on a UNIX domain socket) and the only contribution is to introduce a huge security vulnerability
Please, for the love of all that is holy, read up on `Shellwords.escape`. NOW.
Or at least do the rest of us a solid and post where you're setting up these gigantic security holes-waiting-to-happen in production, so we know where NOT to put personal data...
For the ones that do not know, the escaping issues could be handled either by Shellwords.shellescape or passing them as individual arguments to IO.popen & friends (Kernel.system, Kernel.spawn ...)
The article is about speeding up a Rails app. Unless you're planning on hosting that web server on OS X, iOS or an Android device, RubyMotion is not going to be of much use.
Are there any other Ruby compilers? I'm not aware of any that would be comparable to RubyMotion (i.e. actually linking a binary that becomes more performant and not just bundling a interpreter with your script).
And no, RubyMotion does not act as a general purpose Ruby, it only targets iOS, OS X, and Android.
It says something sad about the state of HN when a well-written blog post that is relevant to the topic gets downvoted merely for not being written in Go.
And the point of this is what exactly ? The author advertises Goss "Easy and cheap concurrency", "Low memory overhead" and "Easy deployment", but the example does not show how Go will help in this particular case.
If you were going to consider this seriously, then you should have the target app start once, listen for comands on some channel like a socket, and have it return information. JSON is probably a little too verbose for this kind of use-case as well, but it depends how efficient Ruby would be at unpacking alternative serialization formats.