Background - the application size

Feel free to skip to the next section if you don't care about it.

I recently finished moving a 5 years old Rails application to a custom stack on top of Roda from Jeremy Evans, also the maintainer of the awesome Sequel ORM. The application is actually older than that and I've been working on it for 6 years. It used to be a Grails application that was moved from SVN to Git about 7 years ago but I never had access to the SVN repository so I don't really know how old this application is. It was completely migrated from Grails to Rails in 2013. And these days I replaced Rails with Roda but this time it was painless and only took a few weeks.

I have some experience with replacing the technology of an existing application without interrupting the regular development flow and deployment procedures and the only times I really had to interrupt the services for a little while was the day I replaced MySql with PostgreSQL and the day I moved the servers from collocation to Google Cloud Platform.

I may write about what steps I usually follow when changing the stack (I replaced Sprockets with Webpack a few years ago among, Devise with a custom solution, among many examples) in another article. But the reason I'm describing this scenario for this article's purpose is only so that you have some raw idea about this project size, specially if you consider it had 0 tests when I joined the company as the sole developer and had to understand a messy Grails application with tons of JS embedded in GSP pages with functions comprising hundreds of lines with many many logical branches inside. Years later and there are still tons of tests lacking, specially in the front-end code and much more to improve. To give you a better idea, we currently have about 5k lines of Ruby test code, and 20k lines of other custom (not generated) Ruby code plus 5k lines of database migrations code. Besides that we have about 11k lines of CoffeeScript code, 6k lines of JS code and 2.5k lines of CoffeeScript tests code. I'm not including any external libraries in those stats. You have probably noticed already how poor is the test coverage currently, specially in the front-end. At this point I expect you to have some raw idea on this project size. It's not a small project.

Why replacing Rails in the first place?

Understanding this section is definitely the answer on why I feel alone in the Ruby community.

More background about Rails and the Ruby community

Again, feel free to skip this subsection.

When I was working on my Master thesis (Robotics, Electrical Engineering) I stopped working with web development for a while and focused on embedded C programming, C++ hard real-time systems and the like. After I finished the Master thesis my first job was back to Delphi programming. Only in 2007 I moved my job back to web development, several years later and I only had experience with Perl so far. After a lot of research I decided for Rails and Ruby, although I have also seriously considered TurboGears and Django by that time, both using the Python language. I wasn't worried by the language by that time as I didn't know either Ruby or Python and they seemed similar one to the other. Ultimately I chose Rails because of how it handled database migrations.

In 2007, when looking at the alternatives, Rails was very appealing. There were conventions that would save me a lot of work when starting to work with web development again, there were generators to help me getting started, great documentation, it bundled a database migrations framework so that I wouldn't have to recreate myself, simple to understand error stack-traces, good defaults for the production environment (such as proper 500 and 404 pages), great auto-reloading of code in the development environment, great logging, awesome testing tools and integrated to generators, quick boot, custom routes, convention over configuration and so on.

Last but not least, a very rich ecosystem with smart people working on great gems and learning Ruby together and they were all amazing by its meta-programming capabilities, the possibility of changing core classes through monkey patches and so on. And since it's possible, we should use it in all places we can, right? Specific-domain-languages (SDL) were used by all popular gems by that time. And there wasn't much fragmentation like in the Java community. Basically almost anyone writing web applications in Ruby were writing Rails apps and following its conventions. That allowed the community to grow fast, with several Rails plugins and projects assuming the application was running Rails. Most of us have only known Ruby because of Rails, including myself. This is already enough reason to thank DHH. Rails definitely raised the bar for other web frameworks.

As the ecosystem matured, we saw the rise of Rack and more people using what they called micro-frameworks such as the popular Sinatra, Merb among others. Rails improved internationalization support in version 2, merged with Merb in version 3, got Sprockets in version 4 and so on. The assets pipeline were really a thing when they were introduced in Rails by that time. It was probably the latest really big change introduced by Rails that really inspired the general web development scenario.

In the meantime Ruby has also evolved a lot, providing better unicode support, adding a new Hash syntax, garbage collecting symbols, improving performance and getting new great tools such as Bundler. RubyGems got a better API, the Rails guides got much better and they have a superb documentation on securing web applications that is accessible to any web developer and not only Rails ones. We have also seen lots of books and courses teaching the Rails way, as well as many dedicated blogs, videos, conferences and so on. I don't remember watching such a fast growing in any other community until JavaScript got a lot of traction recently, motivated not only by single page applications which are becoming more and more common, but also by the creation of Node.js.

Many more languages have been created or re-discovered recently including Go, Elixir, Haskell, Scala, Rust and many many more. But up to this day, despite the existing of symbols and a poor threading model in MRI and lack of proper support for threaded applications in stdlib, Ruby is still my preferred general purpose language. That includes web applications. What about Rails?

Enough is enough! What's wrong with Rails?

If you guessed performance was the reason, you guessed wrong. For some reason I don't quite understand, developers seem to be obsessed by performance even in scenarios where it doesn't matter. I never faced server-side performance issues with Rails. Accordingly to NewRelic most requests would be served by less than 20ms in the server-side. Even if we could cut those 20ms it wouldn't make any difference at all. So, what's wrong after all?

There's nothing wrong with Rails in a fundamental way. It's a matter of taste in my case I guess because it's really hard to find an objective way to explain why I wasn't fully satisfied with Rails. You should probably understand that this article is not about bashing on Rails in any way. It's a personal point of view on why I feel like a strange and why it's not a great feeling. [Update: after writing this article, I spent some time trying to list the parts I dislike in Rails and wrote a dedicated article about it, which you can read here if you're curious]

To help you understand where I come from, I have never followed the "Rails Way" if there's such a thing. I used jQuery when Prototype was the default library, I used RSpec when test/unit was the default one, I used factories when Rails teached fixtures, I used Sequel rather than the bundled ActiveRecord, but instead of Sequel's migrations I used ActiveRecord's migration through the active_record_migrations gem. Some years ago I replaced Sprockets with Webpack (which fortunately Rails just embraced in Rails 5.1 release, while I wasn't using Rails anymore when it was released). After some frustration trying to get Devise to work well with Sequel I decided to replace Devise with a custom solution (previously I had to customize Devise a lot to make it support our non-traditional integration for dealing with sign-ins and custom password hashing inherited by the time it was written in Grails).

Since we're talking about a single page application, almost all of the requests were JSON ones. We didn't embrace REST, or respond_to, we had very few server-side views and often had to dig into Rails or Devise source code to try to understand why something wasn't working as we expected them to. That included several problems we had with streamed responses (which Rails calls Live Streaming for some reason I don't quite follow, although I suspect that's because they introduced some optimizations to start sending the view's header sooner and called it streaming support, so they needed another name when they introduced ActionController::Live) after each major Rails upgrade. I used to spend a lot of time trying to understand Rails internal source whenever I had to debug such problems. It was pretty confusing to me. The same happened with Devise.

At some point I started to ask myself what Rails was adding to the table. And it got worse. When I first met Rails it booted in no time. It got slower to boot at each new release and then they introduced complex solutions such as spring to try to fix this slowness. For a long time they used (and still use to this day) Ruby's autoload feature to lazily evaluate code as it's needed in order to decrease the boot time. Matz don't like autoload and I don't like it either, but this article is already long enough to discuss this subject too.

Something I never particularly enjoyed in Rails was all that magic related to auto-loading. I always preferred explicit and simple code over sophisticated code that auto-wires things. As you can guess, even though I loved how Rails booted quickly and how auto-reloading just worked with Rails (except when it didn't - more on that later) I really wanted to specify all my dependencies explicitly in each file. But I couldn't just use require or auto-reloading would stop working. I had to use ActiveSupport's require_dependency and I hated it because it wasn't just regular Ruby code.

I also didn't like the fact that Rails enforced all monkey patches to Ruby core classes made by ActiveSupport extensions, introducing methods such as blank?, present?, presence, try, starts_with?, ends_with? and so on. That's related to the fact I enjoy explicit dependencies as I think it's much easier to follow a code with explicit dependencies.

So, one of my main motivations to get rid of Rails was to get rid of ActiveSupport, since Rails depends on ActiveSupport, including its monkey patches and auto-loading implementation. Replacing Rails with Roda alone didn't allow me to get rid of ActiveSupport just yet as I'll explain later in this article, but it was an important first move. What follows is the kind of frustration with the Ruby community in the sense of how very popular Ruby gems are written with about the same mentality of those from Rails core. Such gems include the very popular mail gem as well as FactoryGirl, for example. Even Sidekiq will patch Ruby core classes. I'll talk more about this later, but let me introduce Roda first.

[Update: after writing this article both the mail and sidekiq gems have worked to remove their monkey patches and I'd like to congratulate them for the effort and give them "Thank you so much!"]

Why Roda?

From time to time I considered replacing Rails with something else but I always gave up for a reason or another. Sometimes I realized I liked Sprockets and the other framework didn't provide an alternative to the Rails Assets Pipeline. Another time I realized that auto-reloading didn't work great with the other framework. Other times I didn't like the way code was organized with the other framework. When I read Jeremy's announcement for Roda, it was just the right time with the right framework for me.

I greatly appreciate Jeremy from a long time since getting introduced to Sequel. He's a lovely person, who provides awesome and kind support and he's a great library designer. Sequel is simply the best ORM I've seen so far. Also, I find it quite simple to follow Sequel's code base and after looking into Roda's source it's pretty much trivial to follow and understand. It's basically one simple source file that handles routing and plugins support and basically everything else is provided by plugins you can opt-in/out and each plugin, being small and self contained, is pretty simple to understand and if you don't agree with how it's implemented just implement that part your own.

After having a glance over the core Roda plugins one stood out particularly: multi_run. For what I want, this plugin would give me great organization, similar to Rails controllers, with the advantage that they could have their own middleware stacks, they could be mounted anywhere, including in a separate app, they were easy to test separately as if they were a single app if desired but more importantly: it allowed me to easily lazy load the application code, which allowed the application to boot instantly with Puma, without the need of autoload and other trickery. Here's an example:

Even if you decide to load the main application when testing particular apps, the overhead would be negligible, since it would only load the tested app basically. And if you are afraid of using lazy loading in the production environment because you want to deliver a warmed app, it's quite easy to change register_app:

This is not just a theory, this is how I implemented in our application and it boots in less than a second. Just about the same as the simplest Rack app. Of course, I hadn't really measured this in any scientific way, it's a simple in-head count when running bundle exec puma, where most of the time is spent on Bundler and requiring Roda (about 0.6s with my gemset). No need for spring, autoload or any complicated code to make it fast. It just works and it's just Ruby, by using explicit lazy loading rather than an automatic system.

So, I really wanted to try this approach and I had a plan where I would run both Roda and Rails stacks altogether for a while, by running the Rails app as the fallback app when the Roda stack wouldn't match the route. I could even use the path_rewriter plugin to migrate a single action at a time to the Roda stack if I wanted to.

There was just one remaining issue I had to figure out how to solve before I started moving the app to the Roda stack: automatic code reloading. I decided to ask in the ruby-roda mail group how Roda handled code reloading and Jeremy said it was out of Roda's responsibility and that I could choose any code reloader I wanted and pointed to some documentation listing some of them, including one of his own. I spent quite some time researching about them and still preferred the one provided by ActiveSupport::Dependencies but since I wanted to get rid of ActiveSupport and autoloading in the first place there was no point in keep using it. If you're curious about this research, I wrote about it here. If you're curious on why I dislike Ruby's autoload feature, you'll find the explanation in that article.

After some discussion around automatic code reloading in Ruby with Jeremy I suggested him an approach I think would work pretty well and transparently although it would require to patch both require and require_relative in development mode. Jeremy wasn't much interested on it because of those monkey patches, but I was still confident it would be a better option than the others I had evaluated so far. I decided to give it a try and that's how AutoReloader was born.

With the autoreloading issue solved, it was all set to start porting the app slowly to the Roda stack, and the process was pretty much a breeze. If you want to have some basic idea on Rails overhead, the full Ruby specs suite were about 2s faster with the same (converted) tests after getting rid of the last Rails bits. It used to take 10s to run 380 examples and thousands of assertions, and after getting rid of Rails it took 8s with an extra example. Upgrading Bundler saved me another half a second so currently it takes 7.6s to finish (about half a second for bundle exec, 1.5s to load accordingly to RSpec report and 5.6s to run).

But getting rid of Rails was just the first step in this lonely journal.

Rails is out, what's next?

Getting rid of Rails wasn't enough to get rid of ActiveSupport. We have a LocaleUtils class we use to format numbers among other utilities based on the user's locale. It used to include ActionView::Helpers::NumberHelper, and by that time I learned the hard way that I couldn't simply require 'action_view/helpers/number_helper' because I'd have problems related to ActiveSupport's autoloading mechanism, so I had to fully require action_view. Anyway, since ActionView depends on ActiveSupport I wanted to get rid of it as well. As usual, after lots of wasted time searching for Ruby number formatting gems I decided to implement the formatting myself and a few hours later I got rid of ActionView.

But ActiveSupport was still there as a great warrior! This time it was dependency of... guess what? Yep, FactoryGirl! Oh, man :( After some research on alternative factory implementations I found Fabrication to be dependency free. An hour later I ported our factories to Fabrication and finally got rid of ActiveSupport! Yay, no more monkey patches to core Ruby classes! Right?

Well, not exactly... :( The monkey patch culture is deeply rooted in Ruby's community. Some very popular gems add monkey patches, such as the mail gem, or sidekiq. While reading the mail gem source I found it very confusing, so I decided to replace it with something simpler. We use exim4 to forward e-mails to Amazon SES, so Ruby's basic NET/SMTP support is enough for delivering e-mails to Exim, all I needed was a MIME mail formatter in order to send simple TEXT + HTML multi-part mail to users. After some more research I decided to implement it myself and this is how simple_mail_builder was born.

At some point I might decide to create my own simple jobs processor just to get rid of Sidekiq's monkey patches, but my point is that I have this feeling of being a lonely warrior fighting a lost battle because of my expectations mismatch with what the Ruby community overall consider acceptable practices such as modifying Ruby core classes in libraries. I agree it's okay for instrumenting code, such as NewRelic, to patch other's code, but for other use cases I don't really agree with such approach.

In one hand I really love the Ruby language, except for some few caveats, but there's a huge mismatch with the Ruby community way of writing Ruby code, and this is a big thing. I don't really know what's the situation in other language communities, so I guess I might be a lonely warrior in any other language I opted for instead of Ruby, but Ruby is the only language I really appreciate so far among those I've worked with.

I guess I should just stop dreaming about the ideal Ruby community and give up on trying to get a monkey-patch free web application...

At least, I can now easily and happily debug anything that happens to the application without having to spend a lot of time digging into Rails or Devise's source code, which used to take me a lot of time. Everything's clean water. I have tons of flexibility to do what I want in no time with the new stack. The application boots pretty quickly and I'll never run into edge cases involving ActiveSupport::Dependencies auto-reloading again. Or issues involving ActionController::Live. Or Devise issues when using Sequel as the ORM.

Ultimately I feel like I got full control over the application and that's simply priceless! It's an awesome feeling of freedom I never experienced before. Instead of focusing on the lonely warrior fighting a lost battle bad feeling, I'll try concentrate on those great benefits from now on.