Docker with Zeus for Rails Development Bliss

I've been running my development environment in Docker for a couple of months, despite the fact that we deploy natively in the various non-dev environments. It's actually working pretty well, chiefly because I am using Zeus as the container command. With this setup, running Zeus client commands for the server, console,  specs, etc, are near-instantaneous.

First -- why would I do this? I had a few reasons.

1. Some components in my native setup on MacOS quit working. In particular, Zeus (application preloader)
2. I moved to a new laptop at work, and I wanted to make it easier to get up and running (it was!)
3. I had already done the legwork to get the stack running as Docker containers (a proof-of-concept spike to port our app to Elasticbeanstalk). It wasn't a huge amount of additional work.

Getting the app to run in Docker locally was fairly straightforward. You can find a lot of good blog posts and Stack Exchange answers explaining how to set up your Rails dev environment in Docker. However, none really matched my workflow. I like having the Zeus preloader running and one or two terminal windows with Zeus client sessions (one for 'zeus s' server, one for 'zeus c' and 'zeus test' sessions). At least I *did*, until Zeus quit working in our project.  I had begrudgingly switched to Spring, but I dislike that Spring doesn't start everything at once. I'd have to wait for "spring rails s", then wait *again* through a cold boot to run "spring rails test", etc. But that's another story.

But Zeus works fine in a Docker container, and I'm happily using it every day now. I use "zeus start" as the docker-compose command for the rails project, like this (simplified a bit):

# docker-compose.yml
version: '3.1'
services:
  db:
    image: postgres
    ports:
      - "5432"
    volumes:
      - ./setup:/setup
  web:
    stdin_open: true
    tty: true
    build: .
    command: "zeus start"
    volumes:
      - .:/app

    tmpfs:
      - /app/tmp
      - /app/log
    env_file:
      - docker.env
    ports:
      - "3000:3000"
    depends_on:
      - db

A shell alias will make Zeus shell commands issued in my host system run via the container:

alias zeus='docker-compose exec web zeus'

With that, zeus commands are now executed in the Docker container transparently with none of the latency I've grown accustomed to with using Spring.

zeus s
zeus g migration addColumn
zeus c
zeus test spec/controllers

And so on. Emacs rspec-mode will use docker and zeus at once when it assembles the command, so test runs are instantaneous. This was the only config I needed:

(setq rspec-use-docker-when-possible t)
(setq rspec-docker-command "docker-compose exec")

Resulting in:

-*- mode: rspec-compilation; default-directory: "~/Projects/customer_portal/" -*-
RSpec Compilation started at Mon May  7 21:18:58

docker-compose exec web bash -c "zeus rspec --options /app/.rspec /app/spec/models/queue_manager_objects/intention_proxy_spec.rb"

Randomized with seed 9956
..

Development bliss! The only problem I have right now is running Selenium for our integration tests. Those weren't working natively on my Mac before, either -- so not a big change -- but I was hoping that it would be simpler in Docker. I'll create another post when I get them working.

Bulk rename files in Emacs

Here's a problem I recently encountered at work which Emacs solved quite effectively with wdired-mode.

I had a large directory tree of Javascript files that I bulk-converted from coffeescript to ES6. The files originally had the extension ".js.coffee," but after conversion they all became ".js.js". Well, a ".js.js" file extension is absurd, right? How can I rename them all to a more sensible ".js"?

Easy -- use Dired mode to display the files, and then switch to 'wdired" (writeable-dired). String-replace the extension, save -- done and done.

But how to list all files from all subdirs in a single Dired buffer for string replacing? In Dired, subdirs can be "included" in the listing by typing 'i' when your cursor is on the subdir, but this is a pain for more than a couple of includes. After searching a bit, I cam across this bit in the Emacs docs for Dired:

The simplest way to include multiple directories in one Dired buffer is to specify the options ‘-lR’ for running ls. (If you give a numeric argument when you run Dired, then you can specify these options in the minibuffer.) That produces a recursive directory listing showing all subdirectories at all levels.

That is, "C-u 1 C-x d" to launch Dired, choose the parent dir, and then type "lR" in the minibuffer prompt for ls options. Voila! All the subdirs listed in your buffer. Now switch to wdired mode ("C-x q"), then query-replace the string ('M-%'). "C-c C-c" to save the buffer. Done.

I don't think there's an easier way to do a bulk rename.

The Trouble With Mail Attachments in Rails

I ran into a problem with a Rails ActionMailer at work recently, and I thought I'd share my findings. Attachments require a multipart mail message -- but adding an attachment does not cause the message to be multipart.

So I had an HTML-formatted email that was working just fine. The Mailer basically looked like this in the action, with only an .html template was in the views directory.

 class DdosReportMailer < ActionMailer::Base  
  layout 'layouts/mailer'  
  def daily_report(rcpts:)  
   mail(  
    to: EnvVar.get('NOREPLY_EMAIL'),  
    from: EnvVar.get('NOREPLY_EMAIL'),  
    bcc: rcpts,  
    subject: I18n.t("ddos_report_mailer.daily_report.heading")  
   )
  end  
 end  

My story required that I add attachments, but the Rails guide is not exactly clear about when and where to call the 'attachments' method. First, I tried calling it on the mail instance after it was created:

 class DdosReportMailer < ActionMailer::Base  
  layout 'layouts/mailer'  
  def daily_report(rcpts:, reports:)  
   mail(  
    to: EnvVar.get('NOREPLY_EMAIL'),  
    from: EnvVar.get('NOREPLY_EMAIL'),  
    bcc: rcpts,  
    subject: I18n.t("ddos_report_mailer.daily_report.heading")  
   ).tap do |m|  
    reports.each { |k,v| m.attachments[k] = v }  
   end  
  end  
 end  

In the Rails previewer, this looked good. HTML formatting, check; attachment, check. Tests which verified the mail body and attachments passed, so I figured the work was done.  

But when I sent the actual mail, it looked like this:

I immediately lay this problem at the feet of the MailSafe gem, which we use to prevent outbound mail in staging. The MailSafe modification seems so close to the attachment boundary, it must be causing a parsing error! Not to worry -- in production, we won't have this problem. Just to be sure, I whitelisted the original recipient address so there was no MailSafe intervention. Surprisingly, that did not help. 

I went back to the Rails guide and decided the 'attachments' call should be sent to the mailer, not the mail.

 class DdosReportMailer < ActionMailer::Base  
  layout 'layouts/mailer'  
  def daily_report(rcpts:, reports:)  
   reports.each { |k,v| attachments[k] = v }  
   mail(  
    to: EnvVar.get('NOREPLY_EMAIL'),  
    from: EnvVar.get('NOREPLY_EMAIL'),  
    bcc: rcpts,  
    subject: I18n.t("ddos_report_mailer.daily_report.heading")  
   )
  end  
 end  

Now the delivered message looked even worse! Something is wrong with the message parts.

After a bit of Googling I found a discussion about a similar multi-part problem that was solved by passing a block to the 'mail' method and using the format object which is yielded. This did the trick. Although attachments require a multipart message, adding an attachment does not cause the message to be multipart. Using a format block will. Here is the final code.

 class DdosReportMailer < ActionMailer::Base
  layout 'layouts/mailer'
  def daily_report(rcpts:, reports:)
    reports.each { |k, v| attachments[k] = v }
    mail(
      to: EnvVar.get('NOREPLY_EMAIL'),
      from: EnvVar.get('NOREPLY_EMAIL'),
      bcc: rcpts,
      subject: I18n.t("ddos_report_mailer.daily_report.heading")
    ) do |format|
      format.html { render 'daily_report' }
    end
   end
  end

Emacs and The Lisp Keybard

I've been using Emacs for my day-to-day developer work for over a year now. I love it, but I do sometimes get a sore left pinky (aka, "Emacs pinky") from hitting the Control key so much. It's not nearly as bad as the mouse-hand tendonitis I used to get from all the clicking in a windowed development environment, but still.

Well it turns out that Emacs' heavy reliance on the control key is explained by the "Lisp keyboards" of the 80s, which influenced its early development. Control is next to the spacebar, where you can hit it with your thumbs.

If you use Emacs, Control is supposed to be big, right next to the space bar. On Windows keyboards, that's typically where the Alt key goes now. Macs put the Command key there. It's time for some key mapping.

I spend about half my day on Macbook Pro docked with a Windows keyboard, the other half using the laptop keyboard on the go. The modifier keys are crazily different, but luckily System Preferences > Keyboard > Modifier Keys lets you pick different mappings for your different keyboards. So I've been trying to get used to this setup:

Whichever key is next to the space bar is now Control, next comes Alt (Meta), and furthest is Super/Command/Windows. CapsLock is an extra Control key, for good measure.

Outside of Emacs, it's going to take some getting used to -- esp. Cmd-C/V, Cmd-Space, Cmd-Tab, the combos I use all the time on the Mac. On the plus side, copy and paste is more compatible with Windows/Linux key combos for copy/paste, where I was already used to "Ctrl-C" with a pinky.

Inside Emacs, using the thumb for Control feels a whole lot better. Control-X O, Control-X G, etc -- I use those constantly, and the thumb just works a lot better. I recommend it!

Learning Emacs, or: Teaching an old dog old tricks

I decided to learn Emacs late last year, and it took some doing. But Emacs in a terminal is now my go-to editor. Why on earth, you might ask.

I've been a professional developer for quite some time, and I've always used IDEs for my work, especially in Java -- Jbuilder, Netbeans, IntelliJ, and, more recently, Rubymine. There are many things to love about these tools: autocompletion, instant symbol navigation, and so forth. Start typing a method call, and the editor prompts you for the parameters by name. Command-click on a symbol and it takes you to the definition, no matter where in the project it may live. But with the power come some annoyances, too. Mysterious freezing, slow start-up time -- and the nagging feeling that maybe I'm outsourcing a bit too much of my knowledge and skill to the IDE.

As a Ruby developer these last few years, I've found myself more and more on the command line. Pry, in particular, made me want to leave Rubymine behind. It's a much more intuitive debugging paradigm, and as far as I can tell, it is completely incompatible with a Rubymine run/debug config. (A 'binding.pry' encountered in the code will just hang the server started in Rubymine.) I wanted to stay in the terminal, where things are always fast and efficient.

It was time to learn Emacs or vi. I decided on Emacs mainly because vi's editing vs. navigation modes have always kind of bothered me.

Some reasons to learn Emacs, in no particular order:

1. Adios, trackpad and right-click menu navigation. Your hands stay on the keyboard
2. Learn the Emacs key bindings and you can use them all over MacOS 
3. Gain a new fundamental skill while doing exactly the same old thing at the job
4. Get IDE-level productivity on a remote system via SSH
5. Earn maximum old-timer street cred!

Out of the box, Emacs has some pretty cool stuff to offer. There's plenty to learn if you start with a vanilla install. To name just a couple of things I really like:

1. Buffers and windows just seem to work better than the typical paradigm (tabs) for dealing with files in a modern editor. C-x b enter always takes you to the previous file, whereas in Rubymine, Sublime, etc, I'm always searching for that other tab I was just using. C-x 3 to split into vertical windows, C-x 1 to get your current frame back to one window. These come in handy -- two files is about all I ever want to see at one time.
2. Dired for browsing/manipulating files. It's more powerful for file management than the project trees in the modern editors.

Add these packages, and you'll be flying.

1. Helm and projectile for navigating the project
2. Magit for git

Learning Emacs is like learning piano (if piano only took a couple of months to learn). It takes time, but you form muscle memory for executing an intention (going to a file, committing changes, etc), which ultimately feels more fluid than poking around in menus and panels with the mouse. Your visual cortex can stay focused on the high-level problem, and your spinal cord takes care of the details.

fun with dataphone emulators

So I'm working on a dot-net project right now, and we are supporting mobile devices. I figured I would just download some emulators for iPhone, Android, etc, and check out how things look.

First stop, Apple. Here, the only way to get the iPhone emulator is to download the entire 2.08 Gigabyte SDK after creating a developer account. I'm not very interested in either of those things, so I figured it would be easier to just find someone with an iPhone when we get it on a public server.

The Android SDK is much easier to obtain (only 186 Megabytes, no account required). After reading up on how to start the emulator, I found it will not connect to our internal servers by name or IP. Another failed attempt.

Moving on, I was happy to see Visual Studio has a bunch of Windows CE device emulators bundled. But try to enable "networking" for the emulated platform, and you'll get a message saying that Virtual PC is required, with a download link. That's right, Microsoft device emulators in Visual Studio cannot simulate a network connection without installing Virtual PC 2007. Again, I am not interested.

Why is this so difficult? Why not separate the emulators from SDKs for download? Why not write them to use the host OS networking transparently?

The Inauguration, Power, and Consent

I just watched the inauguration and, like many others, I was moved to tears. I've generally been a little skeptical of Obama-mania, but he is the right person for the job, without question. His eloquence, confidence, and humility are truly remarkable.

But I want to comment on how many times I heard the commentators assert amazement at the "peaceful transfer of power" that the inauguration represents. They are astounded that our leaders cede power without violence. But is a violent refusal to yield power even within the realm of possibility? Isn't "peaceful transfer" pretty much standard for most of the world nowadays? England, Canada, El Salvador, etc etc, all see peaceful transfers these days.

In fact, I suspect we are being exposed to a propaganda assertion: "consider yourselves blessed and lucky that these people rule over you peacefully, without infighting." The comradery displayed between these potentially violent adversaries (Clinton and Bush Sr. smiling and hugging, Gore and Bush Jr. sharing a stage) confirms that familiar critique of American two-party politics: these are just two sides of the same coin, a single oligarchical class. Don't mistake me, there are real and profound differences between Republicans and Democrats, and I am very grateful that Obama is president. But everyone on that inauguration stage clearly belongs to the same club.

What else is happening on that stage? The tooting horns, the bunting and banners, the processional entrance -- it's just a good old fashioned pagaent, used by rulers throughout history to impress their majesty and power. The subjects / constituents are always taken right in.