Running a Private Gem Server

There are a few different ways to share code between Ruby applications, but perhaps the best known is by creating a Ruby gem. However, RubyGems.org (where gems are published by default) is a public system, and all gems published there are publicly available. This is a problem if you’re dealing with proprietary code that you need to keep private.

Thankfully, it is very easy to get up and running with your own private gem server.

Why not use Bundler to pull code from a private Git repository?

Another popular way of sharing code between applications is to use Bundler to pull code directly from a git repository. Pulling in code from a private git repository allows you to share code between applications while keeping that code private. However, this approach has some drawbacks.

Transitive dependencies don’t work well

Gems are able to specify their dependencies in the .gemspec file.

Gem::Specification.new do |spec|
  ...
  spec.add_dependency "activesupport"
end

But, what happens when you need to specify a dependency on a private gem? The gem specification does not allow you to point to a GitHub repository. Since we cannot specify the location of the dependency in the .gemspec, the only option is to do that in the application’s Gemfile.

In the gem’s .gemspec

# Some gem has a dependency on a private library
Gem::Specification.new do |spec|
  spec.name = "some-private-library"
  ...
  spec.add_dependency "another-private-library"
end

In the application’s Gemfile

# The application's Gemfile must tell bundler where to find that library
source 'https://rubygems.org'

gem 'some-private-library', git: 'git@github.com:jwood/some-private-library.git'

# Required by some-private-library
gem 'another-private-library', git: 'git@github.com:jwood/another-private-library.git'

The application should only need to care about the libraries that it directly depends on. It should not need to specify the location of transitive dependencies. This tightly couples the application to the implementation of the some-private-library gem, for no good reason.

It is less clear which version of the library you are using.

Looking at an example from a project’s Gemfile.lock, compare this:

GIT
  remote: git@github.com:redis/redis-rb.git
  revision: 9bb4156dcd43105b68e24e5efc579dddb12cf260
  specs:
    redis (3.0.6)

with this:

    redis (3.0.6)

Which version of the library are you working with in the first example? How about the second? If you said 3.0.6 for the first example, then you may be incorrect. You’re actually using the version identified by 9bb4156dcd43105b68e24e5efc579dddb12cf260. That may in fact be the same version of the code with the 3.0.6 label. Or, possibly, it is pointing at a version of the code that is several commits past the 3.0.6 label.

In the second example, it is abundantly clear that you are using version 3.0.6 of the gem.

Harder to stick to released versions of the library.

Like any other codebase, libraries are often under continuous development. Commits are constantly being made. Branches are constantly being merged. While the contents of master should always work, it may not always represent what the library author/maintainer would consider releasable code. At some point, the maintainer of the library will deem the version of code in master releasable, tag the code, and cut a new version of the library. If you use Bundler to point to a private git repository, you could end up using code that is not considered releasable.

One way around this issue is to tag releases in git, and then use bundler pull in that version of the code.

gem 'rails', git: 'git://github.com/rails/rails.git', tag: 'v2.3.5'

While this will in fact lock you to the tagged version of the code, it makes updating the gem more difficult. You can no longer use bundle update rails to update to the newest released version of the gem. You will remain locked on the v2.3.5 tag until you remove the tag directive from the gem in your Gemfile.

How do I setup a private gem server?

Thankfully, setting up your own gem server is very easy. The fantastic Gem in a Box project is a self contained, full featured gem server. It is even recommended as the “way to go” by Rubygems.org if you want to run your own gem server.

After your gem server has been setup, you can continue to pull public gems from Rubygems.org, while pulling private gems from your new private gem server.

How can I publish gems to my private gem server?

It doesn’t take much work to get a set of rake tasks that you can use to publish gems to your private gem server.

First, in your .gemspec file, make sure you specify rake, bundler and geminabox as development dependencies. Make sure you use version 1.3.0 or greater of bundler, as 1.3.0 added a feature which we will take advantage of later.

  spec.add_development_dependency "rake"
  spec.add_development_dependency "bundler", ">= 1.3.0"
  spec.add_development_dependency "geminabox"

Next, require bundler’s gem tasks in your project’s Rakefile.

require "bundler/gem_tasks"

bundler/gem_tasks gives you three rake tasks that come in handy for publishing gems.

rake build    # Build some-private-library-0.0.1.gem into the pkg directory.
rake install  # Build and install some-private-library-0.0.1.gem into system gems.
rake release  # Create tag v0.0.1 and build and push some-private-library-0.0.1.gem to Rubygems

By default, the release task will publish your gem to Rubygems.org. But, that’s not what we want. With a few lines of code in our Rakefile, we can tell bundler to push the gem to our internal gem server instead.

# Don't push the gem to rubygems
ENV["gem_push"] = "false" # Utilizes feature in bundler 1.3.0

# Let bundler's release task do its job, minus the push to Rubygems,
# and after it completes, use "gem inabox" to publish the gem to our
# internal gem server.
Rake::Task["release"].enhance do
  spec = Gem::Specification::load(Dir.glob("*.gemspec").first)
  sh "gem inabox pkg/#{spec.name}-#{spec.version}.gem"
end

Now, to release your gem, you simply type rake release

% rake release
some-private-library 0.0.1 built to pkg/some-private-library-0.0.1.gem.
Tagged v0.0.1.
Pushed git commits and tags.
gem inabox pkg/some-private-library-0.0.1.gem
Pushing some-private-library-0.0.1.gem to http://gems.mydomain.com/...
Gem some-private-library-0.0.1.gem received and indexed.

The first time you do this, geminabox will prompt you for the location of your gem server. After that, it will remember.

How do I use the gems on my private gem server?

To use the gems on your private gem server, you simply need to tell bundler where to find them in your application’s Gemfile. This is done using the source command:

# The application's Gemfile

source 'https://rubygems.org'
source 'https://gems.mydomain.com'

gem 'some-private-library'  # Will be found on the private gem server
Be Sociable, Share!

    One thought on “Running a Private Gem Server

    1. That’s pretty cool. I’m never thought of setting it up in the rake file. I just used geminabox interface to upload the gems from the directory. Automating upload to my gemserver seems really useful. Thanks for sharing.

      One thing I could not figure out is how to run a gemserver as an init.d service. It seems that rvm makes the task more difficult. I’ll share if I figure that out.

    Leave a Reply

    Your email address will not be published. Required fields are marked *