Occasionally in the life-cycle of an application we find that we need two distinct domain objects to quack alike, even if their underlying structures differ greatly. Moreover, often the second, third and thirty-ninth such objects come long after the initital object entered the application. One approach would be to shove all of the necessary functionality into the original object and add a ”type” flag of some sort. Another would be to use single table inheritance or even multiple table inheritance (here’s a gem) if we’re feeling fancy. Another approach is to decorate the objects identically so they all quack like the round-duck-in-the-square-hole you want them to be.

Consider some Widget class, persisted through ActiveRecord, with a couple of attributes and a virtual attribute:

class Widget < ActiveRecord::Base
  attr_accessible :name, :feature
  def display_title
    "Hi my name is #{name} and I can do #{feature}"
  end
end

along with, for instance, some view template for the show action (here presented in Slim):

h1 Widget!
p
  ' Name:
  = @widget.display_title

While this may not be an optimal starting place, its simplicity makes it a great place to start. If more flexibility isn’t demanded at this point, why would you introduce decorators? At least the display logic in the view is isolated to a single method. Later, if things start to get out of control, extraction of #display_title to a more appropriate home should be simple enough.

Several iterations later…

Say that after launch, and during the course of new feature development, a new business need arises. Our previously flimsy Widgets have a more musically inclined MetalWidget cousin that must come into play. These new loud, rock-themed widgets need to be displayed in exactly the same way as their humbler cousins, but with different underlying attributes. One option would be to put some business logic in the view and render a different partial depending on the Widget type. Another option would be to leverage the Decorator pattern and build decorator classes to wrap up the two objects with a consistent interface. The first step in such a refactor is to simply extract the Widget display logic from the view into a decorator of some type. In the following examples we will use Draper, but any implementation of the decorator or presenter pattern should work. We can move the Widget#display_title method into a decorator, remove it from the model and update the WidgetsController show action to decorate the Widgets before rendering the view.

class ApplicationDecorator < Draper::Base
end
class WidgetDecorator < ApplicationDecorator
  decorates :widget
  def display_title
    "Hi my name is #{name} and I can do #{feature}"
  end
end

Then we can introduce the MetalWidget class in order to make the application significantly louder.

class MetalWidget < ActiveRecord::Base
  attr_accessor :band_name, :number_of_demo_tapes
end

In order to render these new leather-clad Widgets, we need to give them a decorator as well, so that they can be displayed in the same show action.

class MetalWidgetDecorator < ApplicationDecorator
  decorates :metal_widget
  def display_title
    <<-BAND_STUFF
    We are #{band_name}! Thanks for coming. Please pick up one of our
    #{number_of_demo_tapes} excellent early albums from the site before
    you go!
    BAND_STUFF
  end
end

So now instances of either Widget or MetalWidget can be displayed equally well with a single template. Another benefit of this refactoring is that the two Widget classes actually look very similar. The virtual attribute has been pulled out so the original Widget is smaller and more focused, and display concerns live elsewhere. Taking this idea further, it can be handy to create a common ancestor for all Widget decorators so that a given Widget type will never blow up in a view, even if it doesn’t implement the entire interface needed by the view. For example, we might have:

class WidgetDecorator < ApplicationDecorator
  def display_title
    ""
  end
end
class WidgetDecorator < WidgetDecorator
  decorates :widget
  def display_title
    #...
  end
end
class MetalWidgetDecorator < WidgetDecorator
  decorates :metal_widget
  def display_title
    #...
  end
end

This final step is similar to a Null Object pattern, in that it prevents views from raising exceptions if unknown methods are called. We could bring other machinery to bear here, but this feels pretty natural in the course of refactors such as the above.

With this process, and leveraging the power of inheritance, we are able to create a family of objects that all quack alike and allow us to iteratively implement the view concerns of new, but related, domain objects as they are introduced to the system. The base class of the inheritance hierarchy allows us to have reasonable default behavior, which may simply be doing nothing except not throwing an exception, and inheriting decorators can override that behavior to provide correct information to the views.

How do you build flexibility into your domain models? Do you replace conditionals with polymorphism or extract service objects, or do you have another preferred method?

The post Creating View-Centric Interfaces with the Decorator Pattern appeared first on Big Nerd Ranch.

I’ve written before about Highgrooves’s “any conference you want, every year, on us” policy. This year, I chose to attend two events, with a cycling adventure between.

ArrrrCamp

When I originally intended to attend ArrrrCamp’12, I planned to go as a participant. I wanted to make the most of the best part of every conference: the gaps between talks. Those “hallway sessions,” during which we get to talk informally with our fellow developers, are the high point of conferences for many of us.

As it happens, I was able to do that and give a talk.

ArrrrCamp, a pirate-themed conference on Ruby, Rails, Radiant CMS and Rum, is hosted in the medieval city of Ghent, Belgium. It was well worth taking some extra time ahead of the conference to simply explore the city. I was able to see the masterpiece Adoration of the Blessed Lamb, consume some fantastic coffee at Mokobon and Barrazza, check out the future of Highgroove ergonomics and drink some local beers while getting a feel for this vibrant university city.

When the conference began, it became clear that the two tracks were split thematically across front-end and back-end lines. Because I am much more comfortable with back-end development, I took advantage of this split to learn more about the front end of things.

• I learned as much from Jim Weirich’s very Lisp-themed keynote on the y-combinator as I did from Jakob Hilden and Roy Tomeij talking about reusable CSS components.
• Steve Klabnik’s talk on the changes on the upcoming Rails 4.0 was great and made me feel better about missing the similarly themed talk from Highgroover Andy Lindeman.
• Laurent Sansonetti’s talk about RubyMotion made me excited to try my hand at native iOS application development, while Bernard Grymonpon’s talk on testing Chef recipes resonated strongly with me.
• Katrina Owen gave her Therapeutic Refactoring talk, which has received rave reviews.

I found myself giving a talk when Corey Donohoe unfortunately wasn’t able to make it to the conference as planned. I stepped in with a slightly updated version of the talk on Static Analysis of Ruby and Security Scanning of Rails Apps with Brakeman that I gave at Ruby Hoedown earlier this year. Giving the talk was great fun and the audience asked some new and complicated questions that I had never considered before.

Cycling adventure

After ArrrrCamp, I made my way to Oudenaarde, an area in the Belgian region of Oost-Vlaanderen. Our methodologist forbade me from working while I was there, so instead I rode bikes for 11 days straight and enjoyed myself greatly. I rode the entire route of the Ronde van Vlaanderen (the famous Tour of Flanders), and I fulfilled a lifelong dream of riding to Roubaix to see the velodrome there.

GreHack

After the bicycle-centric interlude, it was time to get serious again. I headed to France for the first year of GreHack, a security conference hosted by the University of Grenoble. The panel covered subjects ranging from tracking and categorizing botnets to iOS and cable TV hacking, along with a large dose of low-level hardware hacking.

I presented on Leveraging Convention over Configuration for Static Analysis in Dynamic Languages, and in my talk I focused on why it’s ok to write simple frameworks for complicated tasks.

Overall, I had a great trip, and I enjoyed being able to immerse myself in the experience. However, after 22 days abroad, two conferences and 11 days of cycling, I’m glad to have my feet back on my home turf.

At Highgroove, we want to make sure we never rest on our laurels, and we force ourselves out of our comfort zones to continuously learn about fields other than our own core competencies.

What do you do to stretch your comfort zone while contributing to the community?

Image credit: davidworth

The post ArrrrCamp'12 and GreHack Recap appeared first on Big Nerd Ranch.

At Highgroove, we love working with the community, whether it’s hosting hack nights, organizing ATLRUG, or contributing to open-source software. One of our key contributions involves auditing those ruby gems upon which we most rely, or the ones that present the most perceived risk. Ernie Miller’s Squeel definitely piqued our security interest. Read on to hear about the results of our audit and how we worked with Mr. Miller to validate our results.

The Problem

Squeel is one of the “scope helpers” listed in our ever-evolving security audit process. It provides a powerful DSL that makes building Rails scopes more “ruby-ish,” rather than being strings of SQL that get parsed and used. It also exposes a huge amount of functionality without requiring developers to get into raw SQL. That said, anytime a gem exposes any SQL functionality, it should give developers pause, particularly if that SQL may happen to contain user input.

SQL Injection (SQLi) is scary because of the level of exposure resulting from leaked information; there are a number of widely publicized compromises of large sites due to its exploitation. Moreover, it is notorious for being difficult to quash in field of web security. Sanitizing user input in such a way that it can be used directly in an SQL query is incredibly difficult, and really shouldn’t be done. There are best practices to ameliorate this problem, such as using prepared statements. Rails also helps by providing powerful query tools that make writing SQL a rare necessity. But when you’re using libraries that compose and execute SQL for you, how can you ensure that those libraries are following best practices?

Resolving the Issue

To audit Squeel, we built a very simple Rails 3.2 application with a model that was intentionally built to be vulnerable to classic SQLi in a scope, for example scope :name_eql_sqli, lambda { |param| where("name = #{param}") }.

Immediately following the vulnerable scope, we would include a version that is mitigated in the standard Rails best-practices fashion, for example scope :name_eql_no_sqli, lambda { |param| where("name = ?", param) }.

Finally, we would add a Squeel-provided scope we hoped to exploit, always ambitiously called _squeeli, for example: scope :name_eql_squeeli, lambda { |param| where{name == param} }. In hindsight, perhaps the latter should have been called _squeeli? or _maybe_squeeli.

SQLi is very much a database-specific practice; for many vulnerabilities, an attack that exploits one site won’t work on the exact same site using a different back-end database. As such, we focused the audit on Postgres because it is used by so many sites and hosting platforms. Being that we love Test-Driven Design so much, the natural route was to use RSpec in the hopes that a tight testing loop would allow rapid discovery of any vulnerabilities. You can find that test suite here.

After many iterations of trying to find a vulnerability, the audit seemed to be a success–with the minor setback that it wasn’t working. We used Postgres’ excellent logging functionality via postgres --log_statement=all, and it seemed that Postgres was reporting that a bad query was hitting the database. After copy/pasting that query into the console, we received the (bad) results that we expected. That Postgres was apparently getting what we wanted is great, but back in the Rails app ActiveRecord was not returning the results we had hoped for. After much toil in an effort to get the desired results, we ultimately decided to report to the author and get his take on the problem.

It’s no secret that we love Github because it makes software development a true joy. Only in this process did I find a small shortcoming in using it. When Highgroove find a bug in software, we open an issue or even a pull request with a patch (though many software maintainers prefer that the issue be opened first, and that a pull request be made only after they’ve approved the fix or change).

But what do you do when the issue may have security ramifications not just for you, but for all users of that library? Posting a vague “your software has a security vulnerability” issue seems next to useless, and posting a full proof-of-concept exploit to the issue exposes all users of the software to compromise. Some variety of “responsible disclosure”/private flag for issues would have been awesome, but since we didn’t have one, I had to try something else.

Reaching out

I decided to reach out to Mr. Miller directly. Other Highgroovers have chatted with him on IRC, and he is available on his Twitter, Google+, LinkedIn and Github accounts, in addition to his personal website. Mr. Miller was extremely receptive to hearing about what we had found, and he examined it carefully. In the end, it appeared that the Postgres logs were giving us a false sense of accomplishment by indicating that we had found a vulnerability, but the query actually was escaped and sanitized correctly before getting there.

Mr. Miller’s openness to the possibility of an issue spoke to his pride as a software craftsman. That quality alone gives us additional confidence in using a gem which, if written without care, could cause immense problems for us and our customers. Now we can use the gem confidently, and we’re freed to focus on our own process while using it.

How do you reach out to software developers with sensitive questions? Do you have a protocol in place for responsible disclosure in your security process?

Image credit: francescomucio

The post Reaching out to the community in pursuit of "Responsible Disclosure" appeared first on Big Nerd Ranch.

One of the most amazing benefits of working at Highgroove is the yearly conference requirement. You are required to go to a conference of your choosing–on the company’s dime–and report back with what you learned. What’s even more amazing is that the conference you choose doesn’t have to be something Ruby-centric or Railsy (though we do love those conferences too). Read on to see how I conned the company into letting me attend two conferences during my yearly outing in NYC.

Summercon is a fascinating security conference. It is a study in contrasts when compared to larger well-known conferences such as Blackhat, Defcon, or even CanSecWest, ReCon, or the bevy of new conferences that have sprung up in the past few years. The main difference between Summercon and the others is its size. Hosted at Littlefield NYC in Park Slope, Brooklyn, the conference has one track, with few but very carefully chosen speakers, and fewer than 200 attendees. The Littlefield is a concert space, not a conference space, and that small difference makes the event feel like a get-together with some of the smartest researchers in the field, instead of a capital-C “Conference.” Big conferences with a focus on multiple tracks, trainings, and networking events can be great, but many of us learn more when the emphasis is on talking with peers, instead of sitting and listening to experts.

The speakers at Summercon are very carefully chosen and the topics are of a highly technical nature, often with a focus on offensive technologies rather than defensive ones, and in particular the low-level details of offensive tools used in computer security today. Highlights of Friday’s round were Jon Oberheide and Charlie Miller’s talk on Google’s Bouncer program, a system of vetting Android applications before they hit the store, and Dr. Raid and Aaron Portnoy’s discussion of their new plugin for IDA Pro that makes reverse engineering a much less painful process. The talks were not only hilarious, but they also showed brilliant research and methodology or great engineering. In many ways, Summercon feels like two straight days of Highgroove tech talks, and to be honest, that sounds like a load of fun!

Then I cheated. I went to another conference while I was in NYC. Loren Norman of MailChimp was in town to attend Barcamp NYC, and he turned me onto this entirely different event. BarCamps are “un-conferences,” meaning that attendees choose topics and self-organize what effectively become working groups on the selected subjects. I helped organize a security group with topics ranging from cloud security to hacker methodologies to how we secure our own particular stacks on various projects. A track on code review was then proposed, and most of the attendees from the security group found themselves there as well. The code-review session was a fascinating chat in which we discussed tools and methodologies for review, as well as our motivations for reviewing. I was surprised to see that even those working in “enterprise” environments using large, heavyweight design process were interested in integrating a simple code-review process–a step in the right direction, I think.

The sponsors at BarCampNYC were extremely generous, providing breakfast, lunch, and a pre-party–and, as a side effect, a fantastic opportunity to meet like-minded developers in a variety of fields. I met many people who call themselves founders but are also lead developers, CTOs, CFOs, and office managers all rolled into one. As someone who is strictly a developer, meeting such driven and interesting people always motivates me to hone my craft and expand my skill set.

Thanks to Summercon’s leisurely Saturday schedule, I was able to duck out of BarCampNYC and arrive only a little late back at Summercon. That day, there was a great talk on very low-level radio hacking from Travis Goodspeed, as well as an amazing presentation by Alex Sotirov on the cryptographic attacks on the Microsoft Update service used in the recently discovered Flame worm.

While these topics are pretty far from Ruby and Rails, web-based API development, and our other specialties at Highgroove, the way we think about engineering is not far from the ways that security researchers think about building and breaking robust systems. If anything, the slight difference in domains forces a non-expert to think just far enough beyond their comfort zone to learn more than if they were talking to a peer on matters about which they are completely comfortable. Summercon and BarCampNYC are both fantastic events because the attendees put so much into making them great!

When choosing your conferences what do you look for? How do you make the most of your conference outings?

Image credit: apreche

The post Conference Roundup – Summercon 2012 and BarCampNYC appeared first on Big Nerd Ranch.

For those following along at home, you know that we love to work iteratively, and as such, work towards our big-picture goals one small step at a time. We also take software quality seriously, and an important part of software quality, particularly when it comes to web applictations, is security. We’ve begun to develop a new process to address security in our applications. In the process we also want to contribute back to the community where we can because it will make the entire web ecosystem that much better. After the break we’ll talk about our high-level process and how we implement it.

Our Process

At Highgroove we take application security very seriously. The good news is that Ruby on Rails helps developers write secure applications if we simply follow the well documented best-practices in the field. As a side effect it is very difficult to introduce the majority of security vulnerabilities that plague other platforms and frameworks. There are trade-offs of course, and as applications become more sophisticated the more powerful the tools that are brought to bear on the problem. With that sophistication comes the potential for security issues of great subtlety to creep in. To prevent this we perform security audits each iteration to catch such subtleties before they are put into production.

As a general overview, we audit the most common vectors for attack on a Rails application:

1. We start by using the excellent Brakeman scanner to give us a general overview of the state of the application. We use this scan to determine any hotspots to analyze. We really like Brakeman, have given a tech talk about it, and contribute as often as we can!

2. Audit the use of model attributes for mass-assignment vulnerabilities, even when attr_protected and attr_accessible are in place to prevent mistakes in granting too much access in a given white or blacklisting effort.

3. Audit the use of scopes and database access throughout the application to prevent SQL Injection.

4. Audit authorization and authentication controls used through the application.

5. Audit sessions for their use within the app to prevent user manipulation or attacker compromise.

6. Audit any extra-database storage mechanisms, such as S3, used throughout the application for potential data leaks or incorrect permissions.

7. Report all findings in an open format such that the customer(s) and developer(s) see and understand the report. This way they can integrate the process of fixing any vulnerabilities in their standard agile workflow. So far we’ve had good luck with creating Github issues with the report as a dialog can be opened addressing the given points. We can also deliver in other ways if the customer would prefer.

8. If any issues are found in either our audit tools, or gems we audit in the above process we either create patches and pull-requests within github, or if we cannot quickly do so, or need the original developer or maintainer’s input, a bug-report (usually in the form of Github issues) in an effort to contribute back to the community.

Our process is constantly evolving based upon our own understanding of the attack space, and with the evolution of tools, both offensive and defensive. As a side effect please fork our process on not-so-secret-sauce and contribute your techniques so we can all get better. If “software is hard”, then security definitely is.

What techniques and tools do you roll into your workflow to help ensure the security of your customers and their data?

Image credit: m thierry

The post Evolving a Security Auditing Methodology appeared first on Big Nerd Ranch.

We love giving presentations. We sponsor the Atlanta Ruby Users Group (ATLRUG), we each attend at least one conference per year where we are highly encouraged to speak rather than simply attend, and we give weekly tech talks at the office. Each Nerd has their own presentation tool-stack ranging from extremely simple using nothing more than a web browser with various tabs they manually rotate through, to sophisticated Keynote slide-decks. One can also use a very editor and CLI-centric collection of tools to build flexible presentations using many of our everyday tools like git, guard, slim, and Github with Github Pages. If you like using those sorts of tools to speed things along read on after the jump.

Deck.js is an extremely polished HTML/CSS/Javascript slide-deck creation tool. It provides functionality to theme and control your slides in a very granular fashion. Slim is one of Highgrooves’s preferred templating languages for its minimalism and expressiveness. It empowers developers to write correct, well-strucutured semantic markup without the overhead of unnecessary angle-brackets or closing tags. When developing Rails apps Slim is automagically compiled down to HTML but when working on non-Rails projects such as a plain-ole Ruby app or a slide deck it is very handy to have Slim automatically compiled rather than having to run slimrb after every save. To enable that compilation Guard and Guard-Slim work well together to produce output that can be continually reloaded in a browser without ever having to manually intervene. For publication Github pages are extremely convenient. For a given project you can simply create a gh-pages branch and push an index.html to the tracked remote gh-pages branch, and it will be published automatically (for more information see the excellent documentation).

To wrap up this whole process into a single generator command I’ve created a ruby gem Presechute, “The parachute for your presentation creation process.” It can be run by installing the gem via gem install presechute then simply running presechute new <presentation name>. It downloads deck.js, cleans up the environment a bit, and sets up the whole family of tools discussed above. It then creates a gh-pages branch, removes HTML files from the .gitignore file, and creates an initial index.html file ready for pushing to the remote branch with a rake task. Finally it checks out the master branch again and prepares to get down to some serious slide-deck creation!

From there the standard workflow is to launch guard via bundle exec guard (if you use Bundler), open your new presentation file, for example prezzy.slim, and write some slides! After each save prezzy.html will be updated by guard-slim, and you can view it at your leisure in your browser. This makes for a very tight feedback loop while polishing slides. When you’re ready to release simply commit your work, then hop over to the gh-pages branch, rebase against master, and run rake to generate your index.html. Commit it as well and push it upstream and you have a working online slide-deck.

What tools do you use for creating web-ready slides? How do you polish your workflow to keep a tight feedback loop?

Image credit: Grand Canyon NPS

The post Slide-decks the Web 2.0 way appeared first on Big Nerd Ranch.

At Highgroove we really like testing and are constantly looking for ways to improve our testing process, how quickly our tests run, and how exactly we execute our tests. How often during your Test Driven Development (TDD) cycle do your tests fail “mysteriously”? You’ve written your tests, written your code, and most of them pass but one or two stubbornly fail even though you are fairly certain they should pass given the testing setup you’ve provided? At Highgroove we bias towards action so we are likely to launch a debugger session or a pry session to get to the bottom of this. Another approach, which won’t break your existing TDD workflow, is to use your test-suite in place of a more traditional debugger. After the jump we’ll talk about how we’ve been using this strategy to dig into code quickly and easily.

Often the above situation comes about during testing of complicated objects where a given assertion relies on several elements of state being correct. For this discussion the examples will be in RSpec 2 though any test-suite should be able to perform the same assertions with a simple change of syntax.

Assume you start with a simple example such as

If the #compound? example above fails then clearly one of the three properties being checked is not in the expected state. You could do something like launch the debugger, launch the rails console, or bind a pry session and dig into the objects. All of these would work so lets examine the rails console version:

If we had used pry, ruby-debug, or even awesome_print and examined the object manually, we would have almost certainly done exactly those queries of our object to determine state. Since we doing proper TDD we probably have both our tests and our code open simultaneously, and some way of quickly running our test-suite such as vim-vroom or guard-rspec or some other custom set of bindings or tool, so why then do we context switch into yet another tool to debug object state when RSpec is essentially a DSL for doing just that?!?

Thus we can change our spec to do our custom querying as follows:

which produces:

In all likelihood we want to change our object creation step in FactoryGirl for this test of the predicate. We can make that deterination and change in one place without context switching, and decide to leave or remove those predicate checks before moving onto the next story or riding bikes. What do you do to debug or otherwise investigate the state of your objects without breaking your flow?

The post Cast aside that debugger and use your test-suite appeared first on Big Nerd Ranch.

Meetings. Quick chats with our customers, daily if possible, don’t count as meetings. They keep everyone on the same page to make sure good progress is being made for them, to address any questions or concerns they may have, and possibly to plan the next steps to take in developing their app. By “meetings” we mean that word that brings dread to the heart of anyone who has ever looked at their schedule and seen only two hours of their daily schedule available for actual work, and in fifteen minute blocks no less.

At Highgroove we have a few tactics for avoiding just that. Firstly, we don’t have many. Moreover, all meetings are optional, and you can come and go as you like. A quick glance at my calendar shows vastly more pairings, personal trainer sessions, hack nights, team dinners, and other informal team outings than it does meetings. We consider this a win. Like all companies we still need some variety of status updates, the kind of subject matter that would usually go into a weekly company or organization meeting. To address that need we “bring it in” for the weekly huddle emails.

From our own documentation on one such huddle “This is a quick huddle-style meeting – not a working meeting. Just a high-level, decision-making quick meeting focussed on removing impediments and moving forward.” To facilitate those goals only the absolutely necessary parties are involved. Anyone is welcome but very often developers are not involved in the sales process and would rather get tacos than talk sales and marketing numbers.

On the business side of the house our huddles have a fairly strict script. The list of bullet points that make up the script can be answered effectively with little analysis, and decisions can be made. This eliminates that weekly PowerPoint snooze fest many are familiar with. To keep the rest of the team in the loop during their taco or Cherri-induced coma, an email is sent out with details from the huddle. Again, in the interests of brevity and focus, the script is copy/pasted into the email with any salient details on each one quickly described.

On the developer side of the house the process is almost identical but with slightly more developer-centric information. The content is entirely collected by email or our tracking tools like PivotalTracker, Reflecticle, and LetsFreckle, and allows everyone to have a feeling for what’s going on around them. After reading the huddle email we know who’s blogging and giving tech-talks this week, and on what subjects, who is on vacation, what clients are saying on other projects, and what special events are coming up. All-in-all it may take some time to compile, in small time slices, but it takes just a few minutes to read at your leisure without ruining your workflow, the one you always get into just before that big meeting…

What do you do to eliminate meetings and make sure you can be more productive to ride bikes and eat tacos?

The post The anti-meetings – The Weekly Huddles appeared first on Big Nerd Ranch.

.

Open-Source software makes all of our lives better. It allows us to do our jobs without reimplementing the wheel each time. It allows us to understand the nuts and bolts of how the software we use operates … and even change those internals if we need to!

Open Source is a two-way street, however. While we aren’t legally bound to contribute back to the software we use, we like to do it out of a sense of community and because we know how much open source has helped us solve big problems. It’s also good for business, of course, since it increases Highgroove’s visibility if other developers are using software we helped create.

Motivation

Scratch an Itch

Many of our developers contribute to open-source projects regularly. Often their involvement in a given project arose from a need such as a feature lacking in a toolset upon which they already relied.

Fix a Bug

Bugs in software are a reality, but also incredibly frustrating for users of open-source libraries. If you run across a bug in a library that you are using, do the community a huge favor by tracking it down, making a bug report, and ideally, fixing it!

Follow all of the same guidelines as if you were adding a new feature (see The Mechanics section below).

If you just happen to have an interest in a project you’d like make better, check the bug-tracking system or issues wiki (on Github) and fix it; even if it doesn’t help you today, you never know when it will down the road.

Adopt Abandonware

Does your favorite gem for doing something only support Ruby 1.8 and break horribly on 1.9? Do its tests fail even if library itself apparently works? Has the original author apparently moved on, or even explicitly given up support?

This can lead to a tricky situation of many forks, all of which fix different, or even overlapping bugs, and each of which is vying to become the definitive new version of an old project. Attempting to coordinate that logistical problem is beyond the scope of this guide, but is definitely a pitfal to be aware of.

In the event that you’d like to help revive a project and there is at least one quality fork, contribute to it. If there is not, make one yourself and invite others who have forked the project to contribute!

Do Something Fun!

Do you have an interest in some field that you can learn more about by developing tools? Graph-theory, artificial intelligence, mobile apps, distributed computing, Linux kernel drivers? Hot frameworks and languages like node.js and erlang?

Take the time, learn the tools, and if you find them lacking … make a small contribution! You could even roll your own if you’re breaking new ground.

Example Contributions

Extraction of General Purpose Libraries from Existing Code

Rails itself was extracted from the code that built the first version of Basecamp. If you find yourself writing some code in a project that feels like it has general-purpose appeal, try extracting it out into a gem.

For example, the rector gem was extracted out of code that parallelized a report for a customer.

The extracted library will be easier to maintain because it is separated from a specific project. It will also be a welcomed contribution that saves other developers time. Also, it might save you time if another developer contributes a feature you need back. And last but not least, it makes you and Highgroove look awesome!

Building a gem is not a difficult process, so even the smallest of gems will be welcomed additions to the community. Seek out other Highgroovers who have built gems before if you want to give this a shot; you might be surprised at how easy it is.

Documentation

Many open-source maintainers took their first steps on a project by simply improving the documentation. Documenting some code or a common task when using software is great way to learn the internals of the a project. It sounds really mundane, but if you are using a library and realize its documentation is lacking, it would be a great service to the community if you contributed documentation back. You’ll help allay a lot of developer frustration and learn a ton along the way.

To make a documentation contribution, figure out the specifics of how documentation is handled for a given project. Sometimes documentation lives within the codebase itself; other times, it lives in an associated Wiki.

Examples:

• The Rails Guides are maintained in the lifo/docrails GitHub repository. There is an open commit policy: anyone can simply clone the repository directly and commit/push to it! This is the easiest way to contribute back to Rails.

• The documentation for the popular authorization library CanCan is maintained in the associated GitHub Wiki for the project. Project Wikis can be accessed by finding the project on GitHub and clicking “Wiki” toward the top right of the page.

• Documentation for code structures is usually maintained alongside the code itself. The Rails API docs are generated from code comments in RDoc format (for an example, see ActionMailer::Base and its generated form). Again, anyone can contribute to these docs by committing to lifo/docrails.

The Mechanics (Existing Projects)

Github

Ryan Bates’ Railscast #300 on Contributing to Open Source covers all the details on fixing bugs and adding features to projects hosted on GitHub. It’s well worth the 9 minutes!

Doing it by hand

If the project resides in another versioning and distribution system such as Sourceforge, or even as a tarball somewhere out there, you’ll have to do it the old fashioned way. There is no forking, only downloading, and if the project does use revision control but not Git you’ll have to change your workflow to match. In the case that the project uses Subversion you can use the git-svn bridge locally but keep in mind that the natural workflow of the project owners will very likely be different from the one to which you are used.

The general workflow is not so different from the Github-centric one above, but you will have to be in greater command of your tools:

1. Retrieve the code
   * Check it out of an SCM (SVN, Mercurial, CSV, RCS!?)
   * Download the source tarball
2. Optional Import the code into a local Git repository to manage feature branches
3. Make your changes to your local code, along with tests if possible.
4. Contribute your patch:
   * If you don’t have commit access to the original SCM (if there is one), which is likely during the beginnings of your interaction with a project, create a patch using diff that the author can easily analyze for merging. Be sure to introduce yourself, include a detailed (but not too detailed) explanation of your bugfix or feature, and make yourself available to answer any questions the maintainer may have for you.
   * If you do have access then contribute your patch per the process of the project. That may involve creating a remote branch for review or other project-specific processes. Be aware that if you utilize git locally with a bridge to a different SCM, local commits may have non-obvious effects (such as creating a new revision number in SVN) that can be prevented by rebasing before committing. This is no different than the git best-practice you already practice but the effects of missing this step here are more obvious than in the git-specific case.
5. Make yourself available, exactly as before.

Wrap-up

As with any project the key is to make your contributions something well tested, within scope, and easily understandable by the recipient. Moreover by being communicative and available you will find your contributions much more likely to be accepted by the project one which you worked, then it’s time for an Open-Source adult beverage!

The post Our Open-Source Methodology at Highgroove appeared first on Big Nerd Ranch.

It’s easy to say “We’re agile” and “We use Behavior/Test Driven Development” and thus “we use the right tools to empower our developers!” but what are those tools? For me that discussion is entirely about the tool stack you choose, how that stack empowers you as a developer to do things right the first time. Luckily thanks to the ruby community as a whole we have a large number of high-quality choice to choose between.

Generally when we talk about TDD and being agile, beyond the process of choosing features to deliver the maximum customer facing value, we refer back to the simplest possible workflow of “Red/Green/Refactor.” Within that workflow you write tests that fail, make them pass (with the focus on getting that done without polishing), then refactoring, to polish where necessary. The subject of testing is a huge one with a large body of prose already written on the subject. Rails Test Prescriptions and The RSpec Book are fine examples of such, along with a huge menagerie of blog posts, podcasts, and screencasts to choose from.

I propose a slightly different day-to-day workflow that I see used to great success.

1. Hack around and choose your course
2. Red – write some tests that fail but will pass when your code meets your needs
3. Green – the code is written, the tests pass, life is good.
4. Refactor – Don’t Repeat Yourself (DRY your code out). Or don’t, if Ya Ain’t Gonna Need It (YAGNI).
5. Continuous Integration
6. PANIC OVER SECURITY
7. Ride Bikes

This isn’t the “pure” TDD process, but one that just adds a couple of unspoken facets of the process that are already used widely, just don’t fall into the pithy “Red/Green/Refactor” process.

Ruby has a functional programming feeling with its composition of methods, distinction of methods with side-effects from those that do not (via the ! suffix), and it comes with some variety of REPL via IRB (or the new awesomeness in Pry). This allows us as developers to spend some time touching code and experimenting before we decide exactly how we might like a function to operate, and thus before getting into the main development workflow for a given feature.

Tools to support step 0:

• Irb – you already know about this but it’s easy to overlook.
• Pry – the new slickness that is 1-part Irb replacement, 1-part debugger, and a pinch (or three) of awesome. Watch the screencast to get a feel for it!
• good-ole-pencil-and-paper, a whiteboard, some origami paper
• whatever tool gives you a feeling for the right way to proceed without being trapped in analysis-paralysis.

Once you’re ready to write some tests that will fail, you likely already have a test framework of choice, and even a workflow for running those tests. You may run them from within your editor, have a terminal window ready, willing, and able to run them with a single command (and a shortcut into your history like ctrl-R), or you may let something else automagically run them for you and report back.

At Highgroove many of us find a combination of running tests manually and automatic test running and reporting to be most useful. To accomplish this I use the testing framework selected for the project along with Spork and Guard. Spork is a test server that constantly keeps your rails code loaded and ready to be tested rather than loading each time in a rake task. Spork should be your first step to speeding up slow-running test-suites. Guard is Spork’s right hand man. It watches code for changes on disk and automagically runs the correct associated tests. Guard will even notify you via Growl, Libnotify, or your other notification framework of choice that you’ve broken things, or that everything is still hunky-dorey and you should keep hacking away and/or ship!

When I run tests manually I often want to run a single test, or set of tests manually. Using Rspec2 you can accomplish this with tags or calling the specific spec by line number via bundle exec rspec <path to spec>:<line no. of spec to run>.

Testing is not only a matter of making your tests all turn green. Poorly written tests can certainly be green and still not indicate that your code is working as intended. Perhaps you simply forgot to include any assertions (I’ve certainly never done that) or you’re testing your factories rather than your models (never ever once again). Code quality tools can help you with that as well, and it’s not entirely certain as to whether they fit in the “Red/Green” phase of development, or in the “Refactor” phase. I like to split the playing field and say that code-coverage tools belong in the red/green phase as they allow you to check the quality of your tests which firmly live in “red/green” while other code quality tools (such as static analysis tools for cyclomatic complexity) belong in the “Refactor” phase. We’ve discussed code-coverage in Ruby before so have a look there for a refresher, but if you’re not using one of these tools it may be time to roll it into your workflow. Cover-me even has a nice post-test HTML coverage report to show you how well you’ve done. This allows a very tight development loop to make sure the tests are covering the right things the first time.

Tools to support steps 1 and 2:

• Testing frameworks: MiniTest, Test::Unit, Rspec, Cucumber, etc.
• Spork and Guard
• Coverage tools: cover_me, SimpleCov, rcov (for Ruby 1.8 apps)

So you’ve got good tests, good test coverage, everything is green, and you’re ready to refactor. If you are struggling to start your refactoring process then either it’s good enough as it is or maybe you need some inspiration. Sometimes your code just doesn’t need to be refactored. It is easy to fall into the premature optimization problem and abstract/refactor/clean your way into a corner that you will undoubtedly undo later when the customer asks for more or different flexibility. So maybe just delay refactoring if there isn’t an obvious need. In the case of needing some hints as to where to refactor the Rails Best Practices site has you covered.

There is no replacement for a good set of eyes on the code and intuition as to where you can refactor. To this end Highgroove’s internal code reviews go a long way to getting our code refactored quickly and elegantly.

Since we are not constantly reviewing or pairing there are some tools to help us along including the excellent Flog and Flay, a pair of tools to find complex code and duplicate code, along with an interestingly macabre theme. Another tool is Excellent which produces warnings about “smelly code” via static analysis. Finally we have the big kahuna: Rails Code QA. This package wraps up other code quality packages (including Flog and Flay) to make a single rake task that does it all.

Tools to support step 3:

• Flog and Flay
• Excellent
• Rails Code QA

At this point you have your local code automatically being tested, you’ve run the tests you need to focus on, you’ve refactored with the help of tools, the rest of your development team, and the community’s collective knowledge. If you are working on a feature branch, that branch needs to be merged into a staging or production branch, and hopefully tested before being pushed to client or customer facing environments. This is where Continuous Integration (CI) comes in.

A CI environment is constantly running all of your tests on whatever branches it needs to in order to make sure that any merge artifacts don’t make their way into production. You may choose to run your own CI environment via Jenkins or Goldberg, both of which are lightweight and well supported, or you may choose to “outsource” your CI to a service. Travis-CI has gotten a large following in the Ruby community thanks to its super simple setup, integration with Github, and quality service. I was recently introduced to Tddium by a co-worker while struggling to setup an internal CI system for a customer. It looks very well backed and supports pretty much all of the services we employ day to day. If you don’t want to do operations to make sure your CI is up and running then looking to one of the latter services is a great route.

Tools to support step 4:

• Local CI tools: Jenkins, Goldberg
• Hosted services: Travis-CI, Tddium

Great! Your code is tested locally and in CI, everything looks solid in terms of functionality, and Rails automagically does everything right for security. It’s time to go ride bikes right? Well maybe not. There is a comprehensive and excellent Rails Security Guide because it’s not so simple. This isn’t to say that Rails security is as hard as it could be, but as with everything getting it right requires knowing and following the best practices created by the community.

Luckily for us the Brakeman Scanner project integrates beautifully with the tight testing workflow described above. Brakeman is a static-analysis based security-focussed scanner aimed at Rails applications. It is under active development and the progress has been phenomenal. We did a brief tech-talk about it recently and to say we are enamoured of it may be an understatement. Even a couple of the bugs described in the tech-talk have been fixed since then! Combining Brakeman’s fantastic scanner and our automated testing tools works out of the box in two ways. Firstly there’s Brakeman’s reporting which is excellent and could certainly be wrapped in a rake task to create a cover_me style report after each test run. Secondly there is guard-brakeman for local-scanning and testing as well as notifications via your system notifier when new security issues are introduced, just as if a test failed. For CI environments Brakeman works with Jenkins out of the box and adding support to other CI environments should be a fairly simple operation.

Tools to support step 5:

• Brakeman, Guard-Brakeman

Making your development and testing loop as tight as possible allows you to have confidence that your code has been written in a process that promotes testing, coverage, and automated quality tests so that you can focus on what’s important: shipping code and riding bikes.

The post Red, Green, Refactor – The Tools For Success appeared first on Big Nerd Ranch.