Boy van Amstel

Boy's nifty stuff, or whatever..

Hip Like Heroku

Usually I deploy projects on my physical server, located at the server park called “former bedroom at my parents’”. This allows me to see how a project develops and migrate to a better server (which costs more money) if it’s successful. This is also the case for hiplikejapie.nl.

I’ve been deploying a couple of experiments to Heroku and the platform really appeals to me. Deploying itself is a breeze and the “add-ons” model is pretty cool. Also, it’s free if you can manage with a 5mb database, 1 dyno en 0 workers. Projects in a developing stage usually do. I’ve never launched a project from it though. Primarily because I don’t want the to run into insane hosting costs if something like Please Rob Me happens. In any case the ability to scale a project and not worrying about server setup and stuff is awesome.

Hip Like Japie hasn’t really took off (yet?), but is doing mildly ok with a bunch of visitors every day. It being a finished product made it interesting for me to see how well it would deploy.

I’ve got Heroku’s command line tool installed, but if you haven’t install it like this:

$ sudo gem install heroku

Next you need to setup a Git repository for your project if you haven’t already.

$ git init
$ git add .
$ git commit -m 'initial import'

Let Heroku add it’s remote.

$ heroku create

Login with your Heroku credentials and you’re done setting up!

This would usually be the point where you start developing your application, but Hip Like Japie being already done allows me to skip directly to deploying.

$ git push heroku master

Did you see what they did there? They just added a new remote, super nice. Anyway, this should start deploying the app. After it’s done, migrate the database.

$ heroku rake db:migrate

I thought about looking online for correct settings to Heroku’s database setup, but decided to just run rake and see what happens. Turns out that Heroku just made it work. My production setup was set to use Sqlite3, but Heroku automatically changed it to their PostgreSQL database.

Technically I was done, but opening the website would give me an error. Crap, something went wrong. You can check the server’s log by issuing the following command:

$ heroku logs

This showed me that a database query failed, due to running on PostgreSQL instead of Sqlite3. I applied a dirty little patch.

if ActiveRecord::Base.connection.instance_values["config"][:adapter] == 'postgresql'
  @comparisons = Comparison.find_by_sql(
      "SELECT * FROM (SELECT DISTINCT ON (username) * FROM comparisons ORDER BY username, id DESC) foo ORDER BY id DESC LIMIT 10"
      )
else
  @comparisons = Comparison.all(
      :order => "id DESC", :limit => 10, :group => 'username'
      )
end

After that the site worked fine: hiplikejapie.heroku.com. There’s an add-on that allows you to hook up your custom domain name to the project. It’s conveniently called “Custom Domains” and can be used via the command line.

$ heroku addons:add custom_domains

This may prompt you to verify your account, by entering your creditcard details (on Heroku’s website, not the command line). After that you’re ready to add your domain names.

$ heroku domains:add www.hiplikejapie.nl
$ heroku domains:add hiplikejapie.nl

I added both the domain and the www. sub-domain. To activate this, you need to change your domain’s DNS settings. It comes down to adding three A records and a CNAME.

@ A 75.101.163.44
@ A 75.101.145.87
@ A 174.129.212.2
www CNAME proxy.heroku.com.

That’s it, hiplikejapie.nl points directly to Heroku, awesome!

EDIT:

The website failed if the worker would spin down. The error logs showed that Compass couldn’t compile the Sass files. Which is obvious as Heroku is read-only. This line in config/environments/production.rb fixes it:

Compass.configuration.sass_options={:never_update=>true}

Improve Javascript Code Quality With JSLint

Created a tiny project on GitHub with a simple piece of Javascript to see what JSLint does.

“JSLint is a JavaScript program that looks for problems in JavaScript programs. It is a code quality tool.” – JSLint.com

Pretty cool and something I’ll definitely incorporate in deploy scripts. Checkout the example website here and the GitHub repo here.

When POSTs Suddenly Turn Into 301 Redirects in Rails

Everything worked fine in my development environment, but when I ran the application from my server every POST would just show the page it was suppost to POST to via GET. Displaying a 301 redirect in Chrome’s developers tools.

After some fiddling around, I commented line 351 in the .htaccess file:

<IfModule mod_rewrite.c>
  RewriteCond %{REQUEST_FILENAME} !-f
  RewriteCond %{REQUEST_URI} !(\.[a-zA-Z0-9]{1,5}|/|#(.*))$
  #RewriteRule ^(.*)$ /$1/ [R=301,L] # <-- this one
</IfModule>

After that everything worked as expected. Weird..

Girrst Displays Your Gists as RSS

Do you use GitHub’s Gists? Then you might like this.

Sharing code among co-workers and friends is quite easy using various ‘code snippit’ sharing websites. I personally like GitHub’s Gists. To make sharing even easier and keep everybody up to date of what’s being shared, I wanted to create a feed out of my Gists. Of course I could just use GitHub’s own RSS feeds, but that would screw up my reason to do something with App Engine.

Using Python and App Engine I threw together a script that will take your GitHub username and turn all your Gists into a convenient RSS feed. You can then use the feed to include it anywhere, like your favorite RSS reader or your blog.

To make things even easier, use the GitHub bundle for TextMate to be able to create Gists from your favorite editor. Don’t forget to setup your GitHub username and token:

$ git config --global github.user username
$ git config --global github.token 0123456789yourf0123456789token

Try it: girrst.appspot.com

The Delicious API Access Over Yahoo OAuth in Ruby on Rails Adventure

I spent a little time today getting Yahoo’s OAuth implementation to work with Omniauth. This seems pretty straightforward, but it isn’t.

First, what I tried to accomplish:

  1. Have a user sign in with his/her Delicious account
  2. Use the Delicious username to retrieve a bunch of feeds

Step 1 might seem unnecessary as Delicious’ API only requires authentication to post/update stuff, but I want to store some user settings and stuff. OAuth allows me to use their existing data, instead of creating my own registration process.

I continued to setup a project in rails, init a Git repo and fill my Gemfile with the gems required for using omniauth plus some extra:

Touch omniauth.rb in config/initializers and the basics for using the Yahoo OAuth provider. Yahoo isn’t supported by default, so we’ll be creating our own strategy later in lib/oauth-strategies/yahoo.rb.

module OmniAuth
  module Strategies
    autoload :Yahoo, 'lib/oauth-strategies/yahoo'
  end
end

Rails.application.config.middleware.use OmniAuth::Builder do
  provider :yahoo, 'consumer key', 'consumer secret'
end

Visit Yahoo’s developer page and create a new project. Choose standard and fill out the rest of the form to match your application. Pay extra attention to Application Domain, enter a url that’s accessible to the outside world. You’ll see why when you’ve completed the form. I’m building a Delicious app so I specified that I require extra user data and choose to have read/write access to Delicious.

When you press the Get API Key button, you’re required to upload a file to your server, accessible via the url you’ve just entered under Application Domain. There are three buttons, one of which falsely suggests that you can skip this step. Pressing it will only tell you that you have to do it anyway.. nice. This still gave me the impression that this the whole Application Domain thing was optional, though. Mistake.. I’ll tell you why in a minute.

Alright, so we’ve completed the form and are presented with our consumer key and consumer secret. Add them to the omniauth.rb configuration.

Create the folder lib/oauth-strategies and touch yahoo.rb. I tried the default OAuth implementation, didn’t work. After some research on GitHub. I found that there was already a pull request waiting with a Yahoo OAuth strategy, neato. Paste the code into your yahoo.rb strategy file. At this point I was pretty confident that this wouldn’t be a hassle after all. Run bundle install and start your server. Now visit the yahoo auth url at http://localhost:3000/auth/yahoo. The first thing you’ll run into happens when you’re an oldskool Delicious user. This means you’re account is not yet hooked up to your Yahoo account and you’ll get an error saying that you’re login is incorrect. After merging my Delicious and Yahoo accounts, I hoped I had fixed the issue and tried again. Aaaand BAM: “401 Forbidden”. Stumped me at first, but after checking the full response Yahoo told me this: “Custom port is not allowed or the host is not registered with this consumer key”. So I thought I was clever and started the WEBRick server on port 80. No effect, same error.

This is when I remembered the Application Domain field.. Yahoo apparently does not like you developing on your localhost, while every other OAuth provider works fine, fuck. So, I setup the whole thing on my server and ran the thing again. Success! It redirects to Yahoo and asks for permission. The callback doesn’t go too well, though. Use this guide to add the route to a custom callback, even though it’s not working yet.

I started debugging by overriding the callback_phase method and checking out where things started failing. The standard callback_phase method uses name.to_sym to get the name of the provider (yahoo in this case). This didn’t work, so I hardcoded it. Calling super also caused trouble. Long story short, it needed a lot of tweaking.

Eventually I got it to work, but noticed that the nickname Yahoo OAuth returns is not the one from Delicious.. crap! Luckily Delicious adds the username to it’s authenticated post feeds. Allowing me to grab it and add it to the user profile. The full yahoo.rb now looks like this:

module OmniAuth
  module Strategies
    class Yahoo >; OmniAuth::Strategies::OAuth       
    unloadable       

    def initialize(app, consumer_key, consumer_secret)         
        super(app, :yahoo, consumer_key, consumer_secret,               
              :site               => "https://api.login.yahoo.com",
              :request_token_path => "/oauth/v2/get_request_token",
              :authorize_path     => "/oauth/v2/request_auth",
              :access_token_path  => "/oauth/v2/get_token"
        )
      end

      def callback_phase
        request_token = ::OAuth::RequestToken.new(consumer, session[:oauth][:yahoo].delete(:request_token), session[:oauth][:yahoo].delete(:request_secret))
        @access_token = request_token.get_access_token(:oauth_verifier => request.params['oauth_verifier'])

        @env['omniauth.auth'] = auth_hash
        call_app!

      rescue ::OAuth::Unauthorized => e
        fail!(:invalid_credentials, e)
      end

      def auth_hash
        OmniAuth::Utils.deep_merge(super, {
          'uid' => @access_token.params[:xoauth_yahoo_guid],
          'user_info' => user_info,
          'extra' => {'user_hash' => user_hash}
        })
      end

      def user_info
        user_hash
        profile = user_hash['profile'] || {}
        username = Nokogiri::XML::parse(@access_token.get("http://api.del.icio.us/v2/posts/recent?count=1").body).root['user']
        {
          'nickname'    => username,
          'name'        => "%s %s" % [profile['givenName'], profile['familyName']],
          'location'    => profile['location'],
          'image'       => (profile['image'] || {})['imageUrl'],
          'description' => nil,
          'urls'        => {'Profile' => profile['uri']}
        }
      end

      def user_hash
        @user_hash ||= MultiJson.decode(@access_token.get("http://social.yahooapis.com/v1/user/#{@access_token.params[:xoauth_yahoo_guid]}/profile?format=json").body)
      end

   end
  end
end

Alright, everything is setup to work marvelously with ASCIIcast’s guide, which explains how to allow users to sign in with OAuth and remain logged in until they decide to sign out. Thanks to Yahoo for adding some pitfalls (some of which I probably forgot to describe) here and there to keep things interesting.

*note to self: install pretty code plugin for WordPress.. EDIT: Done!

WordPress Project Creator

Working on WordPress projects can be a hassle. Especially if there’s more than one developer. There are a lot of files and chances are huge that you’ll be working on the same files quite often. This slows down development tremendously, not to mention the annoyance it causes.

You can fix the ‘working on the same file’ and ‘working on the live server’ issues quite adequately by using version control. I prefer Git, because of it’s distributed character. WordPress has another issue however, when trying to run the same site on different machines. Among the data it stores are urls, absolute urls.. This means that if you’d load a database dump from a live WordPress site into your local install, nothing will work correctly.

WordPress Project Creator [github] is a tool I created to streamline working on WordPress projects with multiple people. It does a couple of things:

  • Download WordPress
  • Create a new Git repo out of your wp-content folder or clone an existing wp-content folder into the WordPress folder
  • Easily import database dumps by alterering absolute urls
  • Create wp-config.php file
  • Create .htaccess file

Adding new developers to your project is as easy as allowing them access to the Git repo and running wpprojectcreator. If you clone an existing project, the only things you’ll have to do is run the python script, import the dump using setup.php and you’re ready to work on the website as if it were live.

I released it on GitHub so everyone can give it a go. Or everyone.. at least everyone running OSX, Linux or 32bit Windows. Git on 64bit Windows can’t be installed in such a way that it’s accessible from the command line, which is required by wpprojectcreator. Be sure to read the README .

Check out WordPress Project Creator on GitHub.

New Internet Technology: Use What’s Already Out There

I’ve had this article on my computer for three months now, but never got around to finishing it. Finally found some time to put it up.

It’s interesting to see that the standard process of thinking of something new, trying to figure out how to make money from it, creating it and then releasing it to the public seems to have changed. A business model does not necessarily come first anymore. A new process could be described similar to this: have an idea, create a very basic implementation and provide an extensive platform for people to use what you’ve created, hand it over to the community, now find a way to make money or hope for a takeover by some enterprise (Google). This makes new technology much more accessible than it’s ever been. A lot of effort is put into making it easy for other people to use and mashup what’s been created.

Detect and use

Needless to say a skill that’s becoming increasingly useful, is the ability to detect these services and find ways to implement them in your own concepts. YouTube, Twitter and Flickr have all been fully embraced by the community, which means the movies, images and messages they store show up on all kinds of applications. Some with a lot easier business model than the services they’ve incorporated. Twitter clients for instance, have a price tag or use adds to turn a profit, while Twitter is still struggling to find a way to make money.

Reasons to use public API’s

Does that mean Twitter is in trouble? Probably not, what they have is data and a lot of people who use that data. Apparently that’s worth multi million dollar investments. Money they need to provide a reliable system and store data the client applications are generating. Which is awesome, because you don’t have to worry about that anymore. It means you could create fairly extensive applications with a minimal amount of effort, as the services provide you with a solid backend.

Maybe the biggest benefit is popularity and content. It’s easier to use an existing user base than to create your own. Let’s say you want to create a website about the Olympic Games. You want visitors to contribute text, photos and videos. This would require quite a lot of work to create from scratch. Incorporating Twitter, Flickr and YouTube would reduce that dramatically. More importantly however, it would generate a lot more content, as it allows your audience to use the tools they’re already accustomed to.

What you have to take in consideration though, is that you’re not in direct control of the content you’re provided with. This may seem daunting at first and it makes the use of services unsuitable for some applications. It takes a new way of moderating the content you’re provided with. Both YouTube and Flickr allow you to only search through video’s and images that are marked safe. Other than that, it’s always a good idea to think about how you can stay in control of your own application.

Start your own projects

The list of services that are available right now seems endless. Some are more successful than others, obviously. Like I said, popularity is a good measure to determine if a service is worth checking out. To get you started, Programmable Web keeps an incredible list of public API’s. I’ve been spending a lot of time with the Google Apps API’s lately, which are pretty awesome and really allow for some cool applications. Especially if you look further than mashing up some of the individual components. For instance, it’s very easy to use Google Spreadsheet as a cloud database. How neat is that :)? Anyway, have fun!

Location Spam, Annoying and.. Risky?

Hey, do you have a Twitter account? Have you ever noticed those messages in which people tell you where they are? Pretty annoying, eh. Well, they’re actually also potentially pretty dangerous. I’m about to tell you why.

Don’t get me wrong, I love the whole location-aware thing. The information is very interesting and can be used to create some pretty awesome applications. However, the way in which people are stimulated to participate in sharing this information, is less awesome. Services like Foresquare allow you to fulfill some primeval urge to colonize the planet. A part of that is letting everyone know you own that specific spot. You get to tell where you are and if you’re there first, it’s yours. O, and of course there’s badges..

Foursquare

The danger is publicly telling people where you are. This is because it leaves one place you’re definitely not… home. So here we are; on one end we’re leaving lights on when we’re going on a holiday, and on the other we’re telling everybody on the internet we’re not home. It gets even worse if you have “friends” who want to colonize your house. That means they have to enter your address, to tell everyone where they are. Your address.. on the internet.. Now you know what to do when people reach for their phone as soon as they enter your home. That’s right, slap them across the face.

To raise some awareness on this issue and emphasize how easy it is to retrieve this information let me introduce: pleaserobme.com. Have fun and please don’t hook up Foursquare to your Twitter account, okay?

How We Created a Twitter Viral

A few weeks ago Barry Borsboom and I had funny idea for a twitter mash-up. We would basically measure your Twitter popularity in the form of a giant penis. Better know as the e-Penis. In order to see if we really created something to talk about, Seth Godin would call this the sneeze-factor, we registered an url: www.epenis.nl and we uploaded our e-Penis app with a few ‘share-this’ buttons. The first button to post it on Twitter, and the second one to Digg it. Would people share this site with their friends? How fast would it spread? All question a marketer would like to know when creating a viral marketing campaign right?

In our first day we managed to reach about 100 of our own friends on Twitter. But to really call it successful, we needed to reach some people on Twitter with a huge amount of followers. We used our own ‘Post on Twitter’ button a number of times on all sorts of people. And after a few tries @michielveenstra (2500 followers), picked up on our message and reposted our website on his own twitter stream on a friday. Overnight we reached 1500 hits. But as it turned out our biggest traffic came from a porn-blog! Apperently a webmaster from this well know porn-blog picked up our site after seeing it at @michealveenstra. In the weekend we managed to get around 8,000 hits, but on monday morning it really took off. Hordes of time wasting desk workers measured their body part. From that point on it really became a trend on Twitter. Our ‘Post on Twitter’ button was used a few times every few minutes now. Because people often measured the popularity off some well known people, they eventually couldn’t help but brag about (cough @stephenfry, 336.599 followers) ;). This caused Great Great Brittian to swarm our website and spread it even more. At this point North and South-America just woke up.

By the way, we were still hosting the website on my server, at my parents’ home. Utilizing almost all of the “massive” 1Mbit bandwith. Thank god my little brother didn’t complain. The reason this was even possible is that epenis.nl has no server-side scripts. It’s all jQuery, HTML, CSS and (small) images. We actually reduced the size of the website by half, by grabbing jquery.js from google.com.

At monday around 4 pm, CET, we finally managed to get the word Penis into the Twitter trending topics. A huge success, because now people twittered about why the word penis was in this list. Generating even more traffic to our site when they found out we were causing it.

Some people saw a connection between Spring Break, which probaly helped accepting the fact the word penis was a part of the list. Twitter however didn’t agree with that. After a few hours we saw some messages from people who were complaining about their twits disappearing, not much later penis wasn’t trending anymore. Did Twitter screw with us here? We can’t be sure. But the damage had already been done, we had spread to the American continent. A couple of big American Twitter users like @MrsKutcher, that’s Demi Moore (257,150 followers) posted about later in the evening, that really generated lots of traffic and added a large amount of females to our users, who seemed to have smaller e-peni. Which actually contributed to the credibility of the website.

The length of the penis most people we’re getting seemed to match what they expected. Which is funny, because we didn’t use a really complicated algorithm. Average people would get averige sizes (between 10 - 20 cm). While very popular twitterers, like celebrities, are showing extremely large numbers.

The next morning we had almost 40k in hits. We called our viral a success, we expected it to die out rather soon. But we were wrong! To our suprise it kept on going. Around 14:00 there was a huge spike in our traffic. As it appeared, @Guykawaski (92,399 followers) was giving a keynote during SES New York about ‘Twitter As A Tool For Social Media’. Lots of people posted live messages during his talk…

@Searchcowboys: Guykawasaki shows epenis.nl and says it’s more powerfull then twittercounter :) #sesny’

And some stuff that made us laugh:

@PaulIAm: Darwin just backed the car up to his grave, connected a hose to the tailpipe and took a deep breath…http://www.epenis.nl/’

@Retecool: There’s lots of stuff out there that figures out your Twitter ranking, value, mojo, etc. But let’s cut the crap it’s all about who’s has the big swinging dick, right?’

@Vatikan: Scheiße, @satan ‘s ePenis ist länger als meiner http://www.epenis.nl/’

At this moment we’re trying to figure out what the hell we’re are going to do with it. To conclude, here is a list of the things we learned:

  • Use Sneezers, find Twitter users with respect and high amounts of followers. (Seth Godin’s Purple Cow)
  • Post your viral at the right time. We had Spring Break, the SESNY Keynote by Guykawasaki and the usual monday morning boredom going for us.
  • Maximize sharability. Our ‘Post on Twitter” button was used extremely well.
  • Make it simple. There are no lists, links, pages. It does only one thing verry well. (KISS: Keep it Simple Stupid)
  • Sex Sells

Thanks for reading, Barry Borsboom & Boy van Amstel