Dev Blog

DuoLingo and Supernatural VR: Stay or Go?

2026-04-07T00:00:00+00:00

There are two popular applications that have had my goodwill, my attention and my subscriptions, that I’ve soured on in the last few months for different reasons.

Supernatural, a VR exercise app, launched in April of 2020. It was founded by Chris Milk, who has music industry connections from his time directing music videos, which meant he was positioned to license popular music set to BeatSaber-style workouts, with human coaches providing (offline) direction. The launch timing was fortuitous, with everyone looking for ways to workout at home during the pandemic. It’s one of the most successful VR subscription apps, with plenty of daily users who swear by it. Facebook aka Meta was so excited about it, they spent a year and a half purchasing the company, including a legal battle with the FTC. I bought my Quest 3 mostly for Supernatural.

Then in January, Meta announced “Supernatural will no longer receive new content or feature updates starting today.” For now, the old workouts are still available; it’s not known how long the licensing deals stay in place. So it’s a zombie app now.

Fans like myself have to decide: are we ok with the existing list of workouts and nothing new coming out? Are we cool giving $10 a month to Meta for this? Zuckerburg told us all to follow the VR dream with him, and even renamed his company around it. Now AI is the new hotness, and VR is being unsentimentally dumped by Meta. (What will their new name be? Clanker?)

For now I’m going to leave my Supernatural subscription in place so I can access the back-catalogue, but I feel like a sucker doing it. Consumer spending sends a message, and I hate sending Meta the message that what they did with Supernatural and their VR effort in general, is ok.

Then we have Duolingo, the language learning app. Duolingo has been around fifteen years or so. They have tons of gamification to keep users engaged. I have a 1035 day streak of doing at least one lesson every day, and a 473 day friend streak with my wife. Honestly though, the streaks are kind of a sham. You can get “streak freezes” in case you forget a day. And I’m pretty sure I was out of streak freezes at least once, and Duolingo gave me a pass, anyway. They’ve also done the opposite: told me I was going to lose my streak if I didn’t do a lesson, when I knew I had streak freezes left. The friend streaks are more effective: if you do your lesson and your friend hasn’t, you can “nudge” them to get it done. I’m not even sure you CAN lose a friend streak, but the peer pressure to do a lesson is real.

Anyway, in March, Duolingo extensively reorganized a bunch of their old Spanish lessons. As a side effect, the app no longer knows what I know and don’t know. It thinks I’ve completed lessons I haven’t, and so asks me questions about words or phrases I never learned. I tried to go back a few chapters to re-do them so I can get reoriented, but the app doesn’t have a way to say “this is where I am now, pretend I never did this chapter and remember I’m here now”. I spent a lot of time and effort getting to where I am in the content, and Duolingo seems to have just invalidated that progress without a second thought. Even though I know in my heart that five minutes a night isn’t going to make me a fluent Spanish speaker, that still feels bad. I’m going to stick with Duolingo for now, but my faith is a lot shakier now.

One Year With Ruby: What I Like and Dislike

2014-11-26T00:00:00+00:00

I came from a long-time C++ background, working in games for many years.

For nearly the last year, I’ve been working full-time building and enhancing a website implemented in Ruby on Rails. One of the major appeals of this project for me was the chance to use and build on what I’d learned doing side projects in RoR.

Here are my thoughts about Ruby nearly one year in.

What I Like

Dynamically Typed

Coming from C++, it was very liberating not having to specify the type of every variable. I overestimated the number of bugs that would result: for the most part, passing the wrong type ends up causing a clear exception pretty quickly, and is probably the easiest kind of bug to track down. And it happens much less frequently than I would have guessed. The tradeoff in terms of reduction in code size has been totally worth it.

Having said that, the code only throws an exception if it gets run. I wonder if some of the emphasis on TDD in the Rails community is to make up for the lack of static type checking, by trying to ensure every code path is run regularly.

Very Expressive

You can do so much with so little code. But unlike perl, the code ends up readable. So readable in fact, that self-documenting code actually seems practical a lot more often.

Having expressive code makes it so much easier to hold a complex problem in your head, when you can see all the related code on one page.

Blocks

Being able to pass a bit of code to functions isn’t new, but Ruby’s block syntax makes it so simple to do that you end up using it everywhere.

Symbols

They’re like C++ enums that you don’t have to declare ahead of time! I’m a big fan of symbols. Yes, you could use string constants instead, but symbols feel so much cleaner and more efficient.

Function Calls Without Parentheses

In many places, parentheses are optional when calling a function in Ruby.

It makes the calling code so much cleaner:

  class Train
    def cars()
      [:engine, :coal, :caboose]
    end
  end
  
  puts train.cars  # So clean!

What I Dislike

Skipping Parentheses With Function Calls Causes Ambiguity

From O’Reilly’s “The Ruby Programming Language”:

  puts(sum 2,2)  # Does this mean puts(sum(2,2)) or puts(sum(2),2)?

Also, because skipping the parenthesis is possible, if you wish to use the parentheses you can’t leave any space between the function name and the start paren:

  sum(1,2) # ok

  sum (1,2) # bad

I started off skipping the parens wherever I could, but I’ve come around to only skipping them in limited cases. They’re great for making:

  a_string.length

look like a property access, for example. Maybe not so great when you’re doing:

  my_object.do_something :now, 3, 'hello'

And just confusing if you’re doing nested calls:

  puts sum 1,2  # Gross!

{} - Block or Hash?

  def func(arg)
    puts arg
  end

  func {} # Ruby thinks I'm passing a block; "I get ArgumentError: wrong number of arguments (1 for 0)"

  func({}) # This actually passes the hash in

This seems like a minor nitpick, but it makes me feel like being able to skip parens, and {} being used for both blocks and hashes, leads to some unintuitive edge cases.

Attributes vs Local Variables

What’s bar do?

  class Foo
    attr_accessor :var

    def bar
      var = 5
    end
  end

That’s right! It creates a local variable called “var” and throws it away!

You have to do self.var = 5. This is kinda crummy because most of the time, you don’t need the self. But in this one case, because calling a member assignment function would look just like assigning to a new local variable, leaving out the self. results in a silent failure that can be a pain to track down.

Unused Blocks are Ignored

It’s easy to do this by mistake:

  MyModel.where(name: 'bob') { |m| puts m }

The block never gets called, because where() doesn’t use its block; you need to do:

  MyModel.where(name: 'bob').each { |m| puts m }

It feels like that block passed in where it isn’t needed should result in an error, just like passing an argument to a function where it isn’t needed does. But because the yield syntax for using a block doesn’t require anything in the function declaration line, Ruby doesn’t have an easy way to detect if the block is ever used.

DSLs (Domain Specific Languages)

Ruby makes DSLs practical to create, and powerful. As long as you’re using the DSL more or less as its creator intended, everything’s great, and you get to deal with things at a higher level of abstraction.

The problem can come when, as a user of the DSL, you try to DRY up your code, because you have the same set of steps with tiny differences happening in multiple places. You may end up having to learn how the DSL is actually implemented, which busts the abstraction wide open.

Summary

I’ve had a great time learning and using Ruby. It’s a wonderful language, and I use it everywhere I used to use Python. Yes, there’s some quirks, but the positives far outweigh the negatives.

Side project: u8this.com

2014-02-19T00:00:00+00:00

Over the last month (first checkin Jan. 23), I’ve been working on a side project. It’s a food diary site, for calorie-counting. I’m working on losing weight, and wanted to do something more complicated in rails than my last project.

You can find the site here: u8this.com Feel free to make an account and mess around with it. (Put garbage data in if you want to see how it works; I won’t be offended.)

The code is here: github.com/csbrooks6/u8this

I’ve tried to use all the latest tools in building it:

Rails 4.0
RSpec
Developing on osx, using Sublime Text 3
Deploying to linode using Capistrano
postgres
Trello to track my TODO list

I wanted to document some of the different challenges I ran into, and techniques I used.

Idea

The idea, in my mind anyway, isn’t revolutionary. At least in terms of getting users, chances are low I’m going to be able to peel users away from the myriad other food diary sites. And even if I could, I think monetizing is unlikely.

Photo by epSos.de

There’s upsides to this idea, though:

Solidify my Rails knowledge
Use some different things I haven’t used before, like RSpec
It’s something I personally want to use, so I can make it work exactly the way I like

Git: Locally, linode and github.

From the start I used git for source control. First a local repository, then I created a remote on my linode server, and now Github. I haven’t yet switched completely to Github, so I’m still pushing all changes both to my linode and github at the same time.

I mostly use SourceTree for osx, which I really like. Clean interface, very flexible. My only complaint is that I’ve been getting dramatic slowdowns at times. Other than that, it’s awesome.

I’ve mostly been working straight out of the master branch, though as the site has gone online, I’ve made a dev branch and switched over to that, so I could do a quick bugfix in master and push it live without waiting for my dev changes to be complete.

RSpec, and TDD

This is my first real foray into test-driven development. I found a good e-book called “Everyday Rails Testing with RSpec” that I used to help me figure out what I was doing.

I have a number of RSpec tests now. I haven’t yet built any integration tests, just model and controller specs. I also don’t really have 100% coverage. I guess I’m still not fully in the mode of writing the tests first and the code second. I need to focus on that more, because it’s worked well when I try it. (Although one disadantage is that if you are still learning RSpec, it’s easy to write tests that aren’t correct. Then inevitably you end up rewriting the tests anyway as the code comes online. It’s getting easier, though.)

All in all, I’ve really enjoyed RSpec.

Authentication

I did some research, and settled on Authlogic for account authentication. Seemed full-featured and well-supported, and there was even a decent example project on their github site that I used as a reference point.

I probably should’ve been warned away when I figured out the sample was written for Rails 2. It took some finagling to get it running in Rails 4. I emailed the author and offered to submit a pull request to update the sample to Rails 4, but no reply. (Another warning?)

The bottom line is, sadly, Authlogic doesn’t seem to be in active development. I get a bunch of rails deprecation warnings when I run my RSpec tests against it, that I think are fixed in a pull request, but that either hasn’t been integrated or something.

The warnings look like this:

DEPRECATION WARNING: ActiveRecord::Base#with_scope and #with_exclusive_scope 
are deprecated. Please use ActiveRecord::Relation#scoping instead. (You can
use #merge to merge multiple scopes together.)

Super annoying to get dozens of those when I rspec.

Anyway, now I’ve figured out everyone is using Devise. It’s on my TODO list to switch everything over.

Bootstrap

I decided to use Bootstrap, to implement a responsive design. Working well on both smartphone browsers and PCs was important to me. Bootstrap also makes things look a lot slicker than I would have had time to do.

I used Bootstrap 3, but it hasn’t been out terribly long, which means a lot of gems and stackoverflow questions are targetting Bootstrap 2.3 instead.

I used a gem called bootstrap-sass. In retrospect, I think it would have been simpler just to directly integrate the css and js downloaded directly from Bootstrap. There’s a particular issue I’m running into with Bootstrap that I think is fixed in the latest version. So I’m planning to try updating the bootstrap-sass gem, and if that breaks, I’ll look at getting rid of it and just putting Bootstrap into the asset pipeline myself.

The form controls look nice, though it took plenty of fiddling to get them to work right. I also lost all the clean Rails helper functions for views to emit form controls. There are gems like simple_form that can help with that, but at the moment, it targets Bootstrap 2.3.

Typeahead.js and Bloodhound

I used Twitter’s typeahead.js library for entering foods, so ones you’d previously entered would autocomplete. (I intentionally chose not to autocomplete to stuff other users had entered, because God knows what they put in there.)

Typeahead used to be part of bootstrap, but is now its own thing. I was a little disappointed with it, just because I had the idea it was going to have something cool behind the scenes to intelligently make queries with my dataset. It doesn’t do that though, and in the end my controller-side code is doing some horrible LIKE sql query that I’m still feeling iffy about. (It’s surprisingly hard to handle all the edge cases in using user-submitted data with LIKE through Rails, without letting bad data through. I couldn’t google up any clean database-independent way to do it.)

Typeahead is all about just knowing when it needs to request data, and showing the results that come back, and letting the user autocomplete or arrow-key select them. Basically, just the user interface side.

It can be used together with something called Bloodhound. Aha, so that’s the part that intelligently finds search results, right? Well, not quite. It’s job is to manage preloaded data, requesting more data, caching and rate-limiting those requests. But in the end you’ll still be making a request to your site with a string query, and it’s up to you to figure out what results match that string. LIKE to the rescue!

Typeahead also had some weird interactions with Bootstrap (surprising since they’re both from Twitter). It’s totally possible I’m doing something wrong, but the Bootstrap input text control text hints fight with the autocomplete hints that Typeahead shows. I ended up just turning off my Bootstrap hint text, for now.

I also ran into something weird where calling the typeahead() function on my input form control made the rounded edges that Bootstrap puts on it, disappear. After some investigating I think I tracked this down to typeahead “wrapping” the form control element when that function is called, which makes the css for the rounded corners not work anymore.

I tried to repro this in a jsfiddle, but it doesn’t seem to happen there. So I’m thinking it’s something with the version of Bootstrap I’m running, which is why I want to upgrade.

Anyway, for now, after I call typeahead() I just do this: $('.foods.typeahead').css('border-radius', '4px');

Gross, but it works. Like Javascript! I kid, I kid.

Javascript/Coffeescript, AJAX

I wanted to avoid having an ugly page reload when you modify or add “Servings” (my name for the ActiveRecord behind “something you ate”). So I used AJAX calls, using the “remote” method on my forms, and jquery/coffeescript listening for the AJAX to return. (Actually I didn’t use XML, I used JSON.)

The add and update controller methods would put the html for the new or modified Serving in the JSON object. Other methods would refer to the id of the Serving to be deleted, or moved up or down, or whatever. Then the coffeescript would update the page as appropriate.

One minor hitch was that I show a progress bar at the top of the page with the number of calories, and I wanted that to reflect any changes. So I took the most straightforward approach, and had the controllers just render the view partial for the progress bar and stuff that in each of the JSON return values, too.

A more major hitch I ran into was that the AJAX seemed to stop responding when I would navigate to the next or previous page. This is simpler to describe now than it was to track down, but it turns out that turbolinks makes jquery’s $(document).ready not get called, which causes all sorts of madness. The answer is a gem called jquery-turbolinks, which fixes this issue. (My first impulse was to turn off turbolinks, but the site did respond much slower. Although that was with rails server in development on my macbook, so probably not a great test, but still.)

One thing I wanted to do was show a context menu when you click a Serving on the page. Bootstrap doesn’t directly support this, but I found some code online that did the trick, mostly. I had to convert it to Coffeescript from Javascript, and make various tweaks for my needs. I also threw in some of the “glyphs” that Bootstrap comes with. I’m pretty happy with the results:

Capistrano

I heard this was the way to go for deployment. It really wasn’t too difficult to setup, though I got stuck for a while getting the ssh-agent forwarding to work. (That’s so that my private key on my macbook can essentially be used to issue commands to my linode server.)

I actually even got a staging server running while I was at it, so I could have an environment to test my deployment to linode before pushing it live.

Now every time I do “cap production deploy” and watch it do its thing, I pretty much feel like a badass. I’ve even been known to shout out “I am become death, the deployer of worlds!”, which no one thinks is funny but me.

Basic Web Scraping in Ruby with Mechanize

2014-01-05T00:00:00+00:00

As I mentioned previously, as a side project, I created a site to collect average prices for World of Warcraft Trading Card Game cards. Here’s how I wrote the code in Ruby to gather the data I needed from various card store websites.

Mechanize

I use Mechanize to interact with the sites I need data from: to visit URLs, pull down the HTML, and even submit forms. Mechanize fills the role of a headless browser, though it doesn’t do javascript.

Here’s code to print out google’s home page:

  require 'mechanize'

  # Create mechanize instance
  mechanize = Mechanize.new

  # Get the page
  page = mechanize.get('http://google.com') 
  puts page.body

The Mechanize::get call returns us a Mechanize::Page instance. We can use this to do some basic html parsing, and to find links and follow them, or to find forms and fill them out and submit them. But primarily, I use Nokogiri through the Page object, to search HTML using XPath.

Using XPath

XPath is a query language used to find nodes in XML, and therefore, HTML. (according to the docs, Nokogiri cleans up badly formed HTML as best it can, and my experience so far has been that it does just fine.)

The trickiest part is figuring out what XPath query to use. I use a Chrome extension called “XPath Helper” for this. You can click an element to get the corresponding XPath, and then play with the XPath query to see how to generalize it to pickup similar elements on that page.

Finding Card Categories

For example, say I wanted to get links to all the categories of cards on a page like this:

http://www.mtgmintcard.com/wow_category.php

In Chrome, I’d hit ctrl-shift-X to turn on XPath Helper. Then I’d hold down shift, and roll over one of the category names (let’s try “Betrayal of the Guardian”), which would turn yellow. While still rolled over it, I release shift, and now I can edit the query to generalize it.

So the XPath query starts out like this:

/html/body[@id='wowcategoryBody']/div[@id='mainWrapper']/table/tbody/tr/td[1]/div[@class='tabBody']/div[@id='normal_search']/div[@id='allEdition']/table/tbody/tr/td[1]/span[@class='symbol']/a[1]

NOTE: As the XPath Helper extension mentions, the <tbody> tag inside tables isn’t really in the html coming down from the site; Chrome adds it artificially. So we need to make sure to remove the <tbody> from the query before you give it to Nokogiri, or the query won’t find anything in Ruby. Delete the <tbody> but leave the double // so the query matches both in XQuery Helper and in Nokogiri:

/html/body[@id='wowcategoryBody']/div[@id='mainWrapper']/table//tr/td[1]/div[@class='tabBody']/div[@id='normal_search']/div[@id='allEdition']/table//tr/td[1]/span[@class='symbol']/a[1]

Now we want to generalize this to find all categories, not just the one we chose. Those [1]’s are what’s selecting only the first column, or first anchor. So we can experiment in the query removing those, and watch as the yellow highlight updates in Chrome.

We end up with this query:

/html/body[@id='wowcategoryBody']/div[@id='mainWrapper']/table//tr/td[1]/div[@class='tabBody']/div[@id='normal_search']/div[@id='allEdition']/table//tr/td/span[@class='symbol']/a

Note in the screenshot that the XPath Helper “Results” listbox shows the names of all the categories that have been matched by the query.

I prefer to simplify the query, to try to make it both cleaner and hopefully more resilient to future site changes. In XPath queries, a // will match anything, so we can delete the first part of the query and simplify it down to:

//div[@id='allEdition']/table//tr/td/span[@class='symbol']/a

This is a matter of keeping enough of the query to match only what we want, while removing anything that’s extraneous. Yes, this is subjective.

Now if we turn this into Ruby code to use Mechanize and Nokogiri to pick out the categories, it looks like this:

  require 'mechanize'
  require 'nokogiri'

  # Create Mechanize instance
  mechanize = Mechanize.new

  # Retrieve the categories page
  front_page = mechanize.get('http://www.mtgmintcard.com/wow_category.php')

  # Find the category links
  category_links = front_page.search "//div[@id='allEdition']/table//tr/td/span[@class='symbol']/a"
  category_links.each do |cat_link|
    # NOTE: actual link is in cat_link["href"]
    puts cat_link.text
  end

When run, the output will look something like this:

Betrayal of the Guardian
War of the Ancients
Tomb of the Forgotten
Crown of the Heavens
Throne of the Tides
Twilight of the Dragons
War of the Elements
Worldbreaker
Icecrown
Archives
... <snip> ...

Drilling down to find products

Let’s modify the loop to actually click through the link for each category, and retrieve the result:

  category_links.each do |cat_link|
    # Create a Page::Link and follow it.
    result_page = Mechanize::Page::Link.new( cat_link, mechanize, front_page ).click
    puts result_page.at('//title').text # Print the page title
  end

The ugly line to create the link and click it is necessary because we aren’t using the Mechanize::Page’s ability to find links (see Mechanize::Page.links and Mechanize::Page::link_with), but are getting the link through Nokogiri instead. (I prefer to run everything through Nokogiri/XPath instead of using the Mechanize::Page method, but you don’t have to.)

Ok, let’s look at one of these product pages, and figure out how to get product data out of it:

http://www.mtgmintcard.com/world-of-warcraft/english-regular/betrayal-of-the-guardian

This page has a table with a bunch of rows, one product per row. Let’s figure out an XPath query to get all the rows in that table.

First, using XPath Helper, select let’s select the column with a product name in it. The query looks something like this:

/html/body[@id='wowsearchresultBody']/div[@id='mainWrapper']/table/tbody/tr/td[1]/div[@id='indexDefault']/table[@class='resultList']/tbody/tr[@class='darkRow'][1]/td[2]

The first thing I notice is that every other row has a class='darkRow', but we want ALL the rows, not just every one. Let’s clip off everything after the last tr. I’ll cut out the tbodys, too:

/html/body[@id='wowsearchresultBody']/div[@id='mainWrapper']/table//tr/td[1]/div[@id='indexDefault']/table[@class='resultList']//tr

That query picks up the top row that has column names as well, but that’s ok, we’ll worry about that later.

Looking at the query, we really want all the rows in the table with a class of resultList. So let’s simplify it to:

//table[@class='resultList']//tr

Again, you can verify this query works in XPath Helper.

Ok, let’s get paths to individual product names and prices. By experimenting, we can figure out that this selects all the names (which are in the second column, hence the [2]):

//td[2]/a[@class='opacityit']

This gets us all the prices:

//td[8]/span[@class='productListPrice']

If we’re not sure those will always be in the second and eighth columns, we could leave the [2] and [8] out of the queries.

Ok, now we’re to going to put it together, to iterate over all the rows in the table, and find the product name and price for each row, if there is one:

  # Iterate over all the rows in the resultList table
  result_page.search("//table[@class='resultList']//tr").each do |row|
    # Name
    name_node = row.at(".//td[2]/a[@class='opacityit']")
    unless name_node 
      next
    end

    # Price
    price_node = row.at(".//td[8]/span[@class='productListPrice']")
    unless price_node
      next
    end

    puts "%s: %s" % [name_node.text, price_node.text]
  end

Note that . at the start of the at queries we’re doing on each row. If we left that off, the XPath query would actually go back to the top of the DOM, instead of only searching inside the row node.

Ok, so now we’re getting the name and price of cards on the first page in a category. We need to be able to click the “Next” link on each page to get all the results:

  next_page_link = result_page.at "//a[text()='Next']"
  if next_page_link
    result_page = Mechanize::Page::Link.new( next_page_link, mechanize, result_page ).click
  end

That query //a[text()='Next' means ‘find all the <a> tags where the text is “Next”’.

Alright, let’s put it all together. This will iterate over all the categories, drill down into each one, and pull out name and price data for all the products in that category:

  require 'mechanize'
  require 'nokogiri'

  # Create Mechanize instance
  mechanize = Mechanize.new

  # Retrieve the categories page
  front_page = mechanize.get('http://www.mtgmintcard.com/wow_category.php')

  # Find the category links
  category_links = front_page.search "//div[@id='allEdition']/table//tr/td/span[@class='symbol']/a"
  category_links.each do |cat_link|
    # Create a Page::Link and follow it.
    result_page = Mechanize::Page::Link.new( cat_link, mechanize, front_page ).click

    while result_page do
      puts result_page.at('//title').text # Print the page title

      # Iterate over all the rows in the resultList table
      result_page.search("//table[@class='resultList']//tr").each do |row|
        # Name
        name_node = row.at(".//td[2]/a[@class='opacityit']")
        unless name_node
          next
        end

        # Price
        price_node = row.at(".//td[8]/span[@class='productListPrice']")
        unless price_node
          next
        end

        puts "%s: %s" % [name_node.text, price_node.text]
      end

      next_page_link = result_page.at "//a[text()='Next']"
      if next_page_link
        result_page = Mechanize::Page::Link.new( next_page_link, mechanize, result_page ).click
      else
        result_page = nil
      end
    end
  end

The output will look something like this:

Betrayal of the Guardian - World Of Warcraft TCG (Cryptozoic)
A Demonic Presence: $0.29
Aegwynn, Guardian of Tirisfal: $0.24
Ancient Moonkin Form: $0.24
Arcane Anomaly: $0.24
Arcane Protector: $0.24
Arcane Shock: $0.24
Archdruid Fandral Staghelm: $2.24
Assault on Blackrock Spire: $0.24
Atiesh, Greatstaff of the Guardian: $0.24
Bastion of Defense: $0.24
Belthira the Black Thorn: $8.49
Bianca, Timewalker Mage: $0.24
Bigbelly, Furbolg Chieftain: $0.24
Bitey: $0.24
Blackfang Tarantula: $0.29
... <snip> ...

That’s the basics of how I pull out product name and price information from different stores in my WoWTCG card pricelist site. I plan to make another post going up one level, and documenting what I do with this information once I have it.

Thanks!

'Carmack Lines': Playing with Javascript

2014-01-04T00:00:00+00:00

I read this tweet by John Carmack:

Visualize an image where a pixel is on if a given bit is set in the sum of the x and y coordinates. I had poor intuition for its character.

I too had “poor intuition for its character”, but I’m pretty sure that’s not because I’m as smart as he is or anything. But still, I was curious how it would look.

I’ve also been looking for an excuse to play with Javascript and html5 canvases, and I couldn’t sleep at 5:30 this morning, so I popped up and had at it.

It took me less than an hour to get this running, even though I haven’t done anything significant in Javascript in years. And only maybe 20 minutes of that was the basic javascript, the rest was getting the form working right to let you dynamically change which bit is tested.

I think that’s more a testament to the power of stackoverflow than anything else. Maybe ten times while implementing this I found the answer there, where before, it would have taken a lot more reading and experimentation. It’s impressive that a single site with user generated content has had such an impact on the programming process. (At a meetup I went to, another programmer said to me “I can’t imagine programming before stackoverflow!” I wanted to say “imagine what it was like before we had the internet”, but I try to repress my inner old fogey when possible.)

I’m kind of amazed at how fast Javascript executes, at least in Chrome. I’m iterating over 16,384 pixels every time you hit the button, and there’s no perceptable delay at all, at least on my Macbook. Makes me want to do some more experiments with graphics in javascript.

Anyway, here it is.

Timing

2013-12-08T00:00:00+00:00

“Ask me what the most important thing in comedy is.”

“What’s the most important thi-“

“Timing.”

Where my last post left off, I was getting ready to rewrite my pricelist website in Ruby on Rails. I decided to do this so I could get some solid hands-on experience with Ruby and with Rails. And the python version had gotten crusty enough that a rewrite was looking enticing.

The rewrite took my spare time for several weeks, and I ended up using all sorts of fun stuff:

Passenger
Mechanize to parse html – much nicer than BeautifulSoup, once you get used to it
rvm to manage gems and other deployment crud
choice, to write a command-line script to gather and calculate the card price data
linode for hosting – a lot better than my old host
The bullet gem, to help optimize rails queries

Well, I got everything up and running, and working better than ever! I made an announcement on the wowtcg forums, and opened the revamped site for “business”. While that was live, I created a new branch to work on moving to bootstrap, to be more mobile-friendly. I also signed up for an ebay developer account, and wrote some scripts to gather price data there.

Then a couple of weeks later, Cryptozoic and Blizzard announced they were terminating the World of Warcraft Trading Card Game, after a run of almost seven years.

So, that sucked.

Anyway, now I need to figure out a new side project. I started working on making a game engine using SDL2 and box2d, but I’ve decided that isn’t the direction I want to go in right now.

I’m looking for a project that combines some (not all!) of these aspects:

Ruby on Rails
Using large public datasets
ebay api
Geocoding/mapping (integrating with google maps, maybe)
Very usable on mobile – bootstrap?
The potential to make money, even if it’s just a little. Not so much because I need the money, but for the experience of building something that generates income.

Side Project: WoWTCG Pricelist

2013-08-14T00:00:00+00:00

I got totally hooked on the World of Warcraft Trading Card Game, from the time it came out six or seven years ago. (It’s like Magic: The Gathering, but without mana screw or mana flood. And the players tend to smell less bad. Plus it’s the Warcraft universe!)

I went from being a casual player who didn’t really understand the rules, to spending hours looking over cards online and trying to work out decks. Then I started going to local weekly tournaments and connecting with local players, and then things really spiralled out of control. I ended up driving to regional tournaments that were a couple hours away, probably once a month or so. Eventualy I ended up going to nationals at GenCon a couple times. (I didn’t do terrible, but I didn’t do amazingly well either: finished about middle of the field.)

If you aren’t familiar with trading card games, you open “booster packs” that have a bunch of random cards. Some cards are a lot more rare, and useful, and therefore valuable than others. There are stores online that will sell “singles”, so rather than buying random boosters you can just pay for the cards you want. These cards vary in price from $100 at the very extreme end, but most are 25 cents, or a dollar, or maybe five or ten dollars.

What I found was that sometimes it was difficult to work out on-the-spot if a given card is worth a dollar, or five, or ten, which made trading tricky. So I wrote a script in Python that would scrape five or six different online stores for prices on singles, and build up an html pricelist with averages every night, so at least people would have a rough idea what a given card is worth.

The website is at hexagog.com, and I posted it under my forum name of “Conrad Hex”, which I use so no one can find out my true identity. Crap, I guess that’s blown.

Anyway, the site worked well for a long time. I loved it, other people loved it. I got emails of appreciation, and even got some free products from one of the developers of the game. But about a year ago it fell into disrepair. The site I used to update my card list database had gone away, and I wasn’t really active in the game itself anymore. Everyday I would get a sad little cron job email with a python exception, because one of the sites had changed their page layout, and the scraper couldn’t do its job any more.

Things stayed that way until I decided about a month ago to bring it back to life, this time in Ruby on Rails.

To be continued…

That’s me on the right with the red shirt, geekin’ out hard:

First Post!

2013-08-13T00:00:00+00:00

I was motivated to start writing after reading Matt Swanson’s article “Do things, write about it” on hacker news yesterday. I posted a comment on HN asking what’s a good way to put up a development blog, and Mr. Swanson and others recommended jekyll and github pages, so here we are.

I’ve had a blog before, but I was terrible at updating it. I think that’s because I was being too much of a perfectionist: I spent about a month polishing my “article” before posting it. (I did get a bunch of hits from it, but man was it painful to produce.)

I’m working on a side project, and I wanted to document how things are going, and also have a place to post tips or fixes as I find them. For example, I got a nasty issue with my site going down because I changed my .bash_profile to exec screen on startup, and somehow that took down Passenger on my production site. It would have been perfect to post the error message here, in case someone else gets the same issue they could google the fix.

Anyway, my goal is to post here pretty regularly. I’m not going to try to target the quality level of a book or anything this time, but maybe more the quality level I would use for email to a group of ten other programmers at work about something important. I think that will be a good balance so I can post stuff I’m not ashamed of, and still actually get stuff posted without endless noodling.