Suppose you're generating some sample data for your project as part of your db:populate rake task and part of that generation involves creating one hundred or so user records with varying bits of generated data. Then you want to associate each of these user records with other users so you execute an innocent User.all - its okay, this is a rake task we're only going to run in development.

Except you start getting error messages like You have a nil object when you didn't expect it! on User.all.each do |user|. Hopping into rails c and running User.all gives you a dump of the user table. Placing a pp "User count #{User.count}" right before the User.all returns 100, and looking in the logs you're even seeing the SELECT statement executing.

So what gives?

After a bit of debugging I was able to discern a hidden error message in all of this: "MySQL server has gone away." Oh, how kind of it to simply go away. There a number of reasons for this and in this particular case I am going to guess - and yes this is most definitely a guess - that its the packet too large problem. (And the more I think about this, the more I think that it cannot be this problem because the entire table size is 64KB. But I'm going to go with this for now.)

My brew installed MySQL server isn't running with a set max_allowed_packet value, so it defaults to 1MB. I'm theorizing that the returned data exceeded 1MB so the connection was dropped. This can be configured around by running mysql max_allowed_packet=16M or setting it in an option file but I don't like that approach. I went for chunking the result set up into smaller pieces - which is okay, this is a rake utility task after all - and it ended up looking like this:

#./lib/tasks/populate.rake  
# before
User.all.each do |user|  
  ⋮
end

#./lib/tasks/populate.rake  
# after
users = []  
10.times { |n| users.concat User.where("id % 10 = #{n}") }  
users.each do |user|  
  ⋮
end

The above after code works for this project; YMMV so smaller chunks may be necessary.

If someone has a better grasp on what's really going on here, please do enlighten me.