Thursday, May 17, 2012

Meteor contest is published on benchmark games

I arranged to have the visualworks version of meteor contest published on shootout benchmark site.

This version makes Smalltalk only 7x times slower than fastest C++ version as expected, I think this time is taking image startup time into account (with -nogui).

Of course, with inclusion of optimized bit ops which are not native in VW, my stupid rotation algorithm, and some of my costly optimizations, this version of the game is also one of the longest, as long as a C++ program! That does not improve the efficiency/expressiveness score.

I must publicly thank Isaac Gouy again for his patience, since I lazily commited 3 non functional versions before taking the burden to test myself in batch mode. A shame.
Though he clearly made me understand this was becoming upsetting, he did not ban me.

My board initialization did rely on count: which is not a VW native selector.
I have first defined such selector in the 90s, and it's native in Squeak/Pharo, it is now so natural that I forgot it's not an ubiquitous standard yet.
It is provided both by Seaside (via Grease) and Smallapack (via some prereq), and of course, for some reason I had both packages installed in the image I used for testing.
That's a domain where Smalltalk sucks : when you leave your own image cocoon, implicit dependencies are killing... Another one is implicit API vs privacy, unless it's an advantage.
Being different does not necessarily means being better...
Fortunately, there are still enough advantages to compensate this poor feature.

I also accidentally removed a digit from the auto-generated optimized bit tables while trying to manually split too long lines, but saw the bug before it strikes. I generally don't care of line length, replacing tab with spaces to be tab-stop agnostic, or selecting source code output format to chunk...

I also failed to print a space before new line on even rows, and forgot the second new line after second solution. I remember I manually fixed this when I checked solution agreement 2 months ago, after all, those trailing spaces are not significant. Then I forgot...

I also failed to return a new String with ^'' in Tests class>>meteor; it thus returns self, and append an ugly 'Tests' on standard output... One thing I corrected 2 weeks ago in my Squeak version by mimetism with other benchmarks...

Of course, that's something that only happen in batch mode... As an excuse, I said to Isaac that I never launch an image from command line, nor with options, or command arguments, and that to me, it's as weird as booting a kernel with own custom flags, or booting on memtest. You generally never do that until you are forced to. But with Pharo progress on this front, I will soon have no excuse.

I also argued that my images tend to be long lived (several years or decades), because they are comfortably customized, and anything I need is generally pre-installed. To me, reconstructing an image is like re-installing a debian or an ubuntu every few days for sanity reasons, something I'm reluctant to do. I only abandon some overly hacked images where I modified too many kernel methods, and the Pharo ones because the update mechanism sucks compared to Squeak/trunk. It's normal, Pharo put the focus on reconstructing rather than updating as it is perceived as more reliable (and mainstream). Of course, Metacello and Fuel will lower the burden, but I still think it is a burden.

After downloading the tarball, digging thru the makefiles, the shell scripts, and all this mainstream stuff was not at all funny, I knew how to, but I suddenly remembered how good it was to be different finally...

5 comments:

  1. Great result, Nicolas! It's nice to see our beloved Smalltalk so high in the rankings, and also sad that the world has progressed so little since 1980.

    Regarding reconstructing an image, I used to feel the same way. Now I've set up a local Jenkins server that builds all my images and loads the objects (stored via simple image persistence on SqS, via ReferenceStream - soon to be Fuel).

    Since I took this step, I've felt a *ton* of freedom, even though I still keep an image running for months/years. If anything goes wrong, I have a fresh image built from a clean Pharo.

    Even for porting to a new version of Pharo, I just change the version string in my build script and go from there...

    ReplyDelete
  2. >> That's a domain where Smalltalk sucks ... implicit dependencies are killing...

    That's a well understood cause of error, and there's a well understood process to catch those errors - the code that's being exchanged must be filedIn to a vanilla image and then tested.


    >> As an excuse

    Sorry but when the program will be run from the command line it obviously needs to be tested by being run from the command line.

    The command line used to run each of the Smalltalk programs is shown with each program.

    (And the platform helper methods are shown on the Smalltalk page http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=vw#about)


    >> To me, reconstructing an image is like...

    To me, reconstructing an image is something that's been scripted and automated for over 20 years.

    ReplyDelete
  3. @seandenigris >>It's nice to see our beloved Smalltalk so high in the rankings, and also sad that the world has progressed so little since 1980.<<

    Perhaps compare against the JavaScript measurements.

    One obvious change is ubiquitous multi-core but the contributed Smalltalk programs only use one core.

    http://www.scribd.com/doc/51270583/Getting-More-From-Your-Multicore-With-Polycephaly-Arden-Thomas

    ReplyDelete
    Replies
    1. Technically, Polycephaly consists in spawning several processes and communicate data by serialization/deserialization on some I/O channel.
      In the case of meteor, it's trivial to parallelize the algorithm, and there would be very few bytes to communicate since only min and max solutions matter. Potentially we can hope a x4.

      Delete
  4. This comment has been removed by the author.

    ReplyDelete