Just so wrong...

Package Management Sudoku is just so wrong.

OSGi vs C3P0

Continuing the OSGi saga...

The web application framework I am building attempts to leverage an existing configuration framework, which has caused me a bit of a headache. The configuration framework is built around a set of database tables holding the configuration information and provides a one-stop source for all of it, including basic configuration data, JDBC DataSource connections, LDAP connections, etc., etc., etc. (And ultimately including the log tables written from Log4j.)

The one-stop is the initial database connection to get to the configuration data. My headache starts with the framework, which uses a c3p0 JDBC connection pool (see also this), which in turn includes the wondrous JDBC-driver-from-class-name thingy. Coupled with the wondrous OSGi class loader hacks, the result is an almost certain headache. Here is my aspirin:

• I bundlized (or used off-the-shelf bundles for) the JDBC drivers: MySQL, Oracle, SQLServer. ("Bundled" does not feel like the right verb.)


• I got the c3p0 bundle from the SpringSource Enterprise Bundle Repository, which was where I also got the Oracle JDBC driver bundle (and the required javax.resource bundle).
  
• I created a fragment bundle to provide an attachment between the JDBC driver bundles and c3p0. This fragment consists only of the MANIFEST.MF file, which itself really only consists of the following lines:
  

  
  Import-Package: com.mysql.jdbc,oracle.jdbc.driver,com.microsoft.sqlserver.jdbc
  Fragment-Host: com.springsource.com.mchange.v2.c3p0
  

  
  When this fragment bundle is resolved, it links to the c3p0 bundle and effectively adds the Import-Package header to the c3p0 bundle. As a result, the c3p0 bundle imports the listed packages from the respective JDBC driver bundles, making the driver classes available to whichever variant of Class.forName() c3p0 uses; ideally, not Class.forName, which has issues.


• I installed all of the above in the SpringSource Application Platform's repository/bundles/usr directory, where they are available to the S2AP OSGi runtime.


• I fired up S2AP.


• I deployed the bundlized version of my config framework in the S2AP pickup directory, which caused it to begin running and to export it's data service interface into the OSGi service registry.


• I deployed my test bundle to the S2AP hot-deployment pickup directory. This bundle imports the services, connects to the default datasource, and runs a basic query. Success!

There may well be a better way of doing this, but the minor configuration bundle attached to c3p0 seems acceptable.

Edit: I just looked at this post and realized I never mentioned why it worked.

Gnome does not support US-Dvorak layouts?

Looks like some of the problems I have been living with since I installed Ubuntu 8.04 are intentional.

(Short summary: I use a Dvorak layout; this laptop has an unmodified (and unmodifiable) Qwerty keyboard. I leave the Qwerty layout unchanged globally in case someone else needs to log in to it, and switch to Dvorak using the Gnome keyboard config. Unfortunately, that now means that control (and any modifiers) key strokes use the Qwerty layout; i.e. I type Ctrl+"I" (Ctrl-C, in Dvorak) and get a tab, not a break. The situation is even worse when I connect a physical Dvorak keyboard to the computer---I couldn't even look at the keyboard to figure out the translation.)

Fortunately, I've switched to xmonad and dumped most of the Gnome stuff. I get to manually hibernate and configure wireless networking[1], so "setxkbmap dvorak" isn't strikingly different.

[1] Who thought it would be a good idea to only provide an applet for network management?

Ruby Hoedown!

Friday, 9:00am Ruby: A Year of Innovation by Jason Seifer and Gregg
Pollack.

I got here a little late, and so may have missed part of the
presentation, which seemed mostly to be a long list of interesting
projects:

• Phusion Passenger: Apache module-served Rails (actually, Rack)
  applications.


• Github: source management + FaceBook. The specific neatness was the
  ability to get a fork off a project, make changes, and then have the
  upstream maintainer *know* that you have made those changes and
  possibly merge them back into the upstream branch.


• EventMachine: a single-threaded, event driven server framework,
  somewhat similar to Twisted. Examples using EventMachine were
  Evented Mongrel, Merb (allowing the use of event driven handling for
  short tasks and threaded handling for longer ones), and Thin (which
  is EventMachine + Rack + Mongrel's URL Parsing).


• Starling (http://rubyforge.org/projects/starling/): a lightweight,
  persistent, reliable distributed message queue.


• Redmine (http://www.redmine.org/): a project management web
  application.


• Pool Party (poolpartyrb.com): balance, monitor, and maintain Amazon
  EC2 cloud computing configurations.


• Rad (rad.rubyforge.org): an Arduino development platform for Ruby.


• Adhearsion (adhearsion.com): integrating and controlling VOIP
  applications based on the Asterisk software PBX.


• make_resourceful (http://mr.hamptoncatlin.com/): a RESTful DRY
  (don't repeat yourself) helper for Rails controllers.


• Hpricot (http://code.whytheluckystiff.net/hpricot/): a HTML parser.


• Juggernaut (juggernaut.rubyforge.org): a Rails plugin that enables
  the server to push data to the client.


• Ambition (ambition.rubyforge.org): an SQL generator (and general
  query language) generator using Ruby syntax.


• Prawn: a pure-Ruby library to generate PDF files.


• Capistrano: the server deployment tool.


• Various Ruby VMs: all are demonstrating innovation and progress.
  
  
  
  • IronRuby
  
  
  • JRuby
  
  
  • Gemstone
  
  
  • Maglev
  
  


• The Roo gem: parsing Google Docs, Excel, and OpenOffice spreadsheets.


• The Ruby DTrace provider: probes into a live Ruby process.


• Skynet: a Ruby implementation of Google's mapreduce.


• Data Fabric: scale ActiveRecord databases horizontally.


• Merb: a lightweight web framework.


• Mack Framework: a distributed web application framework for
  portal-like applications.


• Sinatra: a web framework for RESTful applications


• Webby: a framework for offline generation of non-dynamic web sites.


• Two from why: Shoes, a small graphical toolkit, and Hackety Hack, a
  site for teaching Ruby hackery to kids.

See railsenvy.com/hoedown (and railsenvy.com/podcast).

Friday 10:30 Cloud Computing with Ruby by Robert Dempsey.

Robert brought the message that Rails CAN scale, at both the web
service and database level, if you know how. Scalability, in this
case, means not only that the system can handle large amounts of work
but also that it can be enlarged to grow as demand increases. Cloud
computing provides that ability. Cloud computing, as defined by
Robert, requires:

• The ability to purchase capacity easily and immediately,


• no need to get involved in purchasing or owning hardware,


• an API for programmatically controlling the configuration, and


• that the whole process take no more than 10 minutes.

A standard production deployment, according to Robert, is two load
balancers, two web servers, and two databases; this runs in the
vicinity of $432 per month.

The options Robert discussed include:

• Amazon Web Services (EC2 and S3), the base of most of the other
  offerings.


• Morph, which provides no ssh or file-level access.


• RightScale and RightGrid.


• Heroku, which is free (for now).


• Joyent Accelerator, which is a VPS, not really cloud computing and
  involves OpenSolaris.


• Vertebra.


• PoolParty, which is a Ruby gem plus AWS and which was not ready for
  production.


• Scalr, which is also free code built on AWS.


• ElasticRails, free for non-commercial use.
  

For recommendations, Robert first suggested deciding whether you are a
sysadmin or a developer; if the latter, then to go with Morph, Scalr,
or RightScale.

For more information, see Programming Amazon Web Services, REST on
Rails 2 Projects, and an AWS article series, Introduction to AWS for
Ruby Developers.

Friday, 1:00pm Mock Dialogue, Joe O'Brien and Jim Weirich

Joe and Jim gave a cute skit on the topic of mock objects and
testing.

Using mock objects (as any testing) requires require the code to be
written specifically to allow the use of the mocks. Using mocks,
however, goes further to allow tests to (in Michael Feather's rules),

• Not talk to the database,


• Not use the network,


• Not use the filesystem,


• Run at any time, and


• Not require anything special in order to run.

Mock objects are an extension of testing fixtures such as
hand-generated stubs. Hand generated fake classes, for example, can
become as complex as the code being tested, and remedying that can
lead to a large number of testing classes. Mock objects

• on the one hand, help clean up that situation by making the creation
  of test-specific stubs as simple as possible, while


• on the other, allow the behavior of the code under test to be
  validated by ensuring the mock object's methods are called the
  correct number of times and with the correct arguments.
  

Just be sure not to go so crazy with mocks that you don't actually
test anything.

P.S. Ugly, difficult tests may indicate bad application code.

P.P.S. The presenters like Flexmock.

Friday, 2:15pm Ruby Best Practice Patterns by Rein Henrichs.

Rein gave a somewhat humorous song and dance about how best practices
are bad on job security grounds....

Rein described best practices as the set of techniques that experts
*currently* recognize and use, and patterns as the decisions experts
make over and over. All are learnable and optimized to communicate
effectively.

The patterns Rein identified in Ruby are:

• Composed method: How to divide a class into methods. Each method
  should perform one task, and have a good name. Any complex method
  should be composed from simpler methods, to allow each method to be
  understood in a single gulp.


• Execute around method: Ruby methods can accept blocks! The method
  executes before and after yielding to the block.


• Yield and return: yield the subject of the method to the block, then
  return the object. [Ok, this one needs further study.]


• Method object: Have many lines of code sharing the same parameters
  and temporaries? Replace the state threaded through the lines with
  a state object which has the code as its methods.
  

The best way of discovering patterns is to read expert's code. Also
read Smalltalk Best Practice Patterns by Kent Beck.

Friday 4:00pm Lightening talks

• God and its forthcoming web interface. See github / pjdavis.


• Bryan Liles - Testing for normal people who say, "Test all the
  fucking time."
  
  
  
  • Think of your application as objects expressing behavior and
    create examples of those behaviors.
  
  
  • test/spec
  
  
  • bacon
  
  
  • shoulda
  
  
  • mspec
  
  
  • Test::Unit + shoulda + RSpec + StoryRunner
  
  
  • Cucumber?
    
  
  


• Youssef Mendelssohn - What is Truth? In Ruby?
  
  
  
  • nil and false are not.
    
  • github.com/ymendel/truthy
    
  
  


• Google charts is cool, especially with the google_chart gem.
  

Friday 5:00pm Keynotes by Obie Fernandez and Chris Wanstrath

First, Obie Fernandez:

• Do great work for a few years.


• Write a book.


• Consult.


• Profit.


• Never Eat Alone


• Secrets of Power Negotiating


• Predictably Irrational


• Purple Cow


• Hirise lead tracking software
  

Then, Chris Wanstrath gave an inspiring keynote essay, which should be online someplace.

Saturday 9:00am Calling Your Code: Gluing Phone Calls to Ruby by Troy
Davis.

Demos:

• Twittervox: Listen to the Mars Phoenix Lander twitter feed by phone.


• Yelpvox: Find good, nearby restaurants by phone, using the phone
  number of a bad, nearby restaurant.


• SendSign: Get real estate flyer information by phone and MLS number;
  also use a web interface to see record of phone queries.

Twittervox is implemented using Cloudvox and a slice from SliceHost.

• Swift text-to-voice engine


• Adhearsion (or Telegraph and RAGI)


• Asterisk


• FreeSWITCH

Tips for building voice apps:

• "Can you hear me now?" is not testing.


• If you have to call, you are not testing.


• Use a paper prototype first.


• Get the core functionality: small first.


• Placeholders first.


• Text-to-speach is a good placeholder.

Saturday 10:15am Ruleby: The Rule Engine for Ruby by Joe Kutner.

Joe began with where declarative programming and rule engines fit into
the programming language landscape. He described a rule engine-based
system as made of a set of facts, a set of rules, and the Rete-based
rule engine. The separation of rules and facts centralizes business
logic knowledge.

In Ruleby, facts are Ruby objects; they become facts by being asserted
and can be retracted. The rules engine can fit effectively into Rails
applications between the controller and ActiveRecord.

Saturday 1:15pm flog << Test.new

Working with an existing codebase is significantly different from
writing new code. One piece of advice: build a scaffold of tests
before starting to change the existing code.

1. Survey the code: Do not get distracted by incomprehensible code.


2. Put the commandline script under test.
   
   
   
   • Some code is resistant to testing: make the smallest change to
     make it testable.
   
   
   • Some code is mysteriously coupled to other code: leave pending
     markers to indicate the mystery.
   
   


3. Put the library entry points under test.
   
   • "Characterization" tests - "This code does A."
   
   
   • Use CURRENTLY to mark things that may be wrong.
   
   
   
   


4. Recursive code?
   
   
   
   • If faced with a "Big blob of code": carefully
     pull things apart.
   
   
   • Small commits are good, particularly for reverting.
   
   
   • At one point, he broke flog, but all of the tests pass, so he
     decided to add integration tests to act as stronger scaffolding.
   
   


5. Pin down behavior with integration tests. (Should have done this
   first.)
   
   • One trick: use a YAML dump to record internal behavior of
     original.
   
   
   • Use code coverage to identify parts which are not tested...and to
   
   
   • Depending heavily on attributes makes code hard to test.
   
   
   
   

I unfortunately missed the second set of lightening talks, as well as Saturday's keynote.

[Edit: Obie Fernandez is actually named "Fernandez". Sorry! Great book, by the way!]

OSGi vs. Log4j

I have just posted a note concerning some recent thinking about building a logging-to-database subsystem in OSGi. (It includes some lovely Powerpoint-based diagrams!)

In case you run across Richard Feynman or Edward Tufte, please let them know NASA is still running mostly on PowerPoint.

(Oh, and speaking of Edward Tufte, his web site is remarkably hard to navigate. This means something.)