Blog of Aaron Theodore - Home

Solaris + Jumbo Frames + Intel Gigabit

2007-08-22T09:59:00Z

For a while I have been wanting to get Solaris working with jumbo frames on Intel gigabit ethernet (I have a PCI-X 82546EB Dual Port Gigabit Ethernet Controller). Initially i wanted this for improved performance between my diskless workstation and Solaris server (NFS). Since moving state in January 2007, I never actually setup the diskless workstation again, so never bothered with jumbo frames, as my switched network was only 100mbit (diskless workstation connected to server via dedicated port).

Anyhow, I still only have a 100mbit switched network, but I just purchased a ST Labs C-210 gigabit expresscard for my laptop and want to play with iSCSI. The plan was to have a direct gigabit connection from laptop to server dedicated to iSCSI.

To get the best performance out of iSCSI I needed to enable jumbo frame support at each end. In windows all i had to do is configure the driver to support 9k frames. In Solaris it was a bit more tricky…

In my case my internal hostname will be tanker-int with an ip of 192.168.2.5, with a mtu of 9000 (as that is the largest frame the other end of the link (the ST Labs card) can support).

Firstly added hostname to /etc/hostname.e1000g0 (e1000g0 being the particular port i was using)
tanker-int mtu 9000 up

edit /etc/inet/hosts to contain:
tanker-int 192.168.2.5

edit /kernel/drv/e1000g.conf
Find The MaxFrameSize line and set to:
MaxFrameSize=3,3,3,3,3,3,3,3,3,3,3,3,3,3,3,3;
to support 16k frames

reboot

A reboot shouldn’t be necessary, but even after unloading the kernel module reloading it and brining interface up, it would refuse to accept my driver config changes, so i just gave up and did a reboot.

e1000g0: flags=1201000843<up> mtu 9000 index 2
        inet 192.168.2.5 netmask ffffff00 broadcast 192.168.2.255
        ether 0:4:23:c3:f6:90

There are other tcp tuning params that seem to be a good idea to change like window size and receive/send buffers. I have yet to work this out, it should be a good post for the future when I work it out.

Solaris Native mode driver for Nvidia SATA

2007-08-22T09:13:00Z

Finally, looks like OpenSolaris will finally get Native mode driver for Nvidia SATA in build 72.

I have been wairing and hoping for this since i first started using OpenSolaris in September 2006. Hopefully I’ll get better performance and hotplug support.

Pitty i only reinstalled my server to build 70 (from build 56), now i’ll have to attempt an update once build 72 comes out.

Fixing utf8 text stored as latin1 in MySQL

2007-04-13T10:18:00Z

UPDATE: After posting this i found a good blog post about the same issue Turning MySQL data in latin1 to utf8 utf-8

While migraging an old PHP based web app to Ruby on Rails I came across an issue where the database and php were talking latin1 but the web interface outputted utf8 and forms posted were also in utf8. Part of the issue was the html the application was outputting had no meta tag such as:
<meta http-equiv="content-type" content="text/html; charset=ISO-8859-1"/>

and the headers sent by apache were:
Content-Type: text/html; charset=UTF-8

After much trial and error i managed to work out how to migrate this data.

Use mysqldump on the table with the option --default-character-set=latin1 and output to a file.

This file will have text encoded as utf8

Then edit the table definition in the file and change the DEFAULT CHARSET to utf8

Its then a good idea to run the file though iconv to drop corrupted characters

iconv -f utf-8 -t utf-8 table.sql -c -o table_fixed.sql

It is also a good idea to set mysql to default to utf8 for everything, edit my.cnf

under [client] and [mysqld] add: default-character-set=utf8

Then can reimport the fixed data back into mysql.

mysql -u user -p -e "source table_fixed.sql" database

Of course now you need to make sure anything talking to the database is doing so in utf8

Open Solaris ZFS boot support!

2007-03-29T23:02:00Z

One of the things that really annoyed me with Open Solaris was not being able to use a ZFS filesystem to boot (well not not without major hacks). This looks like it will finally change now.

It will be great to be able to have a Raid1 boot/root filesystem without having to resort to UFS and SVM.

A better view source for internet explorer

2007-03-28T22:31:00Z

Reading my usual RSS feeds i came upon the following post in the IEBlog: IE7 in Windows Vista: Configuring Your View Source Editor.

I don't run Vista, but what caught my eye was:

If you are looking for a better View Source Editor option than Notepad, install Microsoft Visual Web Developer 2005 Express Edition and add:

C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\VWDExpress.exe

to the following registry key:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\View Source Editor\Editor Name

Which means add the following to a .reg file and run it:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\View Source Editor]

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\View Source Editor\Editor Name]
@="C:\\Program Files\\Microsoft Visual Studio 8\\Common7\\IDE\\VWDExpress.exe"

Note: the actual download is quite large ~ 42mb, then a 30mb service pack for it. Quite hefty for just a view source editor

Anyhow this all worked fine and dandy, the speed it loads is instant which is fantastic (i wish the same could be said for Dreamweaver or eclipse (RadRails)), So I think that I will keep it as my view source handler.

The first issue I noticed when doing a view source of a ruby on rails application, Visual Web Developer wasn't sure what to do in the code highlighting department since the temp file it has opened had no extension.

There is a simple fix to this:
Tools -> Options -> 'Show all settings Text Editor -> File Extension -> Map extensionless files to: HTML Editor

Search Engine Friendly URL’s with Ruby on Rails

2007-03-27T04:19:00Z

Sometimes it is nice to be able to show meaningful text in your URLs

URLs like: http://www.site.com/sports/1/teams/1;results
Although relatively short, don't have a lot of meaning

http://www.site.com/sports/cricket/teams/first-11;results
Has more meaning, but is not ideal from the web applications point of view.

http://www.site.com/sports/1-cricket/teams/1-first-11;results
Is the best of both worlds, and surprisingly easy to achieve in ruby.

In all the models you want a descriptive id, just overwrite the to_params method. By default model.to_params returns the value of the models primary key.

In my case the name field of all my models is the field i want to appear in the URL.

The following will do that. Sets the name to lowercase and replaces spaces with -

 
  def to_param
    "#{id}-#{name.downcase.gsub(/ /, '-'))}"
  end

Unfortunately this is not ideal, what if you had non ascii characters in your name, you would end up with escaped characters and an ugly URL

So i chose to make use of the PermalinkFu plugin from Rick Olson

 
  def to_param
    "#{id}-#{PermalinkFu.escape(name)}"
  end

The PermalinkFu plugin will convert the string so it is suitable to be placed in the url, it uses Iconv to do a unicode translation to ascii. This means strings such as café get translated to cafe

You might think that by overwriting to_params rails would get confused, but it dosn't. Well as long as you observe a simple rule:

When generating a url pass the model in where it expects an id
:id => model NOT :id => model.id This works as rails automatically calls to_params to get the id, if you call model.id it will bypass to_params

After writing this i noticed SEO on Rails posted about Even better looking URLs with permalink_fu. I don’t think this is ideal since if you update the name/permalink anyone linking to your URLs will end up with a broken link. Also there is no way to ensure unique permalinks (unless you add extra code into your model which i will cover in a future post).

Too many RSS feeds!

2007-03-24T05:00:00Z

I think it is safe to say i have WAY too many RSS feeds in FeedDemon!

ImageScience and mod_fcgid

2007-03-22T04:11:00Z

Recently converting a bunch of rails apps to AttachmentFu from ActsAsAttachment to make use of ImageScience i found that anything that refered to a model that has a: 'has_attachment' in it caused rails to fail with a cryptic message:

ActionView::TemplateError (exit) on line #41 of app/views/pages/section_home.rhtml:
38:           <h2>Story</h2>
39:           Scobie sponsors many of Tasmania's community organisations...
40:         </div>
41:         <img src="/images/stories/1.jpg" border="0" alt="Story" />
42:       </a>
43:     </div>
44:     <div class="story">

/usr/lib/ruby/gems/1.8/gems/RubyInline-3.6.2/inline.rb:70:in `exit'
/usr/lib/ruby/gems/1.8/gems/RubyInline-3.6.2/inline.rb:70:in `rootdir'
/usr/lib/ruby/gems/1.8/gems/RubyInline-3.6.2/inline.rb:84:in `directory'
/usr/lib/ruby/gems/1.8/gems/RubyInline-3.6.2/inline.rb:257:in `so_name'
/usr/lib/ruby/gems/1.8/gems/RubyInline-3.6.2/inline.rb:291:in `load_cache'
/usr/lib/ruby/gems/1.8/gems/RubyInline-3.6.2/inline.rb:627:in `inline'
/usr/lib/ruby/gems/1.8/gems/image_science-1.1.1/lib/image_science.rb:84
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
/usr/local/lib/site_ruby/1.8/rubygems/custom_require.rb:27:in `require'

etc...

Error was quite bizare especially as line 41 refered to a static file that gets served up by apache, taking out that line would cause errors on other lines of the html.

Anyhow after much debugging and looking through the source for ImageScience and RubyInline i noticed that RubyInline was not generating the shared object for ImageScience.

The default in my case was:
ENV['HOME']/.ruby_inline/Inline_ImageScience_aa58.so

That seemed like it should work, i expected fastcgi would have the HOME environment variable set. I was wrong...

as a quick test i edited dispatch.fcgi to include:
ENV['HOME'] = '/home/username'

Alas it worked!

I didn't think it was ideal to have to edit dispatch.fcgi for each rails app,

some quick research led me to find the Apache module mod_env.
PassEnv directive which could be set in the global 'server config' looked usefull!

But no... Apache reported: [warn] PassEnv variable HOME was undefined

SetEnv also didnt work in any of the contexts (server config, virtual host, directory, .htaccess)

Now because im running multiple clients and accounts on this server i use suexec.

It seems suexec blocks Environment variables that are not in its whitelist (which is compiled into it).

I was already manually compiling suexec (to replace the one from the Debian package) so added the following: /usr/src/apache2-2.2.3/support/suexec.c

    "TZ=",
+    "HOME=",
+    "RAILS_ENV=",
    NULL

recompiled, coppied to correct place and reloaded apache

"That has to solve it" I thought to myself...

Wrong again!

I had hit another brick wall, this time it was mod_fcgid.

It appeared it was blocking my attempts to set an environment variable in apache. The only option the module has for environment variables is: DefaultInitEnv and that is done in the 'server config' context, which is useless for my needs.

At this point i thought it May be posible to hack mod_fcgid to pass the environment variables i want, but i had already spent way too much time on this.

So i will continue to set ENV['HOME'] in dispatch.fcgi till i find a better solution for my fastcgi rails apps.

If anyone has a better solution to this i'd love to hear about it!

FYI: The majority of the rails apps on the server are run under mongrel, fastcgi is just used for the ones that don't get much traffic and/or are for testing.

How not to upgrade rubygems on debian

2007-03-22T03:01:00Z

I was having some issues with a certain rails site not working when run as fastcgi (worked as mongrel and webrick), in my process of debugging i realised the server i run the app on (Debian) didn’t have the latest rubygems.

It had version 0.9.0 (at time of writing 0.9.2 was latest), naïvely i thought rubygems could be to blame!

Unfortunately the newest Debian rubygems package was 0.9.0 (even in unstable!), for the life of me i could not locate a 3rd party .deb.

No problems… compile from source it is!

Now I’m getting even worse errors trying to run the fastcgi (how bizare)

Try mongrel again…. Uh Oh!! doesn’t work!

Yes i managed to break the rails hosting server i run for multiple clients, and new rails processes would not run.

bit of debugging later.. Errr where did all the system gems go??

What happened is..

Debian rubygems stores its gem archive in: /var/lib/gems/1.8/ ,
whereas the compiled rubygems default was: /usr/lib/ruby/gems/1.8/

I could have possibly symbolic linked or copied the files to new location, but to be on the safe side i just reinstalled them all

NOW… i REALLY have to get that virtual server setup at home so i can test system changes on a non production system!

Ok my blog is finally back

2007-03-22T02:33:00Z

Ok well my blog is finally back…

I thought it was about time to set it up as i keep discovering things i should blog about, but to do so i would need to go though all the effort of setting up my blog again, so i never bother.

This time i setup mephisto to test to see if it also had the same issue as my own rails app did with fastcgi, and it did!