Painful lessons - Porting Centos 5.8 from Virtualbox to Amazon EC2
Posted by Sean MacGuire
Notice - this post will save you a massive amount of headbanging if you ever need to do anything similar to this. (Thanks to HN for posting... almost makes it worthwhile)
Amazon has a really nice cloud infrastructure. It's really easy to spin up machines and attach volumes and do stuff.
I have one caveat. Sometimes I can't get trusted versions of an operating system and version that I need. By trusted, I mean from the distributor, as opposed to some fine third parties. The facinating thing about Amazon is you can spin up all sorts of machine images (AMIs) from all over the place. They can literally contain anything. Really. Anything.
So recently I wanted version 5.8 of Centos, a flavor of linux, and I couldn't get a pristine copy from AWS from the Centos people, so I had to make one myself.
There's a great free program out there - Virtualbox - http://virtualbox.org, (originally from Sun, which is now owned by Oracle). Install this on your machine and you can run other machines inside your computer... this is called Virtualization. So for example, if you're running a Mac, you could have a version of Windows running on your computer, in it's own little window, at the same time. Or if you're running Windows, you could spin up your very own little linux box to play with, and so forth.
So my thinking was, spin up my trusted Centos 5.8 using virtualbox on my mac, then export the disk image to Amazon, and coax it into running there.
I'm by no means the first one to do this.
Carson McDonald has the definitive post here http://www.ioncannon.net/system-administration/1246/converting-from-virtualbo...
Michael Martinez took a shot at is as well: http://amazonaws.michael--martinez.com/
I had a remarkably difficult time with this, due to some really subtle issues which were buried inside of moving gigabytes of crap around repeatedly over the course of a pile of hours. I'll list the problems here first, for any poor soul searching for the same solutions, the actual instructions are below:
1. My virtualbox image is the vanilla Centos 5.8 installation. Amazon needs the xen kernel installed:
yum install kernel-xen
2. The hard disk consists of a boot partition, and a logical volume located on the second partition. This means that the menu.lst file will be different than any of the examples out there - the key is root=/dev/VolGroup00/LogVol00. Symptom: kernel panic at AWS. Settings which worked for me below:
default=0
fallback=1
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.18-308.4.1.el5xen)
root (hd0,0)
kernel /vmlinuz-2.6.18-308.4.1.el5xen ro root=/dev/VolGroup00/LogVol00
initrd /initrd-2.6.18-308.4.1.el5xen.img
3. Even if you have the right kernel installed with the correct menu.lst file, it doesn't matter if it's not associated correctly. Get this wrong you get a kernel panic. We need the magic mkinitrd command to make it work:
mkinitrd -f --preload=xenblk --preload=xennet /boot/initrd-2.6.18-308.4.1.el5xen.img 2.6.18-308.4.1.el5xen
4. Because I've got /boot in it's own partition, Amazon will not end up looking in the correct location for my kernel. You need to copy the /boot stuff into /boot/boot (really):
cp -Rp /boot /boot
5. Now just because you're set up correctly doesn't mean that Amazon will recognize this. You need to boot up your instance with a special Amazon kernel.
Amazon can boot a Xen kernel using pv-grub which we installed at step 1, then you need to find which of Amazon's initial kernels will allow you to boot your kernel. Here's the list: http://docs.amazonwebservices.com/AWSEC2/latest/UserGuide/UserProvidedkernels... - there are 4 kernels per availability zone, 2 for 32 bit versions, 2 for 64 bit versions:
Separate /boot: kernels with "hd00" in their name like pv-grub-hd00_1.03-i386
One big disk: kernels with "hd0" in their name like pv-grub-hd0_1.03-i386
Once you've located the correct kernel, in the correct availability zone, permanently associate it with your snapshot using the ec2-register command - otherwise if you launch your kernel from the AWS dashboard with the defaults, you'll get, wait for it, Kernel panic:
ec2-register -n "centos5.8" --kernel "aki-b2aa75db" -d "Centos 5.8" --root-device-name /dev/sda2 -b /dev/sda=SNAPSHOTID:10:true`
6. If you're paranoid like I am, then you'll be taking all sorts of snapshots of your Virtualbox instance, just in case things go badly. Good practice. Of course, when we need to get a copy of the running machine's disk we use the following command:
vBoxManage internalcommands converttoraw centos5.8.vdi centos5.8.img
This command takes a copy of the virtual hard disk, and converts it to an img file, which we can then compress and transfer up to AWS and write onto a volume which we snapshot and turn into an AMI. I'll outline this process later, don't panic.
I did everything correctly. What did I get? Kernel panic. It's especially heartwarming since this happens after having waited the excruciating amount of time to upload 500mb to Amazon. So this debugging is a slow and painful process, which is why I'm documenting it.
Here's what went wrong. If you've snapshotted a Virtualbox machine, the changes are not reflected at the "top level" in my case, centos5.8.vdi. That file is still the state of your machine before any snapshots.
You must delete all snapshots from the top, down. Each time you delete a snapshot, your changes are merged with the later snapshots. When you no longer have any snapshots, your changes have been rolled into the top level machine instance, which you can now copy and send to Amazon. You can verify this by checking the date on your .vdi file.
This pearl of wisdom can be found in the Virtualnbox blog here: https://forums.virtualbox.org/viewtopic.php?f=2&t=20810
* * *
Here's a step by step guide
Steps done on your local machine
- default install of Centos 5.8 on virtualbox 4.1.12
- Download netinstall iso http://mirrors.bluehost.com/centos/5.8/isos/i386/CentOS-5.8-i386-netinstall.iso
- Select HTTP and these parameters for the rest mirror.centos.org / centos/5/os/i386
- yum -y update
- yum -y install kernel-xen
- Edit /boot/grub/menu.lst (see note 2 above)
- cp -Rp /boot /boot
- Powerdown your Centos virtual machine
- cd to the location of your virtual images (under Virtualbox VMs/YOURMACHINE)
- vBoxManage internalcommands converttoraw YOURMACHINE.vdi YOURMACHINE.img
- gzip YOURMACHINE.img
- scp -i YOURKEY.pem YOURMACHINE.img.gz YOURSERVER.amazonaws.com
Steps done on Amazon AWS
For this part you'll need an instance on AWS running some form of linux, and a volume at least as big as your Centos image - mine was the default 8mb so...
- Create a 10 gb volume on AWS
- Associate it with your running instance - note it's name, like /dev/sdg
- cat YOURMACHINE.img.gz | gzip -d | dd of=/dev/sdg bs=10M
- Snapshot your volume
- Create an AMI from your snapshot
- Run the AMI
script: volume to running instance (steps 4-6 above)
This script is run from your local machine - make any changes to your volume, then run the script and it'll try to fire up an instance... which may or may not work. At least you'll know what the steps to follow are.
You'll need to have the EC2 tools for AWS installed: http://aws.amazon.com/developertools/351
And for the script, you'll need to have your volume-ID from above, plus a keypair. The stuff you need to set is shown inside the shell script. No warranty, public domain, enjoy.
http://kimosabe.net/aws-vol2ami.sh
Twitter - create the Defensive Patent Foundation now!
Posted by Sean MacGuire
I've been having a nice quiet time, not a lot going on to upset me. Until this morning when I read the latest via HackerNews http://news.ycombinator.com/ about The Innovator's Patent Agreement: https://github.com/twitter/innovators-patent-agreement/blob/master/innovators...
Twitter is trying to create a sort of 'patent which can't be used to hurt anyone' - from the preamble of their agreement:
"WHEREAS Company and the Inventors believe that software patents should only be used to make a positive impact in the world and, accordingly, should only be used for defensive purposes"
It's essentially a patent assignment document. These documents are interesting because, in the US inventions are invented by people; not companies. They can be *assigned* to companies, which often happens, but the inventor generally gets to keep their name on the patent. (Of course Edison has everyone working in his labs assign all their patents to him, but that's another post entirely).
So here's what they consider a "Defensive Purpose"
An assertion of claims of the Patents shall be considered for a “Defensive Purpose” if the claims are asserted:
- (a) against an Entity that has filed, maintained, threatened, or voluntarily participated in an intellectual property lawsuit against Assignee or any of Assignee’s users, affiliates, customers, suppliers, or distributors;
- (b) against an Entity that has filed, maintained, or voluntarily participated in a patent infringement lawsuit against another in the past ten years, so long as the Entity has not instituted the patent infringement lawsuit defensively in response to a patent litigation threat against the Entity; or
- (c) otherwise to deter a patent litigation threat against Assignee or Assignee’s users, affiliates, customers, suppliers, or distributors.
If Assignee needs to assert any of the Patent claims against any entity for other than a Defensive Purpose, Assignees must obtain prior written permission from all of the Inventors without additional consideration or threat.
And if they dont?
if Assignee asserts any of the Patent claims against any entity in a manner that breaks the promises of paragraph 2, the Inventors, individually or jointly, may grant a patent sublicense to the entity under the Patents
And how long does this last?
This license to the Inventors is not assignable but may pass to the heirs of an inventor in the case that the inventor is deceased.
Wow. It's nice that Twitter wants to make the world a better place; but this isn't the most useful way to go about it; because there are problems here... let's review a few:
- OK, so they can use the patent offensively against any entity that has even threatened an IP lawsuit against any of their users. (a) aTwitter has over 500 million users. That's a pretty gaping hole dontchathink?
- Any company that has voluntarily participated in an IP lawsuit in the last 10 years.
- To deter a patent litigation threat against users...
OK, so I think I can probably drive a truck through a, b, and c., How about if we wanted to get permission from the Inventors to let a lawsuit proceed? They need to get prior written permission from all the inventors. Good luck with that. Can't find one? Too bad. And if they're dead. Too bad... ask their kids for permission. That's a bit foolish really.
Otherwise the inventors can just sublicense their invention to the enemy. Just like that. Sounds like a pretty good strategy to me.
If Twitter really wants to make the world a better place through an agreement to use patents only defensively, then we need a trusted 3rd party to decide the questions of whether the use is defensive or not. No 500 million user loopholes.
How about a Defensive Patent Foundation - sort of a cross between the EFF and FSF who, through the use of a license, can do this correctly, impartially, and better than even the Inventors ever could (provided you could find them).
Patents don't silence little girls... but infringment makes them cry.
Posted by Sean MacGuire
Techdirt has an article about a little girl potentially losing her ability to improve her speech due to a patent lawsuit. Read it here.
Poor little girl, evil patent-enforcing company... the usual nonsense... and I can't take it any more. Inventing stuff intentionally, as opposed to sitting in a room with lawyers trying to figure out if there's a patent in there they can file, is hard. It's also really expensive. And the odds against a lone inventor are terrible... on the order of well over 90% of the time you'll never get your money back.
Yes, there are junk patents out there that probably shouldn't have been issued. This really doesn't look like one of them. The patent is for a Dynamic Keyboard.
I've written in the past about my great-grandfather Thomas E Murray, who was a well-respected inventor with 462 patents around the turn of the century. What I never wrote about was another measure of an inventor's influence - and that is the number of times a patent is cited by other inventors. When you site a patent, it's like telling people "these are the shoulders I'm standing on", they're my prior art. This keyboard patent is cited by 23 other patents. It's a real patent.
Another interesting part of a patent are its claims. The number of claims is a good indicator of the complexity or completeness of a patent. Mine have 30 or so. This one has 124. That's insane.
Now, to the lawsuit. Semantic is suing the makers of the app "Speak for Yourself" for infringement. Guaranteed this isn't the first time these two parties have come into contact. The first thing you generally do in this sort of situation is say "Hey - I see you're using my technology, would you like to purchase a license?". Only if they've been ignored or haven't been able to come to a satisfactory agreement on the license would anyone file suit.
One never goes to court lightly. And this suit was filed in the Western District of Pennsylvania, which isn't exactly Troll Country. These lawsuits can be expensive, an infringement suit can be like $5-$10 million dollars expensive.
The real point here is that the app must be making enough money to make persuing a lawsuit worthwhile. Otherwise it makes no sense to do it.
Off the top of my head, there are a couple of simple solutions to the problem:
- Pay a reasonable license fee. This would be something like 10% of sales or whatever you can negotiate. The probem is they're making money off their app.
- Change the app so as not to infringe.
- Move the app out of the country - to a country where it's not patented. Deliver it as a service. Try Canada. US patents stop at the US border.
A couple of interesting alternatives:
- Hackers - go check out the app. Clone it. Give it away. Richard Stallman did this with Lisp and started GNU. This is right up his alley! Suing Open Source projects with no money makes no sense. And when you take the money out of the equation, then you can really build up some good ol' moral outrage!
- Crowdsource the license fee for the "Speak for Yourself" app. Go to kickstarter, everyone throw in $1, it'll make you feel good.
But the patents aren't the problem here. Making money off someone else's patent without paying a license fee is. And that makes little girls, and poor inventors, cry.
How much cloud do you *really* need?
Posted by Sean MacGuire
Some stuff just makes us lazy. TV, instead of books. Cars, instead of walking. The Cloud, instead of hosted servers. And often, when we get lazy, we also get sloppy...
That's right. The cloud has made us lazy. I can spin up a zillion machines as fast as I can click. Make it absolutely great for prototyping, or cloning environments, or QA... or some incredibly successful client-facing internet thingy that goes from 100 users to a million overnight...
But just because it's the easiest tool for the job doesn't make it the best tool for the job.
And if you've had stuff in the cloud for a while, maybe it's time for a quick review:
- Have I changed the size of any of my instances in the last 6 months or a year?
- How about storage? What does my usage look like?
- How about network bandwidth and traffic? What does that look like?
- Am I sized 'correctly' - even sensibly?
If you've answered 'no' to any of these questions, you're probably paying way too much for flexibility you don't (and may never) need.
I have to admit, some cloud service providers are really getting on my nerves. I hate being nickeled and dimed; S3 compatible storage @ $0.15/gb is bad enough especially when I get charged another $0.15/gb for transport, but when you add extra for GETs and PUTs? Connectria (http://connectria.com) got it right by killing that charge. Just yesterday I ran into a provider that's charging $0.01 per hour per IP address and $0.01 per hour per internet service, Oh Pleeeeasse!!
Availability? Amazon has had a few issues, and Microsoft Azure's leap-day incompetence.
Of course, I won't mention security; and generally it's been good unless you've been holding bitcoins @ Linode... still, hacking the cloud dashboard was a nice touch :) .
Servers are cheap. Virtual servers are even cheaper. So cheap that you could run three of them in different parts of the country (or different countries) and load-balance them. With a little bit of monitoring, you can watch usage trends - and actually plan for what you're going to need.
Of course, migrating all your crap off the cloud can be non-trivial, time consuming and scary, and requires the sort of specialized expertise the cloud takes care of implicitly. The reward is that you're back in control. But it's your data after all... and if all hell breaks loose in the cloud, all they're gonna give you is an apology and a couple of days service credit... you've been warned.
I wonder how long before YC12 sees a 'cloudmover' startup?
three simple rules for geeks
Posted by Sean MacGuire
OK... I've been around for a while and coding for a long time. Here are my top 3 rules which I repeat over and over and over:
What problem are you solving?
Not rocket science. But surprising if you ask that question how often people don't have an answer. If I don't know what problem I'm solving, I'm basically screwed. (Note: this can, and does often happen at the corporate level).
Make it Work, Make it Right, Make it Fast
I don't know who came up with that (Rob or me), but that's been my coding mantra for the last 20 or so years. There's no other sentence that I can repeat to myself that has the impact that has. Because I'm A-D-D, and human, I can spend days doing what Sid-the-sailor called "painting the windowsills" - I may have an entire house in dire need of attention, but I can, and do, become obsessively focused on stupid things.
Make it Work. Tim Gunn stole that part, but no matter what I'm doing, if it doesn't work, Nothing Else Matters. Until it works, don't do anything else.
Make it Right. Debug it. Fix whatever.
Make it Fast/Pretty. In the old days, 'fast' was important. These days, I suppose 'pretty' takes its place. And a lot of the time, this is the fun part.
If you create a Giant Pile of Shit, put a flag in it and claim it as yours.
Guy Martin, my boss, circa 1987. I was working with very expensive SCSI disks, making 4 disks look like one big disk. These disks were expensive; like $7500 a pop expensive. I plugged the power supply in backwards. I saw a small puff of smoke. I brought it into Guy and said 'defecive disk'. He wasn't buying it (probably because of either the smell, or the look on my face).
Sean, "When you create a giant pile of shit, put a flag in that pile of shit, be proud of it, and claim it as yours!" This technique taught me to admit that I make mistakes (and be proud!)... which is always better than trying to hide it.
fear, loathing and coding
Posted by Sean MacGuire
Coding under the influence is fun. Until it's not.
Fear, Loathing and coding
Fear and Loathing in Las Vegas was great. But Hunter S. Thompson never coded. There were times in the early 80's when I'd summon the junior and say Go out and get me Vodka! and I would code into the night, and perhaps into the following day (or 2).
The real problem with this was when programming in an altered state, I might not be able to understand why I'd written something the way I had (certain combinations were notorious for amazing, beautiful, incomprehensible code).
I solved the problem by writing the comments before writing the code, like a little story, about what I was going to try to do next...WRITE ALL COMMENTS ALWAYS IN CAPITAL LETTERS AND TELL THE STORY OF MY LITTLE FUNCTION.
More than once other programmers have commented on the beauty of these comments and how easy it made the code to understand. They didn't realize that I did it so I'd understand my own code the next day.
You might be sicker than you think.
One of the problems with excessive use of mind-altering chemicals is that perception gets a little warped.... that's a feature which becomes a bug. Oddly enough, the perception rarely is that this is a bad thing, and is causing real problems. It's everything and everyone else.
It's virtually impossible to self-debug your own mental problems. If the computer is broken, it's going to be putting out crap results, no matter how good and sensible the input is. So that phrase "you might be sicker than you think" is just there to keep the door ajar - that maybe your perceptions of your conditions are just plain wrong, and sometimes you need to let someone else do your thinking for you, for a while. Then you look back and say holy shit, was I ever sick!
You're where you want to be.
There was a point in my life where things really and truly sucked. (See above) I was drinking a lot, blaming a lot, and generally depressed and miserable. Then the small light went on... From the age of about 18, nobody was responsible for my actions, decisions, and all that crap but me. If I'm here in the dumps, I've chosen this path; it's nobody's fault but mine, and nobody owes me shit. You can choose to change your path anytime. Accept help.
The truth is simple, lies are complicated.
I'm not going to go into the etymology of this one, except to say that in my family, honesty is a commoidity. If something sounds unduly complicated, there's probably a lie in there somewhere. Act accordingly.
If you think you have a problem, you probably do.
And if you don't know what to do about it, or who to talk to, drop me a line at sean@maclawran.ca. You don't know me from a hole in the wall... so it may actually be easier to mention what's up.
And as my old friend Jean B used to say, "Good luck to you".
We sold out...
Posted by Sean MacGuire
I got a surprise a few weeks ago... my manager at Quest called and just said "Hey Sean, your last day is next Friday... (blah blah blah)... do you have any questions?". I said "no" and hung up.
I wrote Big Brother - the first web-based Systems and Network monitoring program back in 1996. I wrote it to make my life as a Sys Admin easier... actually I wrote it because a salesman pissed me off and wanted to charge the company I was consulting for $250K but demanded I spec the entire system out. And he was mean to his technical guy. That was it. I wrote the bones of BB in a weekend. And I put my face on it... "Big Brother is Watching".
I also wrote it because I got tired or watching things manually. Netscape had just opened up the NSAPI, literally that month, and BB was one of the first programs to dynamically generate web pages. The idea was I could install it, leave it onscreen and tell from across the room whether things were OK (green), or there was a problem (red).
The problem with being a Systems Administrator is visiblilty. If you're visible, it's generally because there's a problem (which is of course, your fault), and needs to be fixed. If you're not visible, it's "why are we paying this guy so much?". Big Brother was helpful here in a bunch of ways - first it published system and network information in a "red is bad / green is good" format easily understood by PHB's. This kept them busy. If there was a problem, they could just hit "refresh" instead of "redial". Good for me, because the last thing i want to do when I'm putting out a fire is discuss putting out the fire..."
And Big Brother became popular. Used everywhere. Like Nagios is today.
I didn't realize it at the time, but we were causing some companies real pain. Quest was one of those companies. They were selling their Foglight software at about $50K a pop. BB covered a lot of the same ground for free. Quest salesmen would hit a place and see Big Brother already there. They called us 'the virus'. So they did the smart thing - offered us a pile of money and bought BB. We sold out.
I didn't expect the downside. Quest is a sales company. They could be selling Ginsu knives - they're a sales machine, it's what they do, and they do it well. BB was a 99.5% free product and the sales guys saw a big mailing list.
For 4 years I tried to hold the community together. I couldn't mention how Quest had forbidden us to upgrade the free product, or cease providing support on the mailing list. Sales guys wanted to spam the list of people who downloaded the free version - I told them they couldn't - that it was against the privacy agreement we had - even went to the company lawyers to make sure we wouldn't violate our own Privacy Agreement.
So the privacy agreement got changed... then there was the "upgrade or die" message that went out - that killed the community. Thank goodness Henrik created Xymon, a Big Brother clone ("can't we sue them for copying... no you don't want to do that"), and salvaged what was left of the brotherhood.
I've spent the last 10 years there taking care of BB. Spent the last year architecting a cloud version (that I really like). It's not like there are a lot of dev resources... there was me, Rob and Jun. I was responsible for 'the vision thing', new features, and the User Interface.
My departure effectively puts BB in maintenance mode. And it's really strange to not be associated with this thing I created.
And I need to look for work. And because I had no idea this was coming, I was really blindsided. Couldn't find a copy of my resume anywhere (I finally found a copy on The Wayback Machine - the archive of old web pages maintained by the Internet Archive).
And it's scary. I'm about to turn 50. 50? And I suspect I'm about to deal with Ageism for the first time. This is going to be interesting. It certainly focused my attention. And living in Key West? Not a huge demand for senior Unix guys down here (except at the Navy base).
I'm lucky because Quest has a sensible policy regarding Inventions - you can invent something and it's yours provided it's done with all your own resources and on your own time. I also always have a side-project going - generally to do with new stuff I want to learn about - so it's not like the skills aren't current (video, cloud, jquery, big storage).
So what's next?
I'd like to go back to doing SysAdmin work. Taking care of machines and networks... there's never been a period where I haven't taken care of machines and a network, so that's pretty comfortable. I suspect I may run into "But you haven't used Version 7.1 of blah blah" (look in the resume I probably wrote something similar to blah blah at some point).
I like being a Sys Admin.... I used to describe my work as a combination of Clint Eastwood and Mr Spock - rarely have I run into problems that required a committee to solve them; that's what I do, it's what Iike to do. Even dealing with users. I've mellowed - I'm no longer the BOFH (Bastard Operator From Hell)... then again put me in a room with enough stupid users for long enough and that may change.
And there are the patents. 2 of them, with more being filed. My favorite is the "Consumer Self-Activated Financial Card"... you get the card first, then activate it yourself. Maybe Paypal, Google, Facebook or some bank might be interested. Activation patents are cool because if you can't activate, you can't do anything else.
See it here: http://telicash.com and click on any of the pictures.
No matter what I'm sure this is going to be a trip.
And, by request, a link to my resume: http://root.sh
Virtual Private Servers - the Wild West of Hosting
Posted by Sean MacGuire
What a great few years this has been. I've always had a machine or 10 online. Initially, we had to go out and find someone willing to allow us to put one of our machines in their facility, so we could have it online. It was Co-location. It was expensive. But it was good.
Now, I'm spoiled. I can get monster sized machines from Amazon with infinite disk. But Amazon is expensive, and I don't expect a billion-user spike any time soon (although, who knows). The next great achievement has been virtualization - the ability to 'chop up' a bigger machine into a pile of smaller, virtual machines.
These virtual machines, called Virtual Private Servers (or VPS's), are cheap. You can get your own instance of linux (I prefer Ubuntu 10.10/11.04 at the moment), for very very little money. Like $10 a month, sometimes less.
You can learn about this wild west of cheap boxes here: http://lowendbox.com and here http://www.webhostingtalk.com/
I use these for lots of things, IPV6 testing, mail servers, web servers, blogs, development, distributed monitoring since I can locate each VPS in a different area (or country for that matter). One of my favorite things to do is use these for scaling, since you can make each VPS bigger, or just buy more of them.
Combine them with Connectria for S3 storage as a linux filesystem - see my post here http://blog.maclawran.ca/s3-compatible-storage-as-a-linux-filesystem, and you too can play 'whack a mole' - someone takes one of your machines offline, if you set it up correctly, you could have it back online at another provider in minutes. Poor mans elevated availability :)
Of course such incredibly cheap access to hardware means all sorts of somewhat nefarious things can happen. On the provider side, they can oversell (so, you've bought a brick), have lousy network speed, or be a haven for bad guys (skiddies, spammers), or just have lousy or non-existent support.
Did I mention "avoid NordicVPS like the plague"? Sure looks like they're circling the bowl: https://twitter.com/nordicvps
So with so many providers, how can you tell which ones are good? Here's my list (remember I've been doing this since 1983... every level of support and administration concievable, including being a BOFH for a while - see http://www.theregister.co.uk/odds/bofh/)
First - the basics. Providers will try to oversell if they can. Virtualization comes in a few flavors, most often seen are OpenVZ, Xen, and KVM. OpenVZ can be insanely oversold. In reality it also means that if you your processes use too much memory, you'll crash hard. Not nice.
I haven't played with KVM, so I'll pass on that. I've standardized on Xen. Under Xen virtualization you're guaranteed a certain amount of memory, so even if you eat up all you've got, your machine will degrade in a predictible manner, swapping, thrashing, then stopping. But, it can't be oversold, which is good.
For management interfaces I like SolusVM, which most providers use. I've also standardized on Ubuntu 11.04, but anything over 10.04 is actually OK, plus for me they have to support IPV6.
So you find someone. How do you know if they're any good? Guess what. You don't. Bwahahahahaha! But if they support IPV6 today plus Xen, their inherent cluefullness is higher from the outset.
But that's what this post is about. I've tested a bunch of them, having accounts on them, asking questions, using their networks, etc. It's also interesting to see how they handle their own outages and issues, and how well they keep their clients informed when things are going weird.
For network speed, here's a command to test with the full discussion here: http://www.lowendtalk.com/discussion/112/test-your-vps-network-connection
wget -O /dev/null http://cachefly.cachefly.net/100mb.test
I've tested their support. Really tested them, because I'm hacking late into the night pretty well every night. And by late, I mean bedtime is after 5 am. And since I've been doing this for such a long time, if I ask support something, it's Level 3 support asking for support, so it's going to be interesting.
So I ask hard stuff in the middle of the night/Early in the morning. On Christmas Eve, Christmas Day... and I know instantly how good these guys are by two metrics; first the response time, second the quality of the response.
What's interesting, is that in some of these cases, these are small hosting companies. It means they haven't become large enough to become cheap and incompetent, maybe you're dealing with one of the owners. It's happened. it's generally a good thing.
And once I've established a link with support (and they figure out the clue-level on the other side), then they're happy and I'm happy. And support has been great.
I'm not going to list everyone. Suffice it to say that about half fail within 2 months.. I'm currently using:
- Empire Hosting http://empire-hosting.net - cheap - great support - blazingly fast @ 10Mbps
- YardVPS http://yardvps.com - really cheap for disk - support OK - network @ 2.7 Mbps
Honorable mention to:
- Host Virtual http://vr.org - I like these guys. They have an excellent worldwide network, support is great. and network speed was 2.46 Mbps. Not too cheap.
- GoVPSGohttp://govpsgo.com - relatively expensive, very good support, fast network @ 4.77 Mbps
- Linode http://linode.com - expensive, no IPV6, really fast network @ 11.9 Mbps (correction - IPV6 in 5/6 locations)
So if I need Worldwide VPS's - http://vr.org wins hands down, with great support and nice guys. If I want blazing speed, value, and great support it's Empire Hosting http://empire-hosting.net. Finally if I want the best bang for the buck and can live with a slower network it's http://yardvps.net.
Any questions?
Depression, SAD, and Zhineng Qigong
Posted by Sean MacGuire
I had SAD (Seasonal Affective Disorder), and brutal Montreal winters guaranteed that by February, I'd be in a black pit. I really really didn't want to take anti-depressants, and looked for another solution.
I learned that the Inuit got SAD when they changed from their traditional diet high in fish oil to a Western diet, which was the first explanation that made sense. So I added Omega-3 to my diet. I helped. Also added a whack of Vitamin D, because it made sense too - we get that from exposure to the sun - that would be about 3 months a year in Montreal.
More here for those who are interested... http://www.moneycrashers.com/frugal-ways-to-treat-seasonal-affective-disorder/
6 years ago I was in Las Vegas with my wife and her mother, Betty. Staying at the Wynn, which was packed and desperate for another room, which they couldn't help with that night. So at about 3 am that night I found myself looking for a room elsewhere, feeling like crap and really mad because I'd lost money. Got a comped room at Harrah's, and upset I turned on the TV... landed on PBS... and saw this:
I was sufficiently upset and miserable that I was willing to try anything. I spent the next 90 minutes following along. After that, I felt good. I noted that - this stuff works, then promptly forgot about it.
5 years ago, in the winter, the black depression, on schedule, returned. In a desperate state, I looked up Qigong in Montreal and had the incredible good fortune to meet Master Yang. Master Yang came from China where he had studied TCM, Traditional Chinese Medicine, Accupuncture, and a variety of Martial Arts.
I began to learn about Qigong at his studio in Montreal: http://internalstyle.com/
Qigong (pronounced Che Gung) is an internal martial art as opposed to something like Kung Fu, which is an external martial art. Qi (sometimes written chi) means Energy. Gung (sometimes written Kung), means Work. So Qigong literally means "Energy Work". It's a form of moving meditation, very slow motion, slower than Tai Chi.
There are hundreds of kinds of Qigong, if not thousands. This style is Zhineng Qigong, created by Grandmaster Dr Pang Ming in China in 1979. However, finding a good teacher is hard. All I can suggest there is that a good teacher will not just be teaching Qigong, but is likely to be schooled in TCM, accupuncture, and martial arts like Ba Gua.
I'd tried meditation in the past but having ADD and having spent a lifetime fidgeting made sitting still for any length of time virtually impossible.
The Chinese system is different than the American system for teaching. It's almost like in the Chinese system you're told to do stuff and when the teacher thinks you're ready, you're told why you've been doing this stuff. So initially I took a lot on faith; that they'd been doing this stuff for a long time - since maybe 200 BC, and millions of Chinese do this daily, so there must be something to it. Besides I was suffering and willing to learn.
It was strange, right away. After a short warm up and doing some very slow motions, we sat down cross-legged and I began to sweat profusely - not from exertion, which is why it was so confusing. Evidently, stuff was getting 'unblocked'. You'll hear a lot about things being 'blocked' in the Chinese system. Get rid of the blocks, restore balance, everything is OK.
I soon began to be able to feel 'chi' - energy. It's strange, sort of like the feeling when you hold two magnets together - the attraction or repulsion... sort of like that. The depression lifted and hasn't returned. And my health improved. The year I began I was constantly sick; my wife was worried about my health. Since practising I've been sick for about 8 hours in the last 5 years, and not at all in about the last 3 1/2 years. It's not that I don't get sick, it's almost like I get sick, get a super-mild version of whatever it is, and it exits. Very very strange.
And there are huge and well-documented benefits to meditation; given that I think for a living, this exercise at the end of the day gives me 30 minutes to myself, to quiet down, and empty my mind for a while. I don't understand how it works, all I know is that I keep doing it because of the benefits. And I haven't missed a day in over 5 1/2 years.
Here is rare footage of Master Yang, showing the warmup and the first routine.
This is the perfect exercise and discipline for sedentary geeks like myself. I continue to be grateful to Master Yang for showing me.
And the SAD? Gone. Moving to Key West cured it for good - now if we could only get Master Yang down here to teach!
In the meantime, an excellent book: http://lifeqicenter.com/CD,_Books_%26_Links.html
Software patents and pony dung...
Posted by Sean MacGuire
If I read one more "kill software patents" article, I'm going to puke.
Joel's is the latest: https://plus.google.com/u/0/117114202722218150209/posts/4GgaRiSyaTf
Here are his 5 points:
- Elimination of software patents
- Legal fees paid by the loser in patent cases; non-practicing entities must post bond before they can file fishing expedition lawsuits
- Roll back length of copyright protection to the minimum necessary "to promote the useful arts." Maybe 10 years?
- Create a legal doctrine that merely linking is protected free speech
- And ponies. We want ponies. We don't have to get all this stuff. We merely have to tie them up fighting it, and re-center the "compromise" position.
Maybe we'll look at these backwards, because there's so much crap here there has to be a pony in here somewhere. (That takes care of #5).
Merely linking is free speech, no. Free speech is free speech. Knowingly linking to things that are dangerous, illegal or whatever should be discourgaged. Child porn and hate speech come immediately to mind, followed by stuff like "how to weaponize the H1N1 virus". (That was just to make sure the NSA reads this post).
Copyright protection 10 years? Why the minimum necessary? For this, whatever a majority of countries can agree on.
Now it gets interesting... Patents. I do have an opinion on this.
Elimination of Software Patents.... My great, great grandfather was Thomas E Murray, an inventor with (at least) 462 patents to his name who lived at the time of Edison. He has patents on everything from the powerplant to the light socket (and piles of other stuff too). He was inducted into the National Inventors Hall of Fame last year... here's his site: http://temurray.com
One of the things he invented was the little valve below. And a pile of other things like it; what he really did was to invent a new method of welding that let him stamp 2 molded steel halves and weld them together. That's a pretty basic thing. But new technology allowed it to happen.
I mentioned in an earlier post how tough it is to be an inventor. I recently did an inventory - of 11 provisional and actual patents filed, I now have 2 granted, and one pending. The patent system is tough. Before being critical of it, maybe learn about it and try it sometime.
For a patent to be granted it has to satisfy 3 conditions:
- Novel (something really new)
- Useful (no perpetual motion machines)
- Non-obvious (this is the tricky one)
Assuming you can come up with something novel and useful (search google and the USPTO at http://uspto.gov), the real challenge is coming up with something non-obvious. It's almost guaranteed that when you file a patent, the examiner will shoot back a reply (a couple of years later), saying "go away - obvious" or "anticpated by so-and-so in this patent here". So the process of argument starts. Back and forth, back and forth. If you can convince the examiner that you're novel and non-obvious - you get a patent. Remember they do this all day; they're really good at saying no.
Now, what appear today to be bad patents may well have been granted, especially a decade or so ago when this was a mystery to the examiners - (see that valve, it's good to be first). But good inventions often inspire the reaction "wow - I thought that already existed", or "I wish I'd thought of that"... that stuff appears obvious when disclosed. And that only gets worse over time. Amazon's one-click *really* looks obvious now, doesn't it? Remeber we're standing on the shoulders of giants.
The problem isn't software patents, it's programmers ignorant of software patents. You think what you're doing is unique? Ha. Prove it. Go check the patent office. Do a search. Search google. And if it is unique? You can file a patent. Or throw it into the public domain to prevent others from patenting your stuff.
Almost all the patents I've filed have been defensive - it tells VCs, me, and the world, that I've done the homework, that this is new, novel, non-obvious, and reduces the risk of us getting sued down the road. And a bit of obviousness? You only get sued if you're successful. Congratulations.
I'm best known for a bit of software called Big Brother - the first web-based systems and network monitor - still available at http://bb4.org even though I'm no longer associated with the product (thanks Quest). Lots of new stuff there at the time. What did I do? I threw the initial version of the product into the public domain - published in an article for Sys Admin magazine. I got a call from an examiner at the USPTO - asking about monitoring software - and I referred him to the article. That's prior art. The guy trying to patent my stuff didn't get his patent...
So finding prior art is pretty well the definitive solution to bad patents. Article One makes money by crowdsourcing this stuff - http://www.articleonepartners.com/ - no reason we couldn't do this in an open-source manner either (and it does look like they've found prior art on the Lodsys patents).
The good news is that with all the open-source source code out there (easily searchable by google), this prior art searching is getting much much easier (no more going to paper journals for example)... so expect the quality of patents to improve just because of that.
I also suspect that there's no reason not to crowdsource a patent-defense system - where each of the Lodsys victims contribute $100 towards a shared defense and the crowdsourcing the prior-art search to invalidate the patent... this would be an effective defense against the next Lodsys. (Anyone interested in this? Y-Comb?)
Now, the issue of "Non-practicing entities", aka Trolls. Joel says: "Legal fees paid by the loser in patent cases; non-practicing entities must post bond before they can file fishing expedition lawsuits".
I think the pony lives in there.
I moved from Canada to the US 3 years ago. The Canadian system, a pretty socialist system, has some interesting points, amongst them:
- Lawyers aren't available on contingency
- Lawyers can't advertize (but their firms can now)
- Loser pays
On the plus side - there's no "After 911 Call 411 - 1-800-411-PAIN" in Canada.
Downside is that the little guy is guaranteed to get fucked by the big Corporation. Unless you are incredibly wealthy you don't sue, you just eat it. (At least with Socialized Medicine you're not stuck with a million dollars in medical bills).
As I mentioned in my eariler post, there are a couple of salient points:
- Inventing is a dismal business with a 90%+ loss rate (my guess would be 99%)
- Even if you get a patent, you have to defend it somehow.
- Patent infringement suits can cost $5MM
- Even with lawyers on contingency, you can still spend hundreds of thousands of dollars
Trolls give the little guy a chance. And they're not stupid, they're not going to take on a case that they don't think they can win, because that would just be stupid, and courts don't take kindly to people wasting their time.
And they exist because they have to. The world has changed. The US is incredibly litigious. Judy Judy is #1 (and look at what she has to put up with).
In the old days (I've been told), large companies used to send their IP guys out to chat with each other. After a nice expensive dinner, one would say to the other "I need to bring something back to our CEO", and they'd do a licensing deal. Nice and simple.
Those days are over. We're in the time of "Go ahead and sue me". Or maybe buy insurance against being sued. Or join a cartel of patent holders where you can play "Patent Cold War".
Stop whining about Software Patents. Learn about the system. Let's see if we can hack it.
1 page of 2 | Next >>
Subscribe via RSS