#arpnetworks 2009-09-30,Wed

ballen

also portupgrade

sroute

sudo portsnap fetch && sudo portsnap extract - first go around

ballen

which is a port in its self

sroute

from then on sudo portsnap fetch && sudo portsnap update
Very fast.

jeev

yea man i have a ton of servers, i just wanted a vps

[10:52]

sroute

First extract will take a while of course but from then on its slick.

[10:52]

mike-burns

This replaces cvsup?

[10:52]

sroute

jeev - for all intents and purposes the vps will look like a real server - you can even rebuild and install your own kernel
mike-burns: yup
mike-burns: as far as I can tell there are no downsides to using portsnap over cvsup - probably cvsup for ports will one day be deprecated. Or should be.

[10:53]

mike-burns

Cool, I'll try it next time I upgrade.

[10:54]

jeev

how funny
a clients server ram just failed

sroute

I'm running an update on one machine right now... just started. I'll tell you when it is done

jeev

i just started using svn on a test server for 7-stable

sroute

Done.
$ sudo portsnap fetch && sudo portsnap update
Looking up portsnap.FreeBSD.org mirrors... 3 mirrors found.
Fetching snapshot tag from portsnap1.FreeBSD.org... done.
Fetching snapshot metadata... done.
Updating from Sun Sep 27 13:13:53 PDT 2009 to Wed Sep 30 08:50:51 PDT 2009....
--snip-- extracts a bunch of new files.
like I said, the first extract will take a while, as would an "update" if you do one say six months from now. But it is far faster than cvsup for most users and certainly is so for frequent users.

mhoran

portsnap is great, as is freebsd-update
(Written by the same guy!)

[11:04]

mike-burns

Makes sense.

[11:05]

mhoran

I wish the ports system had a config management system similar to that of mergemaster.
I don't like having to figure out that on my own.
Granted, /usr/ports/UPDATING usually helps, but still.

[11:06]

mike-burns

Which ports give you issues?

[11:06]

mhoran

Dovecot changes its options every point release. It's obnoxious.

mike-burns

Oh, that's harsh.

mhoran

There are always little things that break when the defaults change. I want to know that.

BeBoo_

sroute, you know you can do portsnap fetch update ?
instead of 2 commands

mike-burns

I wish something were automated for PostgreSQL upgrades, along that line too.

mhoran

... not have to recovery a week later when I realize my mail server has been down because Dovecot hasn't started.
BeBoo_: Yeah, the combined command is nice.
There's alsoa cron command that doesn't actually do the merge to be safe, but I'd like to implement that someday to e-mail me weekly with pkg_version -v -l < or something similar.
mike-burns: Debian has some stuff, but, yeah.

[11:08]

***

ballen has quit IRC ()

[11:09]

jeev

you guys use suphp ?

mhoran

Never heard of it.

jeev

its like suexec, for php

mhoran

Huh!

sroute

BeBoo_: thanks. an old habit which will change now...

BeBoo_

jeev: i believe i have in the past

mhoran

I use lighttpd, so my config is a bit different.

jeev

cool
yea i use lighttpd for a cust

BeBoo_

sroute: does the same thing but saves some typing. =]

jeev

funny thing is his server got hacked with lighttpd cause of a script

mhoran

jeev

no, the sites on apache got owned, not lighttpd
i dunno why, they were all in one fodler
i suggested we move each site to a new username

mhoran

Interesting.

jeev

we have yet to do so though
it's best to segregate *

sroute

sroute thinks it is best to rm php*

mhoran

We're in the process of moving our apps off to separate VMs, since our Wordpress install just got owned.

BeBoo_

jeev

seriously

mhoran

sroute: Agreed.
Too bad we just deployed a huge Joomla installation.
Whoops!

BeBoo_

jeev: can always jail each site if you wanted to go that far

[11:13]

sroute

I think I will have to break down and write a minimalist webmail app so that I can actually remove PHP from one of our boxes. At least I have the one app - squirrelmail - running via PHP in a jail

mhoran

Hm. Maybe I'll look into suphp for our wordpress install.

jeev

i've gotta get more into it and see what's going on
what do you use to jail

sroute

This jail is by hand; I have used ezjail as well too and like it when I am doling out large numbers of them.

jeev

ahh

sroute

I don't suppose anyone has a decent webmail app, non PHP, preferably in Python, they have experience with and like?

jeev

nope, i use horde

mhoran

Hm. That'd be nice.

BeBoo_

sroute: http://bobomail.sourceforge.net/

[11:16]

sroute

Am seriously thinking of firing up a project. I have some experience writing mail apps but not sure I want to dive that deep.

[11:16]

jeev

cat believeskewed
[11:11am] <xsamurai> fatalnix:
woops
cant believe i clicked on the localhost link

[11:16]

mhoran

bobomail really needs some UI help. :)

[11:17]

sroute

BeBoo_: I believe I've run across that link before... quite ancient - dates back to Python 1.x days and that was a LONG time ago.

[11:17]

BeBoo_

ah. i just googled =[11:17] <mhoran> # Apache 1.3.11, Apache 1.3.12

[11:17]

mhoran

# Mozilla M15/ Netscape Navigator 4.7

[11:18]

sroute

"Bobo" is a term that Zope used to be referred to as.

[11:18]

mhoran

Yeah, this uses Zope, it seems.
Looks dead. :(

[11:18]

sroute

There just isn't anything for Python that seems active; a couple of times a year I check out the state of affairs.
http://posterity.edgewall.org/ is one... but also seems inactive. Just not as old as bobomail

[11:19]

mhoran

I'd like the UI of Roundcube (without the OS X style, but the AJAX and general layout is nice) written ins omething other than PHP.
Then I'd just need to find a replacement for Mediawiki (probably not going to happen) and phpBB (yuck).

[11:20]

sroute

http://divmod.org/trac/wiki/DivmodQuotient is another, seems stalled. Not quite sure what those folks are up to these days.
Someone started a project on Google Code last year http://code.google.com/p/webpymail/
But no updates since January.

[11:21]

***

Thorgrimr has quit IRC (anthony.freenode.net irc.freenode.net)
toddf has quit IRC (anthony.freenode.net irc.freenode.net)
timburke has quit IRC (anthony.freenode.net irc.freenode.net)

[11:24]

mhoran

Hm. Why is my lighttpd serving up an expired certificate. :(

[11:24]

sroute

I guess OpenWebMail (perl) might be an option. http://www.openwebmail.org/
Or I stick with sqmail in a jail, where all PHP apps belong. squirrel mail probably isn't that bad off although there have been the odd security issues with it, as there are with most all software regardless of language

[11:24]

.... (idle for 17mn)

***

Thorgrimr has joined #arpnetworks
timburke has joined #arpnetworks
toddf has joined #arpnetworks

[11:42]

.... (idle for 17mn)

up_the_irons

wow, i got some scrollback to read, don't i

[11:59]

sroute

while the cat is away...

[11:59]

BeBoo_

lol

[12:01]

up_the_irons

sroute: i also tried to "sleeping" thing last night, but woke up at like 4AM and had to get on the laptop

[12:04]

BeBoo_

"sleep" ? what is that?

[12:05]

mike-burns

It's something that doesn't work right when you mix FreeBSD and KVM.
mike-burns is pleased with that pun

[12:05]

up_the_irons

as i'm reading the scrollback about portsnap and cvsup, let me say i wish they deprecated CVS entirely and just used git; git is so much faster than any alternative out there, by a mile; anyone who has used it knows what i mean

[12:08]

mike-burns

git is the fastest I've tried, but there has been a crazy amount of improvements to darcs over the past month.

[12:10]

jeev

git is cool but was pissing me off
i had messed up the kvm branch of git and wanted to redownload everything
it kept saying already up to date, do you know what to do in that situation ?

[12:10]

mike-burns

git reset --hard ?

[12:11]

up_the_irons

BeBoo_: "sleep" is something they used to make a while ago; it got discontinued due to lack of sales
jeev: why not just re-clone?

[12:11]

jeev

i didn't know what to do
i rm'd it

[12:12]

up_the_irons

if you want to really "start over"
what to do would depend on how you "messed up the [branch]"
no universal solution

[12:12]

mike-burns

I've suggested before: I'd love if there was a 'git undo' command, that I could keep typing until I got to the initial checkin.
It'd just magically undo whatever I just did, and if it gets to the point where I cloned it then it'd undo what the prior person did.
That's my dream.

[12:13]

up_the_irons

i saw some alias for 'undo' i think, not sure if it worked well
mike-burns: there's some nice tidbits here sometimes: http://www.gitready.com/

[12:14]

mike-burns

Hah. gitready worked next to me for six months.

up_the_irons

worked?

mike-burns

He's a CS students at RIT.

up_the_irons

ah , haha
nice

mike-burns

He had an internship at my company.
Any git problem I had, he fixed. It was great.

up_the_irons

LOL
office time
cd $office

[12:16]

BeBoo_

No such file or directory

[12:17]

............... (idle for 1h14mn)

dbgi

[13:31]

***

ballen has joined #arpnetworks
BeBoo_ has quit IRC ("Leaving")

[13:42]

....... (idle for 30mn)

ballen has quit IRC (Read error: 113 (No route to host))

[14:16]

vtoms has quit IRC ("Leaving.")

[14:24]

......... (idle for 43mn)

visinin has quit IRC ("leaving")

[15:07]

.............. (idle for 1h6mn)

ballen has joined #arpnetworks

[16:13]

ballen

ping

[16:14]

.............. (idle for 1h8mn)

***

timburke has left "Leaving"
timburke has joined #arpnetworks
nuke` has quit IRC ("gone.")
nuke` has joined #arpnetworks

[17:22]

..... (idle for 24mn)

sroute

python
wrong window.

[17:50]

ballen

indeed

[17:50]

sroute

for x in range(100000000): print("oops")

[17:51]

ballen

(0..100000000).each { print "oops" }

jeev

dam man
qemu is going slowwwwwwwww, without kvm

ballen

ruby is so much better :-)

jeev

install ;D

ballen

prints = puts though

sroute

ballen - whatever fits our brains is better
thankfully PHP isn't among the choices for you and me LOL

[17:53]

ballen

yea I agree, its just fun to argue
yea PHP is lame

[17:53]

sroute

i'm too old for language wars myself
but it can be fun... once in a while

[17:53]

ballen

ever do fuzzy logic searches ?

[17:54]

sroute

not as a matter of course... why?

[17:54]

ballen

just trying to figure out a way to do it that doesn't involve multiple calls to the db
searching for a name
the db has First Last in a field

[17:54]

sroute

ah; for that I've had to rely on search engine technology in my past.

[17:55]

ballen

yea

[17:55]

sroute

Used to do a bunch of work with Fulcrum, OpenText
document and content management systems, workflow, imaging - that was my area of expertise.

[17:55]

ballen

ah right on
meh screw fuzzy logic
not worth the crap ton of extra queries
or storing an extra index

[17:56]

sroute

Postgres has (or had, been a while since I looked) a "soundex" library
Might be worth looking around for something similar for your db of choice.

[18:00]

ballen

yea not using a relation db for this project

[18:01]

sroute

Ah.

[18:01]

ballen

likly I have the speed to do the queries
of the various permutation of the search
but it seems like an ugly way of doing it
I guess I could ensure everything is downcased
for the search
that would eleminate some issues

[18:01]

sroute

Not looked at soundex algorithms of late but is one approach. Of course non-English names complicate matters greatly.

[18:02]

ballen

hmm yea very true
http://unirec.blogspot.com/2007/12/live-fuzzy-search-using-n-grams-in.html
there n-grams approach

[18:03]

............ (idle for 59mn)

***

getchar has joined #arpnetworks

[19:02]

up_the_irons

getchar!
how very C of you

[19:02]

ballen

what now

[19:02]

sroute

ballen: just for fun I built a quicky Levenshtein distance calculator; a brute force run through a list of 200,000 words takes quite a while to return this for the search value "Mike":
>>> def closest(key, iterable, top=5):

***

up_the_irons is now known as fgetc

sroute

... result = []
... for i, value in enumerate(iterable):
... result.append(i, value, levenshtein(key, value))

fgetc

getchar: sup

sroute

... return sorted(result, key=lambda x: x[2])[:top]

***

fgetc is now known as up_the_irons

sroute

sroute wonders if I've stumbled into #Remedial_C_for_insomniacs

[19:06]

ballen

hah
sroute, whats the levenshtein method look like

[19:06]

sroute

gonna grab some dinner

here's an implementation: http://hetland.org/coding/python/levenshtein.py

[19:07]

ballen

thx

[19:07]

sroute

precalculating ngram style like the better approach for a big word corpus
but for a smallish body... lev might be just fine. 10 seconds for 200,000 words on my slow dev machine

[19:07]

ballen

yea trade off being using up more storage
in which my db runs in memory

sroute

anyway, fun to think about this stuff again. I actually need a fuzzy match solution.

ballen

so storage is a premium

sroute

my db is a python object db, not a SQL back end

ballen

thanks for the suggestions

[19:09]

sroute

see, Python and Ruby folks DO get along LOL

[19:09]

ballen

hah
indeed

[19:09]

sroute

sroute looks for a PHP maven to stun with a Taser

[19:10]

ballen

lmao
hmm I'm guessing I could iterate though name index
see how it does

[19:10]

sroute

mostly (partly?) kidding.
c u folks later... dinner calls.

[19:11]

up_the_irons

l8r

[19:15]

ballen

prolly could use RubyInline to write the method in ruby
in C

[19:15]

jeev

up_the_irons
qemu is so nasty without kvm

[19:16]

up_the_irons

jeev: it's a software emulator, so sure it's nasty ;)

[19:19]

jeev

nasty.
xen is SO much faster

[19:19]

up_the_irons

xen is not a software emulator
you're comparing apples and oranges on that one
i gotta leave the office now, talk to u guys later

[19:21]

ballen

k bye

[19:22]

***

heavysixer has quit IRC ()

[19:24]

jeev

ahh
what's the difference then between xen and qemu

[19:33]

ballen

Xen is paravirtulizaed, Qemu/KVM is full-vert

[19:34]

jeev

what if it's not KVM
just Qemu vs Xen

[19:35]

ballen

same diff just not running in the kernel
or rather not using Intel VT or AMD-V CPU extentions

[19:37]

jeev

hmm
the qemu box is crawwwwwwwwwwling on a slackware install
but xen, debian was fast
i know it's not necessarily debian that's fast
but i think it's just qemu that is crawwwwwwwwwwwwling

[19:42]

ballen

xen is inertially faster than Qemu, but requires a modified ghost OS kernel

[19:43]

..... (idle for 22mn)

***

dj_goku_ has joined #arpnetworks

[20:05]

...... (idle for 26mn)

dbgi2IAm has joined #arpnetworks
dbgi has quit IRC (Read error: 104 (Connection reset by peer))

[20:31]

sroute

ballen... turns out the fastest way to get fuzzy matching in Python is to look at the docs. difflib module fills the bill...

[20:32]

ballen

heh how about in Ruby?

[20:32]

***

dbgi2IAm is now known as dbgi

[20:32]

sroute

Here's an example - lets get 17,000 words or so in a list
>>> W = [w.strip() for w in open('/usr/share/dict/words') if len(w) < 7 and len(w) > 3]
>>> len(W)
16920
>>> difflib.get_close_matches('Mike', W)
['Mike', 'Mikey', 'sike']
>>> difflib.get_close_matches('Fred', W)
['Fred', 'red', 'roed']
>>> difflib.get_close_matches('naancy', W)
['nancy', 'nanny', 'nacry']
>>> difflib.get_close_matches('heelga', W)
['heel', 'wheel', 'shela']
>>> difflib.get_close_matches('sheilla', W)
['shill', 'shell', 'shela']
>>> difflib.get_close_matches('Bobbty', W)
['Bobby', 'sobby', 'pobby']
Not bad. Pretty fast too.

[20:33]

ballen

nice, any idea on big o complexity of it?

[20:33]

sroute

sroute chalks this up as learn something new today.
difflib module I see is 2000 lines long. Let me have a quick read

[20:34]

ballen

feck
2k lines

[20:35]

sroute

get_close_matches however is not very long at all.
However it does depend on a SequenceMatcher class which is somewhat lengthy.
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/293796

[20:36]

ballen

nice thanks
likely it won't work with Ruby 1.9
but will try

[20:39]

sroute

meh. apparently it is October somewhere on the planet already.

[20:40]

ballen

ew
I don't want snow

[20:40]

sroute

mailing list reminders have an undesirable side effect of reminding me how fast time passes
snow? isn't everyone located in Monaco like me?

[20:40]

ballen

heh, nope

[20:41]

sroute

lol it was 33 degrees (C) in Calgary last week while I was there; back in Vancouver it was 6 or 7 C this morning.

[20:41]

ballen

50F here

[20:42]

sroute

Not looking forward to snow either although I think the 2010 Winter Olympic organizers in this city are sure hoping it does before Feb.

[20:42]

ballen

likely

[20:43]

jeev

i wish i could code

[20:45]

sroute

me too ;)
more seriously... if you want it, just dive in.
http://diveintopython.org/
Or in Ruby (I don't know anything about this site) http://rubylearning.com/
cul8tr

[20:48]

ballen

heh
CSS can be annoying

[20:51]

***

up_the_irons has quit IRC (Read error: 60 (Operation timed out))
mhoran has quit IRC (Read error: 54 (Connection reset by peer))
mhoran has joined #arpnetworks
up_the_irons has joined #arpnetworks
ChanServ sets mode: +o up_the_irons

[20:58]

................. (idle for 1h20mn)

nuke^ is now known as nuke

[22:19]

jeev

damn
slack has been isntalling through qemu without kvm for 4 hours, still going

[22:31]

ballen

me thinks you should use something else

[22:32]

.... (idle for 16mn)

jeev

yea qemu bows
blows
didnt you suggest vmware

[22:48]

ballen

yes

[22:48]

jeev

it will work well without kvm ?
this is for a server with no kvm
xen spanks qemu without kvm

[22:48]

ballen

like a literal kvm?
or the software

[22:49]

jeev

vmx or whatever amd's capability
you iknow, the cpu capability
that makes your vm run sexy
you know what kvm is ?

[22:49]

ballen

oh, yes you must have virtuization extensions for ESXi

jeev

www.linux-kvm.org
oh

ballen

yes I know what KVM

jeev

damn

ballen

jeev

so im stuck with xen ;)
my kvm capable server is doing well

ballen

can grab a trial of regualar ESX
which will run

jeev

which is easier
virtualbox or esx
not easier but
less of a hassle

ballen

why the hell am I tired its only 1:52

[22:52]

jeev

heh
im tired too and it's only 1052

[22:52]

ballen

getting pissed at my head
I can usually stay up till 4-5 no prob when working on a fun project

[22:52]

jeev

last fun project i had was vagina

[22:53]

mike-burns

ballen: You could use Hubris and write the method in Haskell, for Ruby. (Instead of using RubyInline.) (Responding to something way in the backlog.)

[22:59]

ballen

whats Hubris

mike-burns

It's a Haskell-to-Ruby marshaller.

ballen

mike-burns

It's so you can write fast code for Ruby without writing C.