| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |
| Who | What | When |
|---|---|---|
| *** | EhtyarWRK has quit IRC (Quit: I was raided by the FBI and all I got to keep was this lousy quit message!) | [00:30] |
| ............. (idle for 1h2mn) | ||
| LT has joined #arpnetworks | [01:32] | |
| Ahmed has left | [01:37] | |
| .......... (idle for 49mn) | ||
| _id has quit IRC (Quit: changing servers) | [02:26] | |
| _id has joined #arpnetworks | [02:38] | |
| .......................................... (idle for 3h27mn) | ||
| DDevine has joined #arpnetworks | [06:05] | |
| .............. (idle for 1h6mn) | ||
| hien has joined #arpnetworks | [07:11] | |
| baklava has quit IRC (Disconnected by services)
baklava- has joined #arpnetworks | [07:17] | |
| hien has quit IRC (Quit: leaving) | [07:22] | |
| ...... (idle for 29mn) | ||
| diamondshakes has joined #arpnetworks | [07:51] | |
| diamondshakes | is there an ongoing network outage again -.- | [07:52] |
| *** | reardencode has quit IRC (Ping timeout: 260 seconds)
[FBI] has quit IRC (Ping timeout: 276 seconds) [FBI] starts logging #arpnetworks at Thu Aug 04 07:53:07 2011 [FBI] has joined #arpnetworks pjs has quit IRC (Ping timeout: 240 seconds) | [07:52] |
| diamondshakes | diamondshakes pokes up_the_irons
same symptoms as yesterday at least | [07:53] |
| *** | koan has joined #arpnetworks
mjp has quit IRC (Ping timeout: 246 seconds) kennyz has joined #arpnetworks milki has joined #arpnetworks | [07:53] |
| diamondshakes | hm
they are back? | [07:54] |
| *** | R496 has joined #arpnetworks
mjp has joined #arpnetworks bitslip has joined #arpnetworks Wraithan has joined #arpnetworks pjs has joined #arpnetworks pjs is now known as Guest45192 | [07:54] |
| Wraithan | What's up with the internet connection down there the last couple days
twice now I've lost my connection in the morning | [07:57] |
| *** | reardencode has joined #arpnetworks | [07:57] |
| diamondshakes | same here | [07:58] |
| *** | mick_laptop has joined #arpnetworks | [07:58] |
| diamondshakes | as you can see from all those ping timeouts as well
no updates on twitter | [07:58] |
| Wraithan | Not acceptable. | [07:59] |
| *** | Guest45192 has quit IRC (Ping timeout: 264 seconds) | [08:01] |
| Wraithan | My monitoring software shows it was down yesterday at 6:55 until 7:10, then it was spotty for about 15 minutes. mind you the resolution on those times kinda sucks, I think it is 3 or so minutes, but still | [08:02] |
| ....... (idle for 30mn) | ||
| jpalmer | my monitoring software doesn't show an outage yesterday. Are you sure it's not a routing issue between you and them? | [08:32] |
| LT | mine did too... I'm sure quite a few other people here saw it as well | [08:34] |
| RandalSchwartz | I too had a problem yesterday at that time
so whatever is being monitored isn't capturing the user experience fully my nagios couldn't ping *itself*. that's insane when maayan reports "maayan is unpingable"... maayan also couldn't reach chris, another machine on a different kvm | [08:37] |
| jpalmer | hrm, are you two on the same host, by chance?
maybe it's something specific to that host? if a host can't ping itself.. thats a bit extreme. are you monitoring the external IP? and internal IP? or loopback? and this a common thing? i'd be really interested in knowing if internal pings fail, and if loopback pings fail.. if it's a re-occuring thing. | [08:44] |
| LT | I think I'm on 13 | [08:48] |
| *** | heavysixer has quit IRC (Remote host closed the connection) | [08:49] |
| jpalmer | RandalSchwartz: another question. does your 'uptime' change? I'm wondering if your seeing a hardware watchdog/restart thing? maybe watchdog reset is failing, and the machine is restarting even though the hosts are responsive
(I've actually seen that before. and as the host is restarting, the networking isn't available at the host level, even though the host is running (in the process of shutting down) err, starting up rather. | [08:50] |
| LT | hmm, actually I'm on kvr08 - my memory is not would it should be | [09:02] |
| .... (idle for 17mn) | ||
| *** | LT has quit IRC (Quit: Leaving) | [09:19] |
| DDevine has quit IRC (Ping timeout: 264 seconds) | [09:29] | |
| ..... (idle for 22mn) | ||
| mike-burns | I'm on kvr02 also also saw some Internet "pausing" earlier.
I waited it out and it came back within 10 minutes. But I don't really do anything but email and IRC from this server. | [09:51] |
| Wraithan | We run our ticket system on a VPS there, not sure what hardware that one is on, my personal VPS is on kvr09 and I lost connection | [09:55] |
| *** | HighJinx has quit IRC (Ping timeout: 240 seconds) | [09:55] |
| Wraithan | jpalmer: server wasn't restarted, I would have had a notification for that, also I would have really felt it on my vps as I have some things just running in tmux not auto started (since I am deving on them) | [09:56] |
| diamondshakes | no uptime restart
10:08AM up 47 days, 16:09, 5 users, load averages: 0.00, 0.00, 0.00 some irc connections dont ping out screen sessions still alive | [10:04] |
| RandalSchwartz | I saw the same thing yesterday from 7:50 to 8:15
my internet went away, but the box was otherwise unaffected. as in "went away", I mean I couldn't even ping my own address. it was very weird. | [10:05] |
| diamondshakes | yup | [10:06] |
| RandalSchwartz | Hmm. I thought you could change the CD in the tray via the console already | [10:08] |
| up_the_irons | kvr12 had some load issues for a bit (yesterday) | [10:08] |
| RandalSchwartz | I guess that part isn't automated. | [10:08] |
| up_the_irons | i just reset a bgp session now (packetexchange peer disappeared, wth?) | [10:08] |
| diamondshakes | :o
up_the_irons++ | [10:10] |
| up_the_irons | diamondshakes++ | [10:11] |
| jpalmer | RandalSchwartz: "own address" meaning external IP? internal IP? loopback ip? | [10:11] |
| RandalSchwartz | my public IP | [10:11] |
| diamondshakes | jpalmer: www.arpnetworks.com was unreachable during the time as well | [10:11] |
| RandalSchwartz | that's what I'm testing in nagios
maayan couldn't ping its own public IP nor could it reach chris (.insightcruises.com) | [10:12] |
| jpalmer | RandalSchwartz: any chance you could set your monitoring to ping a local IP? as well as loopback? I wonder if it's a deeper issue. | [10:12] |
| *** | pjs has joined #arpnetworks | [10:12] |
| diamondshakes | jpalmer: well, maybe up_the_irons's reset might help things? | [10:13] |
| jpalmer | diamondshakes: if he can't ping from his box TO his box, I doubt it's a BGP peering issue.
diamondshakes: he's saying his nagios box can't ping itself. | [10:13] |
| diamondshakes | o | [10:14] |
| RandalSchwartz | yeah - I've seen this before too. | [10:14] |
| up_the_irons | yeah that's not a bgp issue | [10:14] |
| RandalSchwartz | about once every six weeks
lasts about 10 minutes basically, net connections die, and the box can't ping itself and as we see, it's not just my boxes. seems to be rather systemic | [10:14] |
| jpalmer | it's odd though. because if you're pinging the same IP (from that box) the traffic shouldn't even be hitting the switch. so it's not an ARP table corruption issue or something.. | [10:16] |
| RandalSchwartz | but it still has to go through the VM's drivers
so maybe there's a VM problem as in, I'm pinging the "emulated" em0 | [10:16] |
| jpalmer | thats kinda the direction I'm headed. | [10:16] |
| RandalSchwartz | you're right in that a good test to add would be to ping the lo0 interface too
and see if they both die at the same time I'll get on that | [10:17] |
| jpalmer | I think you should add 2 more "hosts" to your nagios. bring up an RFC1918 address (as an alias on em0) and also monitor the loopback address as a seperate host. this will tell you if it's specific to "em0" or something else. | [10:18] |
| RandalSchwartz | ooh. yeah
more things to add for this weekend's "hours billable to Neil" weekend :) | [10:18] |
| jpalmer | I wonder if there is a newer version of the virtualization osftware than ARP is using. might be worth investigating the Changelog from that version on. | [10:20] |
| toddf | you don't have to do rfc1918 | [10:20] |
| RandalSchwartz | ok - in transit... back soon. | [10:20] |
| toddf | just ping the link local v6 address | [10:20] |
| jpalmer | toddf: good call. also, it's probably worth finding out if it's isolated to ipv6, or ipv4. | [10:21] |
| *** | heavysixer has joined #arpnetworks
ChanServ sets mode: +o heavysixer | [10:21] |
| toddf | given the description, I'd lend towards os specific handling of the emulated nic. would be interesting if any openbsd people had the same issue, or other os's .. I suspect as has been stated it is an 'emulated nic stops responding properly' maybe interrupts stop happening properly or something .. for 10mins .. then resumes again | [10:29] |
| *** | diamondshakes has left
HighJinx has joined #arpnetworks | [10:31] |
| Wraithan | toddf: completely different versions of the linux kernel having this problem? Seems unlikely.
especially given that we use kvm/qemu at work and I've never seen that issue on our own hardware. | [10:34] |
| toddf | Wraithan: linux? I thought guests were generally *BSD
I'm talking guest os support for the emulated nics here | [10:37] |
| jpalmer | toddf: thats a good call. all my VM's are centos, and I haven't noticed my monitoring complaining. | [10:38] |
| toddf | I guess from context you use loonix inside as the guest os. interesting to know. more info is useful until it is determined what is exactly happening. (I suspect one vps going bezerk for a bit then settling down, but thats just a wild stab in the dark) | [10:38] |
| Wraithan | toddf: The two VPSs on ARP that I have are linux, one Ubuntu 10.10 the other Arch Linux | [10:38] |
| jpalmer | I have a total of 3 VPS's, all are centos. (2 5.x, 1 6.x) and I haven't seen this issue. | [10:41] |
| Wraithan | toddf: and it has been 2 different hardware machines, since LT said he is on 09 and I am on 08 | [10:41] |
| gamarco | murphy | [10:42] |
| toddf | bezerk can be something that effects networking in general since vlans can (and are for some clients) available on multiple host systems | [10:44] |
| ...... (idle for 28mn) | ||
| *** | heavysixer has quit IRC (Remote host closed the connection) | [11:12] |
| .... (idle for 15mn) | ||
| RandalSchwartz | got to promote arp on the freebsd mailing list a moment ago | [11:27] |
| toddf | awesomeness | [11:28] |
| milki | oo, i see the email
13 minutes ago freebsd-questions@ and its already archived in the web http://tinyurl.com/3gdwzdm :P | [11:30] |
| toddf | http://v.gd/arp_plug comeon, create one with some style?
;-) | [11:33] |
| milki | >.>
it doesnt go direct >.> | [11:33] |
| toddf | though I will take exception, last I checked, elastic hosts is not what I would consider a similar pricepoint. | [11:34] |
| milki | oo, qr code too | [11:34] |
| toddf | its in the 'cheaper than ec2' bucket perhaps but .. I've yet to find anything similar to ARP until it gets to be a very large machine and then why not just host real hardware at that point? ;-)
(I look when people try to convince me otherwise, and always walk away knowing I've got the best deal around) | [11:35] |
| gamarco | maybe i am like 3 years behind, but is "cloud" another word for "virtual server" ? | [11:36] |
| RandalSchwartz | it wasn't "triple the price of ARP" like the other vendors I found | [11:36] |
| milki | no | [11:36] |
| gamarco | and people are just selling the same old thing under a new name? | [11:36] |
| RandalSchwartz | so it made it closer to "similar" | [11:36] |
| milki | but you can treat it as such | [11:37] |
| RandalSchwartz | similar but more :) | [11:37] |
| milki | tons more | [11:37] |
| RandalSchwartz | and no ipv6 yet | [11:37] |
| milki | haha indeed | [11:37] |
| gamarco | cheapest one i can find there is $4/mo
*$44 | [11:38] |
| .... (idle for 18mn) | ||
| RandalSchwartz | yeah - but once you start pricing the $75-ish and up VPS, it gets closer | [11:56] |
| toddf | randalschwartz: ah, I hadn't remembered that, yeah the size of vps's you are using makes them similar indeed. I'm looking at the low end ;-) | [12:08] |
| ............... (idle for 1h11mn) | ||
| jpalmer | gamarco: "cloud" is a useless marketting term, that has no real meaning or value. ask 100 people what "cloud" is, and you'll get 85 different answers.
it's a vps, no it's a colocated server, no, it's a web app. no, it's virtualization on the local lan, no it's.. all bullshit. the only 'concrete' definition for cloud is "it's a real, or virtual thing, where you can host your data or application, either on the internet, or locally" which essentially means.. every server, website, application, or database is a compnent of this nebulous 'cloud' idea. jpalmer keeps meaning to write a blog article about the uselessness of that term, and post it.. in the cloud. | [13:19] |
| vcs | whenever i heard cloud computing
i think of peoples heads up in the clouds being blinded by them cluelessly saying the world clouds over and over | [13:24] |
| jpalmer | and only apple could take such a useless term, and make it not only useless, but redundant by calling it 'icloud!' | [13:26] |
| vcs | lol
what have those clouders been smoking | [13:26] |
| jpalmer | (AND successfully SELL it, no less.) | [13:27] |
| vcs | i want some | [13:27] |
| RandalSchwartz | o/~ I've looked at clouds from both sides now... o/~ | [13:32] |
| vcs | Its no shocker that magazines such as PC world
are pushing the term cloud just another buzzword | [13:32] |
| jpalmer | HAH! I just quick polled the office. asked 18 people. guess what? no TWO people answered "when I say something about using 'the cloud' what exactly am I referring to' no two people said the same answer. | [13:33] |
| vcs | lol
its because its so rediculously vague | [13:33] |
| RandalSchwartz | yes - everyone has their own individual cloud! that's the beauty!
My Personal Cloud Jean Cloud van Damme | [13:33] |
| vcs | I hope my cloud does not rain away | [13:34] |
| .... (idle for 18mn) | ||
| jpalmer | ok, finally.. 2 similar answers (I'm at 33 people polled now) and similar in a "white and balck are similar, in that they are both colors" kinda way.
(one said "internet" one said "anything on a network, or not on your local machine") | [13:52] |
| mike-burns | That's about as vague as possible; seems like the right answer. | [13:55] |
| *** | wallshot has joined #arpnetworks | [13:55] |
| jpalmer | hehe | [13:55] |
| *** | heavysixer has joined #arpnetworks
ChanServ sets mode: +o heavysixer | [13:56] |
| jpalmer | by the second definition.. if bobby and sue are in an office, and bobby saves a word doc to his own machine.. to bobby, it's not a cloud document. but to sue, it is! | [13:56] |
| wallshot | love that cloud
it's got everything in it | [14:05] |
| *** | heavysixer has quit IRC (Remote host closed the connection) | [14:19] |
| ..... (idle for 22mn) | ||
| nomadlogic has joined #arpnetworks | [14:41] | |
| nomadlogic | i wanted to say thanks to the folks at arpnetworks!
just setup a new vps and was really happy to see ipv6 configured by default! it was also trivial to reinstall my node to freebsd-8.2, so thanks guys! | [14:54] |
| wallshot | yeah the ipv6 is nice | [14:55] |
| nomadlogic | gonna spend tonight migrating my rootbsd server over to my new arpnetwoks node
:) | [14:56] |
| wallshot | that's always good fun
tho i guess almost anything with FreeBSD is good fun | [14:56] |
| nomadlogic | lol - good point | [14:57] |
| RandalSchwartz | nomadlogic welcome to Arp
are you using zfs as / on your node? if not, may I strongly suggest it? :) | [15:10] |
| nomadlogic | thanks! | [15:12] |
| RandalSchwartz | there are wiki entries about how to do it here
or I can give you the URL I have bookmarked | [15:12] |
| nomadlogic | my plan was to purchase more space from you all (or get a second instance) and run zfs on there | [15:13] |
| wallshot | i'm gonna need to buy more space probably at some point :( | [15:13] |
| RandalSchwartz | zfs as root is easily extended though
echo "please increase my disk to 120GB" | mail support@arpnetworks.com (time passes) | [15:13] |
| wallshot | heh | [15:14] |
| RandalSchwartz | boot from DVD, type "gpart recover" "gpart resize blah"
and reboot | [15:14] |
| vcs | can you do that live
or no | [15:14] |
| RandalSchwartz | and now you have more ZFS disk
I don't think so or not sure with ZFS | [15:14] |
| wallshot | i have boring UFS / =/ | [15:14] |
| nomadlogic | i was initially going to run 9.0-CURRENT on this guy for some testing, but there is a bug with the installer that ships with the 9.0-BETA1 iso :( | [15:15] |
| RandalSchwartz | I tend to avoid bleeding edge
in fact I stay off a release for at least a few weeks just call me conservative :) | [15:15] |
| nomadlogic | heh - yea, for prod systems no doubt :)
i was hacking on the freebsd amazon port that colin percival is doing for a while and was hoping to do more testing on my own guess i'm not a good tester though since i didn't report the installer bug i ran into :p | [15:16] |
| RandalSchwartz | heh - I interviewed him about that yesterday
twit.tv/floss176 | [15:17] |
| nomadlogic | nice! | [15:17] |
| RandalSchwartz | "it's what I do" | [15:17] |
| ........ (idle for 35mn) | ||
| gamarco | jpalmer: i like the way you think | [15:52] |
| wallshot | is there a place to check how much bandwidth I've used so far this month? | [16:06] |
| .... (idle for 17mn) | ||
| RandalSchwartz | cricket graphs, from your dashboard | [16:23] |
| wallshot | thanks! | [16:23] |
| *** | EhtyarWRK has joined #arpnetworks | [16:29] |
| bob^^ has quit IRC (Read error: Connection reset by peer)
bob^^ has joined #arpnetworks Yamazaki-kun has quit IRC (Ping timeout: 260 seconds) Yamazaki-kun has joined #arpnetworks plett has quit IRC (Ping timeout: 264 seconds) plett has joined #arpnetworks | [16:38] | |
| ...... (idle for 25mn) | ||
| nomadlogic has quit IRC (Ping timeout: 246 seconds) | [17:05] | |
| ......... (idle for 44mn) | ||
| heavysixer has joined #arpnetworks
ChanServ sets mode: +o heavysixer | [17:49] | |
| gamarco | RandalSchwartz: you could have easily gotten $25 consult fee for that | [17:58] |
| wallshot | for the interview, or for pointing me at bandwidth graph?
:) | [18:00] |
| gamarco | graph. didn't see the interview. $50 easy for that =) | [18:00] |
| wallshot | yeah i mighta gone another month or 3 before actually asking support@
but irc helps me fight my own laziness | [18:01] |
| .... (idle for 16mn) | ||
| *** | wallshot has quit IRC (Remote host closed the connection) | [18:17] |
| ....... (idle for 33mn) | ||
| HighJinx has quit IRC (Ping timeout: 252 seconds) | [18:50] | |
| ............ (idle for 55mn) | ||
| Wraithan | A number of your customers complain about inability to access their machines, I have data, others have data, showing it happened yesterday as well. Please, please, tell me what to call that, other than down.
Don't be dicks and try to hide when bad shit happens to multiple customers, address it, even if you are still investigating just say so, don't call me out as a liar instead. And if you aren't investigating it, say so, so I can take mine, my company, and several friends to some other service who cares. | [19:45] |
| *** | tigerpaw- has left "rawr?"
tigerpaw has joined #arpnetworks | [19:48] |
| G | Wraithan: have you created a support ticket? | [19:50] |
| Wraithan | G: I had assumed that due to mutliple @'d people in this channel replying to the multiple people in this channel reporting problems that it was being taken care of | [19:51] |
| G | Wraithan: as the topic says, all bar one @ are long-time customers, and generally IRC isn't really an official communication method | [19:52] |
| Wraithan | As this is a channel with a single # that means it is official, unless there is a violation of the freenode rules. Under that assumption, reporting something here and having an @ say something in return, espcially one like up_the_irons who I know for a fact works for the company. | [19:52] |
| up_the_irons | I only had one report of an issue and my monitoring did not alert me to any issues either; I reported this morning that I bounced a BGP session as well | [19:54] |
| Wraithan | up_the_irons: should I assume you don't read anythign when you are typing in here
as the problem this mornign was reported right before you said you rebooted that | [19:54] |
| up_the_irons | So, apparently, the "outage" was enough to have people talk about it in here, but not enough to email support. That doesn't sound like much of an outage. | [19:55] |
| G | up_the_irons: well the day before the network really went nuts | [19:55] |
| Wraithan | up_the_irons: If I talk about it in here, when you are around, I'd assume you are paying attention.
Should I not assume that. ? | [19:55] |
| up_the_irons | yeah, this morning i saw some people talk about an issue, so I checked my bandwidth graphs, noticed a session down. So I bounced it. I do not monitor IRC 24/7, but I check the support desk quite often
Wraithan: do not assume that at all, I run IRC in a screen session. I'm *always* logged in, but don't assume I'm actually at the computer. | [19:56] |
| Wraithan | up_the_irons: I assumed because you spoke during the convoration
not because you were online I am online all the time too. similar setup. | [19:57] |
| *** | mike-burns has quit IRC (Read error: Operation timed out)
mhoran has quit IRC (Read error: Operation timed out) | [19:57] |
| up_the_irons | if I speak, then I'm here | [19:57] |
| Wraithan | fuck it, not worth the energy
I'll file a ticket | [19:57] |
| up_the_irons | i may not necessarily read all the scrollback, but i usually do | [19:57] |
| *** | Wraithan has left "WeeChat 0.3.6-dev" | [19:57] |
| G | eek what the... cloudkick deleted my node from it's monitoring | [19:58] |
| up_the_irons | ouch | [19:58] |
| *** | mhoran has joined #arpnetworks
ChanServ sets mode: +o mhoran mike-burns has joined #arpnetworks ChanServ sets mode: +o mike-burns | [19:58] |
| G | I have no idea either | [19:58] |
| *** | Wraithan has joined #arpnetworks | [19:58] |
| up_the_irons | Wraithan: for the record, I don't hide when bad things happen. I was *not* investigating anything either, since I had no reports of problems and my monitoring did not page me (this is a weakness on my part for not having geographically disperse nagios setups). So if this makes you uncomfortable, I'm more then happy to admit it now so you can find a provider that suits you better. I value
honestly and transparency; I don't want to hold anyone here if they don't want to be here. I was moving into a new house all day yesterday, so Iwasn't online at all except for checking emails (e.g. support) from my android and with that, I must leave the office cd $home | [20:05] |
| *** | HighJinx has joined #arpnetworks | [20:18] |
| ............... (idle for 1h13mn) | ||
| tuv | tuv just discovered lovevps.com. 512MB xen vm for $7 | [21:31] |
| ........ (idle for 38mn) | ||
| *** | DDevine has joined #arpnetworks | [22:09] |
| .... (idle for 16mn) | ||
| RandalSchwartz | hey up_the_irons
at some point, we shall meet :) I'm in pasadena this weekend Wraithan - up_the_irons is pretty transparent, but ARP may not be the right place to host you. lots of nines, but maybe not enough. :) it's lots of cheap nines, but if you want more, you'll pay tuv - remember that xen = paravirtualized | [22:25] |
| tuv | RandalSchwartz: hmm.. didn't know that. what's (or where can i learn about) the difference? | [22:39] |
| RandalSchwartz | try http://lmgtfy.com/?q=xen | [22:40] |
| DDevine | RandalSchwartz: xen != always paravirtualised.
Xen can do full virtualisation as well. HVM. | [22:41] |
| RandalSchwartz | not originally | [22:42] |
| DDevine | Yeah | [22:42] |
| .... (idle for 16mn) | ||
| RandalSchwartz: IMO 9's are practically worthless. The simple fact is that you have to architect your own availability not your hoster.
Not even Amazon is bulletproof - whole regions *have* gone unavailable before. recently. They basically DDOSed themselves. Not DDOS, just DOS. | [22:58] | |
| toddf | too bad chunkhost is so persnickety with their setup, supposedly full HVM, but 'one partition filesystem' doesn't convince me. | [23:04] |
| ..... (idle for 22mn) | ||
| tuv | in any case, $7 for a 512MB xen vm is pretty damn cheap, and googling lovevps didn't yield any complaints. though they are relativelly new | [23:26] |
| DDevine | Interesting.
They could be overcommitting. | [23:27] |
| tuv | who knows. but they do have 'NO OVERSELLING' on their home page header | [23:28] |
| DDevine | heh yeah I just noticed that. | [23:29] |
| tuv | they seem to be reselling from few other services: burst.net, hostdime, ...
(they have an openvz option) | [23:30] |
| ..... (idle for 21mn) | ||
| up_the_irons | RandalSchwartz: ah cool :)
heh, 'NO OVERSELLING'. 1) don't believe everything you read, 2) "overselling" is relative ;) | [23:51] |
| ↑back Search ←Prev date Next date→ Show only urls | (Click on time to select a line by its url) |