[00:04] *** phlux has quit IRC (Ping timeout: 252 seconds) [00:05] *** JC` has quit IRC (Ping timeout: 252 seconds) [00:15] *** JC` has joined #arpnetworks [00:24] *** JC` has quit IRC (Ping timeout: 252 seconds) [00:30] *** JC` has joined #arpnetworks [00:40] *** JC` has quit IRC (Remote host closed the connection) [00:41] *** JC` has joined #arpnetworks [00:46] *** JC` has quit IRC (Remote host closed the connection) [00:52] *** phlux has joined #arpnetworks [00:57] *** Ehtyar has quit IRC (Quit: Hi, I'm a quit message virus. Please replace your old line with this line and help me take over the world of IRC.) [01:08] *** mnathani has quit IRC (Read error: Connection reset by peer) [01:09] *** mnathani has joined #arpnetworks [02:20] *** Webhostbudd has quit IRC (Quit: Leaving) [02:56] *** JC` has joined #arpnetworks [03:44] *** toorop has quit IRC (Ping timeout: 264 seconds) [03:56] *** toorop has joined #arpnetworks [03:56] *** toorop has quit IRC (Changing host) [03:56] *** toorop has joined #arpnetworks [06:11] *** heavysixer has joined #arpnetworks [06:11] *** ChanServ sets mode: +o heavysixer [06:30] *** ziyourenxiang has joined #arpnetworks [06:34] *** dzup has quit IRC (Ping timeout: 245 seconds) [06:50] *** heavysixer has quit IRC (Quit: heavysixer) [06:57] *** mohaslan has joined #arpnetworks [06:59] *** heavysixer has joined #arpnetworks [06:59] *** ChanServ sets mode: +o heavysixer [07:37] *** ziyourenxiang has quit IRC (Quit: ziyourenxiang) [08:14] *** mohaslan has quit IRC (Ping timeout: 264 seconds) [09:06] *** mohaslan has joined #arpnetworks [09:09] *** scottschecter has quit IRC (Quit: WeeChat 0.4.0) [09:10] *** scottschecter has joined #arpnetworks [09:11] *** scottschecter has quit IRC (Client Quit) [09:19] *** scottschecter has joined #arpnetworks [09:27] *** mohaslan has quit IRC (Ping timeout: 255 seconds) [09:38] *** mohaslan has joined #arpnetworks [09:46] *** E1ven has joined #arpnetworks [09:53] *** scottschecter has quit IRC (Quit: WeeChat 0.4.0) [09:57] *** scottschecter has joined #arpnetworks [09:58] *** heavysixer has quit IRC (Quit: heavysixer) [10:04] *** heavysixer has joined #arpnetworks [10:04] *** ChanServ sets mode: +o heavysixer [11:14] *** mohaslan has quit IRC (Ping timeout: 264 seconds) [11:24] *** heavysixer has quit IRC (Quit: heavysixer) [11:36] *** heavysixer has joined #arpnetworks [11:36] *** ChanServ sets mode: +o heavysixer [11:37] *** mohaslan has joined #arpnetworks [12:02] *** heavysixer has quit IRC (Quit: heavysixer) [12:11] *** heavysixer has joined #arpnetworks [12:11] *** ChanServ sets mode: +o heavysixer [12:16] *** alexstanford16 has quit IRC (Quit: alexstanford16) [12:49] *** toddf has quit IRC (Ping timeout: 264 seconds) [12:54] *** toddf has joined #arpnetworks [12:54] *** ChanServ sets mode: +o toddf [13:18] *** toorop has quit IRC (Ping timeout: 252 seconds) [13:21] *** scottschecter has quit IRC (Quit: WeeChat 0.4.0) [13:23] *** mohaslan has quit IRC (Read error: Operation timed out) [13:35] *** E1ven has quit IRC (Quit: Later.) [13:36] *** scottschecter has joined #arpnetworks [13:48] *** nesta has quit IRC (Remote host closed the connection) [13:50] *** CaZe has quit IRC (Ping timeout: 248 seconds) [13:51] *** heavysixer has quit IRC (Quit: heavysixer) [13:51] *** nesta has joined #arpnetworks [13:59] *** CaZe has joined #arpnetworks [14:00] Anything weird going on with the network? [14:02] not that i can see [14:03] Hmm. Getting some packet loss. [14:03] CaZe: can you get a mtr? [14:04] *** mohaslan has joined #arpnetworks [14:04] mtr? [14:04] linux command [14:05] I don't have Linux. [14:07] should be som eqvivalent on bsd i guess [14:08] 100 packets transmitted, 83 packets received, 17.0% packet loss [14:08] *** toorop has joined #arpnetworks [14:08] CaZe: FreeBSD? [14:08] OpenBSD [14:09] Well I'll bbl. [14:09] Maybe it'll have fixed itself by then. [14:09] mtr is in OpenBSD ports [14:09] mtr is in ports [14:09] my openbsd works fine :D [14:09] should upgrade :D [14:10] also apparently there is a fiber cut in the SF Bay area [14:19] * mikeputnam plugs arp on http://milwaukeemakerspace.org/ [14:20] CaZe: # pkg_add mtr [14:20] Well maybe when I upgrade the snapshot. [14:22] I think my VM crashed. [14:22] I can ping it, but I see nothing on serial. [14:23] caze: did you configure serial output? [14:23] Yes. [14:23] Heh. [14:23] control-ecl0 will send a reset incase you had ddb.console enabled [14:23] I saw a prompt on VNC. [14:24] I typed in my login, but the password prompt doesn't come up. [14:25] CaZe: where are you seeing packet loss from? (Location) [14:25] Boston [14:25] The packet loss actually just went away. [14:26] Weird, my machine's still responding to pings, but I can't get it to respond on serial or VNC. [14:27] I guess I'll just try a hard reboot. [14:28] Well, I still have 2% packet loss. [14:30] boston links migt be full due to yesterdays bombing [14:31] *** CaZe` has joined #arpnetworks [14:31] *** CaZe has quit IRC (Quit: Page closed) [14:31] *** CaZe` is now known as CaZe [14:32] Well I think it's fixed. [14:36] I wonder if the packet loss I was experiencing was OpenBSD slowly dying. [14:37] Though it had gotten a lot better right before I rebooted it. [14:38] Down to 0% now, though there were a few pings in the 1000ms range. [14:39] maybe it was related to that cable cut thing [14:39] Maybe, how long ago was that? [14:40] no idea [14:40] hmm looks like 4 hours ago [14:50] *** nesta has quit IRC (Read error: Operation timed out) [15:01] caze: I've misdiagnosed my vps having issues vs he.net having issues getting me to my vps [15:09] *** nesta has joined #arpnetworks [15:14] *** nesta has quit IRC (Quit: leaving) [15:14] *** nesta has joined #arpnetworks [15:32] weird my vm is going slow [15:32] it never goes slow [15:33] oh [15:33] it's just mirrors.arpnetworks.com [15:33] ipv4 doesn't seem to respond to ping ipv6 does [15:34] :o [15:34] well by being slow [15:34] i meant pkg_add was taking forever [15:34] can other people get to mirrors.arpnetworks.com on ipv4? [15:35] I see it on v4 [15:35] ok [15:36] why can't i [15:36] 208.79.88.133... [15:36] do you get that ip? [15:36] yes [15:36] # telnet mirrors.arpnetworks.com 80 [15:36] Trying 208.79.88.133... [15:36] and you can connect on port 80? [15:36] my browser works, if that's what you mean [15:36] # telnet -6 mirrors.arpnetworks.com 80 [15:37] Trying 2607:f2f8:0:101::2... [15:37] Connected to mirrors.arpnetworks.com. [15:37] from arp i meant [15:37] oh [15:37] oh, I'm doing it from outside [15:37] i can trace to mirrors.arpnetworks.com from home [15:37] mercutio: yep can't reach v4 from my arp node [15:37] i wonder what changed [15:38] up_the_irons: are you around? [15:38] i wonder if there's a way to make pkg_add use ipv6 [15:38] i think it's timing out after forever [15:38] and falling back, but the ipv4 isn't doign connection resfused or anything to it move on quickly [15:42] *** pjs has quit IRC (Read error: Connection reset by peer) [15:42] *** pjs_ has joined #arpnetworks [15:49] *** pjs_ is now known as pjs [15:55] all: there was a fiber cut between LAX and SJC earlier, causing some Any2 peers to bounce [15:55] "UPDATE: Field technicians have confirmed that the cable was cut in the existing slack coil in the manhole. Field Operations are prepping the cables at this time while awaiting the arrival of the splice crews." [15:55] holy crap: [15:56] "UPDATE: The fiber cut was located in San Jose, CA and the technicians have completed prepping the fibers. The two splice crews are onsite at this time and have begun splicing the first fibers. The Transport NOC [15:56] +is monitoring alarms as systems should begin to clear. Field Services reports it was a full cut of a 132 count cable. They estimate 2 - 2.5 hours to complete all splicing and for all services to restore." [15:56] 132 fibers! [15:56] fun [15:56] not as much fiber as SUPER colon blow [15:56] this now appears fixed though: [15:56] "UPDATE: Our Transport provider has completed splicing and we have verified services are back up. Outage is now complete this will conclude our updates for this issue." [16:04] seperate paths are good [16:05] up_the_irons: can you look into why mirrors.arpnetworks.com is down on ipv4 from arp vm's? [16:05] but up from internet at large, and up on ipv6 [16:19] *** mohaslan has quit IRC (Ping timeout: 260 seconds) [16:20] *** heavysixer has joined #arpnetworks [16:20] *** ChanServ sets mode: +o heavysixer [16:31] tonight - laundry night. exciting [16:36] *** heavysixer has quit IRC (Quit: heavysixer) [16:47] *** Ehtyar has joined #arpnetworks [16:51] *** heavysixer has joined #arpnetworks [16:51] *** ChanServ sets mode: +o heavysixer [17:02] *** heavysixer has quit IRC (Quit: heavysixer) [17:04] *** [Derek] has quit IRC (Ping timeout: 246 seconds) [17:05] *** heavysixer has joined #arpnetworks [17:05] *** ChanServ sets mode: +o heavysixer [17:05] *** [Derek] has joined #arpnetworks [17:05] mercutio: strange, i can hit it find from our VMs [17:06] well it didn't work me and static [17:06] at a minimum [17:06] with ipv4 [17:06] it doesn't make it past gateway [17:07] still not working? [17:07] dies at - 208.79.93.113 [17:07] RandalSchwartz: did you try it from inside? [17:07] static: is that your gateway? [17:07] actually - not for me inside either [17:07] ie is your vps ip one higher than that [17:07] v6 no problem [17:07] v4 hurting [17:07] mercutio: yep its my gateway [17:08] mtr shows it not making it off the first hop [17:08] RandalSchwartz: yeh same as us [17:09] I wonder how many other servers I can't talk to [17:09] *** Webhostbudd has joined #arpnetworks [17:17] *** pib2009 has quit IRC (Remote host closed the connection) [17:19] *** dzup has joined #arpnetworks [17:32] *** heavysixer has quit IRC (Quit: heavysixer) [17:37] pkg_add -uvr 0.00s user 0.00s system 0% cpu 3:09:59.82 total [17:37] dunno why user/system time showing as 0 [17:37] but over 3 hours to run pkg_add -uvr [17:46] *** heavysixer has joined #arpnetworks [17:46] *** ChanServ sets mode: +o heavysixer [17:54] *** mohaslan has joined #arpnetworks [18:07] *** heavysixer has quit IRC (Quit: heavysixer) [18:32] NTT just went live [18:32] *** heavysixer has joined #arpnetworks [18:32] *** ChanServ sets mode: +o heavysixer [18:32] (yet i will probably not announce all our prefixes over it until further testing) [18:34] *** medum has quit IRC (Ping timeout: 257 seconds) [18:34] \o/ [18:52] * mikeputnam wets himself @ 17:55 < RandalSchwartz> not as much fiber as SUPER colon blo [18:52] that was SNL wasn't it? [18:54] yes [18:55] *** tabthorpe has quit IRC (Read error: Connection reset by peer) [18:56] *** koan has quit IRC (Read error: Connection reset by peer) [18:56] *** koan has joined #arpnetworks [18:56] *** Lucifer7 has quit IRC (Read error: Operation timed out) [18:57] *** lazard has quit IRC (Ping timeout: 252 seconds) [18:57] *** mikeputnam has quit IRC (Ping timeout: 245 seconds) [18:58] *** Lucifer7 has joined #arpnetworks [18:58] *** heavysixer has quit IRC (Quit: heavysixer) [18:59] *** medum has joined #arpnetworks [19:09] this is the 5th time pingdom is reporting my site as down while its not.... [19:09] who's watching the watcher over there [19:12] are there any known peering/network issues going on? [19:14] *** tabthorpe has joined #arpnetworks [19:15] there was, and maybe is [19:15] ssh: connect to host github.com port 22: Connection timed out [19:15] well, to most places I try to talk to. [19:24] http://hastebin.com/weherequnu.bash [19:45] *** heavysixer has joined #arpnetworks [19:45] *** ChanServ sets mode: +o heavysixer [19:51] oh that's nice, tinet route from los angeles to san jose to connect to ntt [19:51] *** brycec has joined #arpnetworks [19:53] mercutio: you also experience connectivity issues? [19:53] not until changed route [19:53] to go over ntt [19:53] took it back out [19:53] it was giving packet loss from tinet onwards [19:53] /and/ tinet was sending lax sj, sj, lax [19:53] connecting to ntt in sj [19:54] steadfast seems ok but i can't download my test file for some reason [19:54] Perhaps a stupid question, but how do i re-route outbound from my arpnetworks? I thought what was out of scope [19:54] outbound from my isp [19:54] I can't do shit from my host [19:54] ah [19:54] and short answer is you can't [19:54] thought what -> thought that. [19:54] haven't found outbound routes going over ntt yet [19:56] looks like no outbound routes over ntt yet [19:56] so yeah it's only partially up and only some subnets [19:57] which is probably a good thing if giving issues [19:57] that tinet issue maybe tinet issue though :/ [20:00] hey i am having connectivity issues [20:00] i think reverse path fiterling or the like [20:00] icmp works :/ [20:00] well stateful crap [20:01] up_the_irons: yuo around? [20:01] hopefully he knows about it [20:02] yeah it can't see return traffic [20:07] *** heavysixer has quit IRC (Quit: heavysixer) [20:08] lol even traffic to www.ntt.com routes over HE [20:08] silly up_the_irons [20:08] all outbound is not going over ntt [20:08] it's not completely silly [20:08] it's a complex issue to have it going over both outbounds atm [20:09] so i understand him doing incoming first [20:09] mercutio: I figured it would be logical for traffic to NTT to go over NTT [20:09] (v4 or v6) [20:09] brycec: his primary router can only take 100k routes [20:09] his ntt connection is on secondary router [20:09] ah [20:09] all the vm connectinos are terminating on the primary router [20:09] he currently has 96k routes [20:10] even all the close ntt routes is likely to be bigger than 4k [20:10] so what i imagine is likely is that further down the line he'll start shifting people to the new router [20:10] that can take full bgp table [20:10] and start shifting the upstreams over to that [20:11] but in the interim he's trying out the ntt with incoming [20:11] where it doesn't need to store huge route list [20:11] ah [20:12] gotcha [20:12] in the interim ntt giving isuses [20:18] mercutio: are you bored or what? [20:19] static: must be :) [20:19] it's lunchtime, it's wet, it's hot [20:19] and i can't even decide what order to eat my lunch in [20:20] since this issue is actually making large parts of my srevices to not work; should I escalate it somehow? I created a support ticket an hour ago [20:21] i kind of assume it's known at this point [20:21] :o [20:21] heh [20:21] well up_the_irons hasn't been talking in channel [20:21] he may be busy [20:21] he may be busy doing related things or unrelated things [20:22] but surely things like these must have monitoring abnormalities? [20:22] depends [20:22] icmp works [20:22] but tcp connections don't [20:22] jbergstroem: you sure its not your ISP? [20:22] big problem I think is the upstream [20:23] althought it's just weird [20:23] jbergstroem: your trace shows ntt in the path right? [20:23] staticsafe: tried from multiple locations, and more importantly - nothing going out from the host is working [20:23] ouch [20:23] nothing? [20:23] can you get to http://lwn.net on the host? [20:23] mercutio: icmp is fine. curl google.com times out, just as ssh [20:24] oh i see curl google doesn't work for me either [20:25] hrm what kvr are you guys on? [20:25] kvr15 [20:25] i think [20:25] I'm on 05 [20:25] 04 [20:25] im not seeing any issues on mine [20:25] what is your ip? [20:25] I can traceroute google [20:25] er /24 is good enough [20:25] ferrovax.asininetech.com [20:26] red.stonehenge.com [20:26] 174.136.101.154 [20:26] you're not being advertised static [20:26] 208.79.88.0/21 isn't being advetised [20:26] oh [20:27] *** mikeputnam has joined #arpnetworks [20:27] derp now i understand [20:27] 174.136.96.0/20 is [20:27] which is what i'm on, and jberg is on [20:27] how are you checking this? [20:27] route-server? [20:27] global bgp route table [20:28] Uh - red.stonehenge.com is reachable from the real world [20:28] or I wouldn't be talking to you [20:28] randal: it's only some routes down [20:28] the paths that come into ntt [20:28] oh [20:28] if you can't get to your vm and traceroute shows ntt you'll be screwed [20:28] but same applies for other hosts [20:28] and it's hard to see reverse path [20:29] like how do you get google to run a traceroute [20:31] i was about to try from my .au vps, but then I realized its Vocus for intl transit [20:31] My traceroute to my VPS shows ntt :D [20:31] so, 1h 30m in; how do we call on the gods? [20:32] Though leaving my VPS takes mzima. [20:32] caze can you ssh to your vps? [20:32] static: vocus had issues last night [20:32] it was routing via hong kong [20:32] return path via he.net [20:33] they're usually on any2ix direct [20:33] Yes, I'm on it now. [20:33] this was before i read about the fibre cuts [20:33] which is prob related to why it was scrwed [20:33] caze: oh intersting [20:33] url to fibre cuts? [20:33] can you reach other ports? [20:33] jber: it was in my email [20:33] jbergstroem: fibre cuts have been repaired [20:33] umm [20:34] yeah, just never read about it [20:34] http://www.santacruzsentinel.com/capitola/ci_23040335/at-t-customers-cut-off-by-outage [20:34] ports? This is over IPV4, btw. [20:34] take that for example [20:34] it was in san jose [20:34] thanks [20:34] oh it was talked about in this channel too [20:34] someone said 132 fibres were lost [20:35] http://permalink.gmane.org/gmane.org.operators.isotf.outages/5634 [20:35] that's the mailing list i saw it on btw [20:36] yea same [20:37] it was on NANOG too [20:38] that's a *lot* of fiber [20:39] ... http://www.dailymotion.com/video/x7ms6e_saturday-night-live-colon-blow_fun [20:39] static: otuages goes to my main mail box, nanog a seperate one [20:39] so yeah didn't see it on nanog [20:39] nanog has toom uch mail sometimes :/ [20:42] mercutio: im on too many mailing lists - http://okazaki.tomoyo.ca/AIi.png [20:43] somewhere along the road too many ml's somehow turns into a chase for that 0 unread for me :( [20:43] nanog has lots of crap [20:44] like that stuff about google tld or something [20:44] heh yea [20:44] did you see the drama on AUSNOG? [20:44] that's way more than me [20:45] you use gentoo-users, ubuntu-users, freebsd-users [20:45] what's the point of those/ :) [20:45] i'm not on ausnog [20:45] linux-kernel orly? :) [20:45] what happened? [20:45] i'm on linux-kerenel too [20:45] ubuntu-users is for entertainment, gentoo-users because i use gentoo, freebsd-users same thing [20:45] only so i can scan for things if i have weird problem or something [20:45] jbergstroem: its marked read by the filter [20:46] i keep it in case i have to look up something [20:46] static: just like me :) [20:46] the only lists i try to stay on par with nowadays is llvm{dev,-commits} and cfe-{dev,commits} [20:46] s/is/are [20:47] so looking at ausnog pipenetworks had big outage [20:49] gentoo-users gets fun every time udev and systemd gets brought up [20:49] :P [20:50] fedora-users was terrible when the last new release with the new installer came out [20:50] staticsafe: USE=mdev emerge busybox :) [20:51] *shrug* im not opposed to udev yet [20:51] I got introduced by $client to Monit the other day [20:51] as an alternative to nagios and/or zabbix and/or upstart [20:52] staticsafe: guess it depends on use case. just saying that it's hard to motivate for headless stuff. [20:52] yea good point [20:52] my only gentoo install is my desktop [20:52] * RandalSchwartz doesn't run *anything* linux [20:53] once you've gotten used to portage, you get a bit picky in what you expect from a package manager. at least one of my reaons for using it [gentoo] for so long. [20:53] I've been fanboying over FreeBSD lately [20:53] jbergstroem: indeed [20:54] portage is <3 [20:54] yeah - longtime user of fbsd [20:54] so. we just passed 2 hours. any suggestions on how to escalate? [20:55] pretty sure up_the_irons is doing whatever he can [20:55] escalation would be finding a way to get ec2 to host your site for a while, or something [20:56] well dns TTL gives me an hour or two more to play with [20:56] i guess i'd just like to be sure that people know there's a problem [20:56] long DNS TTLs-- [20:57] staticsafe: i don't optimise for exeptions. first time since i've moved to arp i've had any kind of issues [20:57] exceptions, sorry [20:59] jbergstroem: could be worse, 24hr TTL :P [20:59] i guess :) [21:01] after my exams are done I'm going to play with DNSSEC [21:08] *** lazard has joined #arpnetworks [21:26] I need more time to play with Dart [21:27] my clients both need it... but I haven't had enough time [21:27] did up_the_irons make an appearance yet? [21:27] mercutio earlier yes [21:27] well still can't reach google etc [21:27] yeah. like nothing can be done here [21:28] push button. light doesn't light up. sad. [21:28] you need to push it harder? [21:28] sometimes, the internet breaks :) [21:28] mercutio: we're test subjects :) [11:31am] :@up_the_irons: (yet i will probably not announce all our prefixes over it until further testing) [21:29] mercutio: You can't reach google from your VPS? [21:30] CaZe: yeh [21:30] [21:30] zsh 3008 # curl -v http://www.google.com/ [21:30] * About to connect() to www.google.com port 80 (#0) [21:30] * Trying 74.125.224.209... [21:30] * Connection timed out [21:30] * Trying 74.125.224.212... [21:30] * Connection timed out [21:30] * Trying 74.125.224.211... [21:31] --- 74.125.224.209 ping statistics --- [21:31] 8 packets transmitted, 8 packets received, 0.0% packet loss [21:31] round-trip min/avg/max/std-dev = 0.687/0.801/1.012/0.093 ms [21:31] ping works [21:31] try curl [21:32] lynx works for me. [21:32] it seems that i actually can curl google now [21:32] Don't have curl. [21:32] through ntt, it seems [21:32] * Timeout [21:32] * Trying 74.125.224.210... [21:32] * Timeout [21:32] * Trying 2001:4860:4007:801::1013... [21:32] * connected [21:32] * Connected to www.google.com (2001:4860:4007:801::1013) port 80 (#0) [21:32] yousure [21:32] my curl fell back on ipv6 [21:33] yeah, i'm sure [21:33] im bouncing all over the place though [21:33] hong kong, japan [21:33] sg.bb? [21:34] singapore [21:37] ping google.com [21:37] PING google.com (114.120.192.59) 56(84) bytes of data. [21:37] 64 bytes from 114.120.192.59: icmp_seq=1 ttl=56 time=278 ms [21:37] icmp is working [21:38] hk > jp > sg [21:38] less than optimal routing for sure [21:41] Arpnetworks DNS is resolving google to : 114.120.192.35 currently [21:41] try google public dns for better results [21:41] 0.841ms for 74.125.224.209 [21:42] couldn't reach googles public dns:es for a good while [21:42] NTT transit related? [21:43] mnathani: most likely [21:43] i raelly wondre if up_the_irons doesn't know [21:44] does anyone have his cellphone number and can sms him? [21:44] 3 hours soon [21:44] cos the simple solutoin would be to stop advertising to ntt [21:44] temp [21:45] mercutio: would seem starnge since he said he enabled ntt on irc. wouldn't you monitor such a change? [21:45] jberg: maybe he went back to dc? [21:45] *** thestereobus has joined #arpnetworks [21:45] i dunno [21:45] I can't imagine he is in a state of both "not knowing this is happening" and "can do something about this" [21:45] you'd think so yes jbergstroem but i'm just looking at what the situation is [21:45] either, sure. but not both. [21:46] he's pretty good about monitoring [21:46] hi folks - anyone else having issues with outgoing connections on port 25? [21:46] thestereobus - yes, some general network issues [21:46] thestereobus: most stuff will be problematic if you're in 174.136.96.0/20 [21:46] okay, thanks [21:46] yup I am [21:48] hey [21:48] it's working for me now [21:49] well to google [21:49] yeah, same here. but not most other things [21:49] oh right [21:49] been like that for the past hour [21:49] still broken to steadfast [21:49] a looking glass on ARP networks would be nice right about now [21:50] mnathani: it wouldn't actually help at all [21:50] but i have bgp to arp [21:50] and could setup lg [21:50] do you see ntt as preferred for any destinations? [21:50] his is getting weirder, my other ip works [21:50] mnathani: nope [21:50] ntt is in the path for only one subnet [21:50] it's all incoming [21:51] i misplaced their as but i checked before [21:51] flags destination gateway lpref med aspath origin [21:51] *> 103.14.194.0/24 174.136.111.233 100 0 25795 25795 25795 25795 9304 55821 9299 2914 2516 9658 58677 i [21:51] that's the only route with ntt in path [21:52] and that's not direct connect to ntt even [21:53] Destination: 206.162.240.0/20 >> 4436 25973 25795 I [21:53] from: http://www.us.ntt.net/support/looking-glass/ [21:54] 4436 is intermediate [21:54] mercutio: you have your own AS ? [21:54] that's via mzima/nlayer? [21:54] mnathani: yeh, why's that? [22:02] so, since we passed 3 hours - is it possible to somehow get reach of him and at least verify that he/someone is working on it? [22:02] have you emailed support@ [22:03] 2h 30min ago [22:03] he probably has access to that on a smartphone or pc of some kind [22:03] i was hoping someone had his cell number [22:05] so there's trouble with that announcement? i'll investigate [22:05] wb :) [22:06] up_the_irons: icmp works, acts like stateful filtering [22:06] with only seeing one side [22:07] quick fix: I stopped the announcement of 174.136.96.0/20 [22:07] up_the_irons: cool [22:08] yeah working now [22:08] my dead connections just came back up [22:08] confirmed. [22:08] up_the_irons: with debugging, arp couldn't see return traffic from steadfast, even when arp created connection.. [22:08] i think the issue is: while I redistributed connected routes into bgp so the new router sees all the customer vlan's, the routes learned via customer bgp sessions are not seen [22:08] where steadfast was routing back via ntt [22:09] hmm [22:09] how come ping worked then? [22:09] mercutio: yeah, that makes sense [22:09] mercutio: no idea, that's weird, ping shouldn't have worked [22:09] that's what made me thing stateful issue [22:10] think [22:10] dig @208.79.88.7 google.com is returning some strange addresses in 114.120.192.0/24 [22:10] singapore [22:10] mnath [22:11] that's prob cos google detected an issue [22:11] like some of fancy thing [22:13] it'll prob fix itself after some time [22:13] yeah, that happened ~2h in [22:30] does anyone know what mobo the dedicated servers are running on? guessing its one of these? http://www.supermicro.com/products/nfo/xeon_3400.cfm?pg=MOBO [22:30] jbergstroem: X9SCD-F [22:31] up_the_irons: thanks. [22:34] up_the_irons: you mention 2 nics on your website (and up to 4 ddrives), but it only seems to be 1 dedicated and 2x6G SATA's. Do you add extra nics/controller cards? [22:36] jbergstroem: i have another MB for >2 drives, but it is functionaly identical to the X9SCD-F [22:36] in fact, it is the X9SCL-F [22:36] mercutio: can u check your routing now? i believe i have fixed the customer route propagation within my ibgp. i am announcing the /20 to NTT again [22:37] ok [22:37] still fucked [22:38] take out the /20 again? [22:38] (fucked here too) [22:39] hmm.. wth [22:39] What's the affected network? [22:39] 174.136.96.0/20 [22:39] a /20 is 16 wide [22:39] removed announcement [22:39] so 96 to 115 [22:39] err [22:39] i can't do maths :/ [22:40] 96 to 111 i think [22:40] I'm in there, and I don't have any problems. [22:41] I just do IRC and my house, though. [22:41] My connection to my house has been a lot more stable than it was this afternoon. [23:10] *** mohaslan has quit IRC (Ping timeout: 245 seconds) [23:24] *** mohaslan has joined #arpnetworks [23:32] *** mohaslan has quit IRC (Ping timeout: 240 seconds)