#arpnetworks 2014-02-06,Thu

brycec

google

mercutio

dundeadly.org

brycec

vps[13].cobryce.com

mercutio

i did google yesterday
asinetech.com is via he.net
(spelt correctly)

brycec

(as is vps1. vps3 is on ARP)

[14:27]

mercutio

bryce: that won't help
i was seeing what goes over ntt :/
oh what
you are on tunnel

[14:27]

brycec

huh?

mercutio

it goes via he.net to you too bryce

brycec

mercutio: to which, vps1? I already said it's on he.net.
(vps3 is native on ARP.)

mercutio

oh right
vps1

brycec

<-- creative VPS naming

mercutio

i thought you said vps 1 to 3 are on arp
yeh ok
yeh i know right
you should see my old naming
i had sj.meh.net.nz, la.meh.net.nz, chi.meh.net.z, da.meh.net.nz, uk.meh.net.nz, am.meh.net.nz
etc
but then what happens if you have more than one in la :/
so i have arp.meh.net.nz now :)
the old la.meh.net.nz was on budgetvm, who had heaps of ddos issues

brycec

lol I thought you were using meh.net.nz as a dummy

mercutio

now i should really cull it from dns
haha

brycec

(but then I looked it up)

mercutio

i have the.net.net too
oops
the.net.nz
but i'm not using it for much atm
argh that reminds me i have to update my nameservers :/
is anyone following the debian upstart vs systemd debate?

mike-burns

is there a ML I should look on to see this?
Either way I'll be annoyed; I just want BSD's init.

[14:32]

brycec

I value my sanity... That's why I listened to the Bill Nye/Ken Ham "creationism" "debate".

mercutio

there's many mailing lists i think
but you shouldn't
it's been going on for months
it's like watching politics on tv :/

brycec

srsly

mercutio

you may be able to do it, but it's much easier if someone else watches it for you and gives you an overview
unless you like watching grown men bickering between each other

mike-burns

Well, what's the overview?

http://aceattorney.sparklin.org/jeu.php?id_proces=57684

mercutio

well they've had months of debate, and have finally voted on... further discussion
http://www.phoronix.com/scan.php?page=news_item&px=MTU5NDk
i pasted some url to a humurous video outlining various things in it though

which at least helped me understand in more deatil
i hope they go with systemd myself

[14:34]

staticsafe

same

[14:35]

mercutio

i don't really like systemd
but i'd much rather see something that everyone uses

[14:35]

mike-burns

It looks like the upstart license is incompatible?

mercutio

mike-burns: but there are upstart people on the board
the upstart license is .. complicated.

mike-burns

https://wiki.debian.org/Debate/initsystem/upstart - the con is confusing.

brycec

I agree with3 mercutio

mercutio

but it's like the fsf license i think
where you have to assign copyright or something

mike-burns

Does anyone use Debian/BSD?
GNU userland and FreeBSD kernel, I guess?

mercutio

nope

brycec

lol nope!

mike-burns

That's what I thought.

m0unds

i grew to like systemd when i messed with arch

mercutio

i have tried ferebsd 10 now
m0unds: how do you do /etc/rc.local?

m0unds

eh?

brycec

Never really understood that... If you want the FreeBSD kernel, just run FreeBSD. If you're soooo attached to Debian-isms, you have personal issues :p

mercutio

to have some local stuff executed after all the systemd crap :/

m0unds

oh, no idea. i used just 100% systemd crap

mercutio

i lvoe how much faster pacman in arch is than apt-get in debian/ubuntu

brycec

mercutio: Write a systemd service :p

m0unds

yea, it's super fast

mike-burns

I never understood GNU userland+FreeBSD kernel: the Linux kernel has tons more hardware support, and GNU userland is awful.

m0unds

systemd is really fast to init too

mercutio

i'm using arch now :)

m0unds

i don't really like the system journal vs discrete logs

brycec

mike-burns: except when it comes to very specific things, like ZFS :p

m0unds

but it doesn't bother me /that/ much

mercutio

i'm using ssd
but yeah it boots pretty quick

mike-burns

brycec: Right, true; and then I'd just use FreeBSD.

brycec

I hear that, m0unds
mike-burns: which is precisely what I'm doing too :p

mike-burns

I would, however, use Linux + BSD userland.

mercutio

for some reason freebsd is insanely slow to boot

brycec

-ENOTENOUGHRICERS

[14:40]

m0unds

i have it on an awful little amd netbook (arch) and it boots in ~15 sec w/a 5400rpm disk

[14:40]

mercutio

but really ther'es not a huge diff between ubuntu and arch boot times
but freebsd seems to take about 10 times as long to boot
i imagine it's not doing enough parallel starting etc
how did you time it m0unds?

[14:40]

m0unds

not sure how long it takes to start freebsd; i don't really reboot the systems i have that run it

brycec

I think that during the migration, Arch had a systemd service that just executed /etc/rc.local too. Easy enough to do. 14:36:50 < mercutio> m0unds: how do you do /etc/rc.local?

m0unds

ah, ok - i tested arch after it was already in use
i'm not a linux desktop person

brycec

And you could always just use cron + @reboot

m0unds

i timed it after post completed - post is 3 seconds

mercutio

ok
i'm about 1 second to grub

m0unds

something like that anyway

mercutio

about 5 seconds booting
i just booted my other pc
it's uefi

m0unds

my win8 box is to the login screen in 5-8 seconds
ssd, uefi

mercutio

yeah this is ssd/uefi

brycec

brycec remembers when...

mercutio

dual boot windows/linux
i also don't boot into X :/

m0unds

yeah, i have arch on a second disk in that box, but arch is on a platter drive

mercutio

i just login and type startx

brycec

brycec too

m0unds

yeah, same w/the little laptop i have

mercutio

i hate lightdm

m0unds

desktop arch install boots to x because i don't care
rarely ever boot it anyway

mercutio

i reckoen if soemthingshould be optimised it's video card
uhh getting the resolution thingy
[drm:drm_edid_block_valid] *ERROR* EDID checksum is invalid, remainder is 128
like crap like that in dmesg

[14:44]

m0unds

yuck

[14:44]

mercutio

ikr
but it seems to get the data ok
it just seems to slow it down

m0unds

huh.

mercutio

radeon 0000:01:00.0: DVI-I-1: Ignoring invalid EDID block 1.

m0unds

my mbp takes ~20 seconds to boot from power button w/an SSD

mercutio

i think it's cos i only using one monitor
m0unds: wow
my hackinstosh boots faster than that :/
are you using mavericks?

brycec

Ooh burn

[14:46]

m0unds

yep, mavericks
has lots of issues

[14:46]

mercutio

weird
windows 8 definitely boots faster than macos though

[14:46]

m0unds

my wife's mba boots in about the same time, but she's still on mountain lion
2013 gen (haswell) mba

[14:47]

mercutio

oh
i didn't know they did haswell on lion
i don't hate macos
i just prefer linux and ion

[14:47]

m0unds

i like it, there's just lots of improvements they could make to ux

mercutio

i like that it installs zsh by default

m0unds

i had to teach myself to use expose instead of cmd tab

mercutio

and ssh

m0unds

because i kept minimizing windows and it's annoying to go to the window menu to find what i'm trying to switch back to
yea, that was why i went with a macbook - better specs than an ultrabook, lots of tools i use on a regular basis baked in
2.9GHz i7, 8gb ram, 200-someodd gb ssd

staticsafe

SSDs sure are great

m0unds

yea, price/gig is coming down a lot too

staticsafe

i got one for my desktop

mercutio

you think that now
wait until they get 1/10th of the latency

staticsafe

256GB Samsung 840 EVO

mercutio

there's some room for improvement
i think the first improvements are coming with parallel requests and more direct access

m0unds

the only thing that sucks about ssds is having to use machines that don't have one

mercutio

bringing latency down
m0unds: haha yeh

staticsafe

best part is no noise tbh

mercutio

taht's why i like the idea of these hybrids
but would never want to use one myself

staticsafe

mechanical drives get noisy

mercutio

yeah i dunno why some drives are much noisier than others
but i kept wanting to have raid at home
but having nowhere to stick it

m0unds

yea, my env at work isn't quiet enough to notice disks

mercutio

it's pretty easy to noitce 3 or 4 disks

staticsafe

yep

mercutio

i used to use 4 disk raid at home, it's kind of annoying/intrusive
but it was pretty fast
so now i'm doing raid on ssd with small disks
and just not storing too much at home
with one backup hard-disk
so at least my important data can be on raid
one cool thing about ssd's is it's easier to do backups
because it forces people not to store too much data in one place

staticsafe

heh true

[14:55]

m0unds

yeah, makes it a lot easier to use time machine
hahaha

[14:55]

mercutio

and i have some files i access remotely
and they're on hdd raid far away from me
other thing i noticed is 2.5" hard-disks tend to be quieter

[14:55]

staticsafe

indeed

mercutio

so i was thinking about raiding 2.5" hard-disks

m0unds

like sas disks?

mercutio

nah

m0unds

or like notebook disks

mercutio

like 7.2k laptop hard-disks
they also have high latency often though

m0unds

and higher than avg failure rates

mercutio

yeah

m0unds

if they're on 24/7

mercutio

hence i only thought about it

m0unds

yeah, haha

mercutio

you can get 7mm ones
the same size as ssd's
it's insane when you think about it
sas disks aren't normally that thin

m0unds

all that mechanical stuff in that small of a chassis
pretty wacky

https://duckduckgo.com
https://duckduckgo.com/goodies

mercutio

ssd's are actually in oversized cases atm
even when they 7mm ;/

[14:59]

m0unds

yea, they're tiny

[14:59]

mercutio

if you see pictures of people who have opened them up

[14:59]

m0unds

those little msata ones are funny

[15:00]

mercutio

yeah

[15:00]

m0unds

http://ts2.mm.bing.net/th?id=H.4733093869193817&pid=1.7
err
wasn't sure if that link would work but it does

[15:00]

mercutio

it said bing
i assumed it was a mistaake

[15:01]

brycec

haha

[15:01]

mercutio

haha yeh
was that mastaa
http://cdn.pcper.com/files/imagecache/article_max_width/review/2013-07-25/130721-132916-7.65.jpg
ahh yeh it is
but see even the normal ssd's you buy are smalleer than the case

[15:01]

brycec

shit man, that's 1TB...
yep

[15:02]

mercutio

yeah they're hell expensive
but out there
i'd go for 3x500gb in raid 5
over 1tb
i'm doing 3x120 in raid10 and raidz

[15:03]

brycec

mercutio: I'd just thought "gee, look at all that room if they wanted to stuff it with more flash, make it a 1TB... oh, it already is"

[15:04]

mercutio

haha
i think 500gb is more popular

[15:06]

m0unds

m0unds doesn't use google :P
typically use duckduckgo, but image searching goes to bing via !bimages bang syntax @ ddg

[15:07]

mercutio

duckduckgo?

[15:07]

m0unds

[15:08]

mercutio

it's on amazon
i wouldn't count on it not being tracked

[15:08]

m0unds

i wouldn't suggest that it being on ec2 means it's being tracked

mercutio

it may be

m0unds

i doubt it

mercutio

it's shared storage
by a company that is into tracking

m0unds

at any rate
not why i use it

mercutio

ahh ok

m0unds

https://duck.co/help/company/architecture
http://highscalability.com/blog/2013/1/28/duckduckgo-architecture-1-million-deep-searches-a-day-and-gr.html if you're curious

[15:14]

mercutio

that is interetsing
dunno why they use amazon though :/

[15:19]

m0unds

why wouldn't they?

[15:20]

mercutio

low network speeds
bad routing
high cost
low performance

[15:20]

m0unds

hasn't been my experience at all with aws
well architected stuff will run well on aws

[15:20]

mercutio

maybe network speeds are good for you, but id on't get good speeds to amazon singapore, amazon east coast, amazon ireland?
for some reason lots of stuff is on amazon east coast, and lots of ti tends to be slow

[15:21]

m0unds

that's the default region, iirc

mercutio

and benchmarks clearly show the performance is worse than real machines

m0unds

but, like i said: 16:20:01 m0unds | well architected stuff will run well on aws
if you have global reach, architect like you have global reach
not like you have users in the US and nowhere else

mercutio

i had a friend with amazon, and his disk performance and cpu performance were shocking

m0unds

was he using ebs?

mercutio

they throttle disk i/o apparently
no idea
he had a local hard-disk
and ubuntu

m0unds

if io sucked, it was probably ebs

mercutio

and ssh was sluggish

m0unds

ebs isn't intended to be used like that, but lots of people treat ec2 compute instances like VPS'

mercutio

oh ok
yeah he was treating it like a vps

m0unds

EBS is like network storage
it's not actually in the host
so IOPS are generally pretty awful

mercutio

he did get some boost by upgrading to higher plan
i told him he should just get vps :/

[15:24]

m0unds

yea, if your app isn't gonna need to scale a ton, it makes more sense to get some semblence of dedicated resources

[15:25]

mercutio

lots of people that want to use amazon dont' seme to consider other options

[15:25]

m0unds

yea, they have a lot of mindshare
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonEBS.html

[15:25]

***

hive-mind has quit IRC (Ping timeout: 264 seconds)

[15:26]

mercutio

With Amazon EBS provisioned IOPS (input/output operations per second) volumes, you can provision a specific level of I/O performance, up to 4000 IOPS per volume.
he was way under 4000 iops
i'd estimate more like 20 to 30

[15:26]

m0unds

micro instance maybe?
dunno

[15:26]

mercutio

yeah i think it was micro

m0unds

at any rate, the point is that storage is independent of the compute instance

mercutio

and he went to small
it was unsable for casual use
it had 768mb of ram i think

m0unds

mercutio

and he went to 1.7gb?

***

hive-mind has joined #arpnetworks

mercutio

not sure on numbers exact, but it was werid numbers :/
that said i'm on 768mb on arp
1.7 seems to beee small
where's micro
oh 0.615gb

toddf

so I hope this is not too far off topic, but I think its worth mentioning .. arpnetworks is the best vps hosting provider I've encountered to date. yet, anybody who aims to be highly available knows you can't put all your eggs in one basket. thus, I've found https://my.iniz.com/aff.php?aff=289 as another basket to put some eggs in. (yes thats a referral link). can't beat roughly $20 USD/year for 256mb/250gb xfer/10gb disk kvm vps ...
... .. but they've no serial console, no secure vnc option, no ssh menu option, no /48 of IPv6 (you get 1 address). fwiw.

[15:35]

mercutio

solusvm?

toddf

I've seen that term. what exactly does it mean?

m0unds

vps management thingie

mercutio

it's a control panel with security issues
and haphazard development
most vps providers that aren't doing custom stuff are using it

m0unds

it can only assign multiple single ipv6 addrs rather than contiguous blocks, iirc

[15:37]

mercutio

it's vnc is exposed to the world as well
and they use centos
and they've had actual security issues befeore
and their mysql database is messy
but it's like $10/month per server
so it's "cheap" for the provider
like i imagine up_the_irons has put MANY hours into doing his control panel

[15:37]

toddf

heh. like I said 'best'. there are many security things at arpnetworks I'm happy with.
one switch taking out all my vps systems for the better part of today makes me want _something_ to show signs of life elsewhere.

[15:38]

mercutio

yeah i just recommend to stay away from solusvm if you can if you care about your data etc

m0unds

toddf: yeah, gotcha

mercutio

you seem to have more network problems than most toddf

m0unds

i don't run any critical stuff anymore, just personal crap mostly

toddf

mercutio: I am on the test 1gbps ports, granted

mercutio

maybe you should shift back :)

toddf

I consider it a kick in the pants that all eggs in one basket is bad period.

m0unds

toddf: did you get your net stuff from earlier sorted out?

toddf

m0unds: yes, up_the_irons kicked the switch that had all the 1gbps ports and it booted up fine
mercutio: 'if I care about my data' aka .. solusvm is like putting a bitcoin wallet on a windows server on the net, expect it to be r00ted soon? ;-)

mercutio

kind of

m0unds

haha, windows is reasonably secure if you're not a dummy

mercutio

i'm not aware of anything happeniing yet

m0unds

not since the last big exploit thing last year?

mercutio

but basically it's reasonably likely that in the not too distant future someone will find an exploit for solusvm that gives them root
and can go into all servers, or put trojans in servers automatically
and reboot :/
but the same as any problem
if you're not a normal target, then you'll probably find them just trying to doos or irc fromy our host if they get into it

[15:42]

toddf

mercutio: I'm planning on having 2ndary dns and perhaps some mx traffic but .. humm .. if only secure vnc or spice were possible so I could have secure access to console I'd just setup cryptoraid and require me to enter the password everytime I rebooted

[15:44]

mercutio

hmm
why don't you just run backup at home?

[15:44]

toddf

I'm trying to get servers away from my home office. at least temporarily. need to re-org and redo.

[15:45]

mercutio

ahh
well it up to you :)
i'm hestitant about dns

[15:45]

toddf

well, I have home business, but if any of my 3 public ip systems reboots, I have to reboot the cablemodem and ping a v4 address for them all to have internet access.
that's not reliable for hosting ;-(

mercutio

beacues if someone exploits your dns
then they can put huge stupid long ttl''sgoing to wrong locations

m0unds

yea, that's been happening more and more

mercutio

but again if you're not a target that is unlikely to happen
maybe i just paranoid :)

m0unds

well, what sucks is that it's hard to know whether you're gonna be a big target or not

toddf

I think you've made some very valid points.

m0unds

like that dude who had a short twitter id - personally, no one cared about him or his blogs or work or whatever

mercutio

yeah i just think people should know the risks
it's ok to know the risks and still take risks

toddf

I'll just not put my bitcoin exchange reflector on the iniz vps then .. ;-)

mercutio

m0unds: oh yeh that was scary
godaddy ftw

m0unds

yeah, hahaha
but it's one of those risks that you juts don't really directly consider til you read about it

http://news.cnet.com/2100-1023-236181.html

mercutio

oh i'd considered it many times
just like stealing bgp routes
it's not like it's a new problem
it probably doesn't help that i was aware that phone hacking was big in years gone by

[15:48]

m0unds

new vector for a problem

[15:48]

mercutio

and that often that could involve things like "social engineering"

[15:49]

m0unds

well, that part of it is 30 years in the making

[15:49]

mercutio

beforep eople really had security policies
didn't hotmail.com's dns expire once?
and someone else paid it?
http://news.cnet.com/2100-1023-234907.html
ok i didn't reailse it was in '99
now i feel old
haha
read the comment

[15:49]

toddf

yes hotmails dns did expire and the check for reimbursement from hotmail got auctioned off for charity iirc

[15:52]

mercutio

oh so they did reimburse

[15:53]

brycec

orly? lolol

[16:02]

mercutio

$500
still tyring to find out how much the auction went for

[16:03]

m0unds

[16:05]

mercutio

ahh $7100
quite a bit

[16:06]

m0unds

yea

[16:06]

mercutio

and i assume that was worth more in 99
oh '99 was before the crash?

[16:06]

brycec

Yes. And when Hotmail was still on FreeBSD

mercutio

hoh
windows 2000 eww

m0unds

https://www.wolframalpha.com/input/?i=%247100+1999+dollars+today
(!wolfram bang syntax via ddg, haha)

brycec

@wa $7100 1999 dollars today
:P

BryceBot

$7100 (1999 US dollars) in 2014;$10120 (2014 US dollars), (based on Consumer Price Index);2.39% per year;42.56%

brycec

oh btw
@ddg cats

BryceBot

Cats: Together with BZN, they were key figures of what came to be called the Palingsound, an umbrella for artists residing in Volendam.
[Related] Cat Meanings http://duckduckgo.com/d/Cat?kp=%2D1
[Related] Cats (musical), an Andrew Lloyd Webber musical http://duckduckgo.com/Cats_(musical)?kp=%2D1

m0unds

hah

mercutio

oh that's damn cool

[16:09]

brycec

it's (@ddg) surprisingly limited :)

[16:09]

mercutio

@wa $7100 1929 dollars today

BryceBot

$7100 (1929 US dollars) in 2014;$98270 (2014 US dollars), (based on Consumer Price Index);3.14% per year;13.84 ×

brycec

@ddg how do I banana?

BryceBot

brycec: I'm sorry, the DDG Zero-Click API returned no results. As this is a Zero-click Info API, most deep queries (non topic names) will be blank. Quite frankly, I suck cocks.

m0unds

hahahah

[16:11]

brycec

It's worth pointing out that it was passport.com that expired, not hotmail.com

[16:12]

m0unds

i wonder how long ago they phased that out
the live.com stuff still exists, and it's mostly gone

[16:19]

up_the_irons

mercutio: AS 11799 is The BGP Collective, which we turned up several months ago

[16:32]

m0unds

such a funny name
haha

[16:33]

staticsafe

is the BGP Collective anything like the Borg collective?
:)

[16:33]

m0unds

yeah, lol

[16:33]

staticsafe

RESISTANCE IS FUTILE. YOU WILL PEER WITH US.

[16:34]

m0unds

we will...add your network distinctiveness to our own?

[16:35]

up_the_irons

mercutio: yeah, there was a reason I put in TONS AND TONS of hours into my own control pane, which is to provide something nobody else has. i had a theory that people can smell a copy-cat, and I didn't want to be that

[16:35]

***

jpalmer has joined #arpnetworks

[16:38]

brycec

It may not be "pretty" or shiny, but ARP's control panel is definitely useful and unique. :)

[16:38]

staticsafe

[16:39]

up_the_irons

brycec: staticsafe : ty!!!
:)

[16:42]

brycec

It definitely feels "better" to have a homegrown control panel. You're not "some guy" who just bought a license for SolusVM or whatever and is running this in his spare time. You're comitted and you really do work on it.
go up_the_irons !

[16:43]

up_the_irons

brycec: yeah that's exactly what I was thinking too. i didn't want to feel like a reseller.
one day i want to migrate it to a nice Bootstrap admin template, then it'll have some polish too!

[16:44]

...... (idle for 27mn)

***

meingtsla has quit IRC (Quit: Leaving)
meingtsla has joined #arpnetworks

[17:12]

..... (idle for 21mn)

m0unds

http://imgur.com/SNq98Rx

[17:33]

mercutio

up_the_irons: oh, i thikn you had direct routes before?
maybe it's part of the migiration
i think it's going via the new router now
brycec: i have more faith in a bad cp homegrown than the average solusvm host
at least they must know what they're doing a bit to get the cp going
for a moment once i thought it'd be cool to try making my own cp and try and sell it like solusvm
then i remembered who the customer base would be

[17:36]

m0unds

lol

[17:46]

mercutio

i would like to see things done better everywhere
but it's complicated
anyway, it's the support system needing top level posting, which is my biggest gripe, rather than anything about cp or anything
but that's not a huge deal

[17:47]

............ (idle for 59mn)

up_the_irons

mercutio: direct routes b4?

[18:47]

............. (idle for 1h3mn)

mercutio

i don't remember seeing that other ip in there
i think it just went via s1?
to any2ix
maybe ip just changed or reverse changed and so looks diff

[19:50]

............................ (idle for 2h15mn)

up_the_irons

mercutio: ASN 11799 also has Any2 peers, some of which I don't have, so that's where that hop comes from

[22:05]

..... (idle for 24mn)

mercutio

ahh ok
well at least it's working again

[22:29]

(there's a general ntt issue effecting me, but not from arp, cos arp is going via any2ix)
forward path is via ntt and seems fine.
weirdly arp seems fine weird
oh it came right anyhway

[22:37]

brycec

*affecting

[22:40]

mercutio

heh

[22:41]

brycec

brycec tips is grammar nazi hat

[22:42]

mercutio

s/is/his/

[22:42]

BryceBot

>>> brycec tips his grammar nazi hat

[22:42]

brycec

bah typo

[22:45]

acf_

seems like there is some sort of issue between Verizon and NTT
http://paste.unixcube.org/k/1d1fd7

[22:50]

mercutio

i'm not going verizon<->at&t
and sitll hitting issues
i think it's a ntt issue
and is it still happening

acf_

NTT says no

mercutio

i've seen multiple different ntt issues recently

acf_

"We are not seeing any packet loss within our network."

mercutio

haha

acf_

it sure looks like it though

mercutio

on diff providers, at diff times
it may be something like link aggregation with one link having issues

acf_

yeah. it sure is annoying though

[22:52]

mercutio

well that traceroute suggests reverse path issue
can you get the reverse path traceroute?
basically because it goes 3.7% to 0 to 3.6%
to 3.2%
the fact it was fine from one point in the middle, and the numbers are diff enough suggetss enough samples to suggest it's sending in other direction being problematic
and like i had find traces to arp over ntt with any2ix return path
but some bad routes to other destinations coming back over ntt for at least one of them
(hard to check reverse path with random destinations)

[22:52]

acf_

the reverse: http://paste.unixcube.org/k/67f9a6

mercutio

hmm
may need more traces

acf_

also, to ntt.net from Verizon: http://paste.unixcube.org/k/3f3b0

mercutio

err longer trace
but it gusggests that it's fine when it comes into ntt at least

acf_

looks like the return path changes in Dallas