In preparation for an upgrade to my VPS, is it possible to request an offline image of my VPS so that in the event of me screwing up the upgrade I can just redeploy the working image?
And I can schedule an outage in order to take the image to ensure it will be a working/bootable image.
mikeputnam: never heard that request before, worst case scenario I bet you could arrange for equivalent disk space for a month (via powerups) as a separate cloned disk of sorts
Perhaps I am misunderstanding the virtualization.  I assumed it was not unlike VMWare/Virtualbox/qemu where you can just shutdown the VM, copy the disk image file, and boot from it later if you hose up the original.
you can indeed do that
the question is, does arpnetworks have disk space you are not paying for to use to have your backup image during your upgrade and/or do they wish to provide such a service?
I simply stated the financially incentive way to make it worth their while above.
I make no representation as to what they will or will not do only what I read on their website and what I understand about running a business myself.
if I were being asked to do something akin to the above and I were in charge of a business such as arpnetworks, I might think 'if you want to have a snapshot service with using extra disk temporarily, lets figure out a way to make it available to customers where they get what they want but we are compensated for such service' ..
given that disk space does actually cost and if you use it long term it will cost long term, the grey area is .. what happens for short term use .. perhaps some sortof discount could be negotiated but who knows I'm not staff just shootin the breeze with my random business hat on that may or may not fit this venture ;-)
key concept being .. if one does it for one person without fee, suddenly everyone will be (potentially) clamoring for it.
up_the_irons has propositioned an rsync repository for fee for backup storage locally that would not count towards bandwidth usage .. I can't see how a 'copy' of a vm would be any different if it were an official service.
what you work out between him and you in private is of course between him and you ;-)
hopefully that clarifies w/out offending you on my thoughts ;-)
The summary of that tome is: Is it technologically possible? Probably, after all it is KVM/QEMU behind the scenes. But snapshots take disk space, and disk space isn't free.
($ACVrtainOtherVPSProvider makes the above pretty easy to do through their web ui, but ultimately it's apples to oranges)
technically yes. no question. one can use -snapshots and/or 'savevm snapshotname' and then 'loadvm snapshotname' presuming the backing store is qcow2 format. all is possible. question is, does arpnetworks libvirt have instrumentation to do so and/or manually would it be worth arpnetworks time to do so. I can see a simple one time fee for a snapshot that stuck around for a fixed amount of time, presumably readonly or something. I'll try to stop ...
... blathering now.
toddf: I'm trying to give the guy a simple answer :P
hmm
v6 seems not to be working
traceroute6 -n google.com
traceroute6 to google.com (2607:f8b0:400e:c00::66) from 2607:f2f8:1800::3, 64 hops max, 12 byte packets
 1  2607:f2f8:1800::1  1.877 ms  0.961 ms  0.957 ms
 2  2607:f2f8:1800::1  0.895 ms !N  0.862 ms !N  0.896 ms !N
aye
Aye for me too
up_the_irons: up_the_irons up_the_irons up_the_irons
oh wait, that's only supposed to be 3x to summon...
v6 outages seem to be quite frequent, which is a shame
First I've noticed in the last couple of months
ipv6 is down
I reported this above
I just joined the channel.
this is the first v6 outage I've seen at arpnetworks but I may be snoozing too often to notice
it's my third
There were a long IPv6 outage the 29th of September. There were also some IPv6 issues the 23rd of October.
(just to be sure they're aware)
ipv6 just popped up again
indeed
er are routes flapping though?
im seeing some weird mtr output
I am too...
126ms O_o
Had a traceroute bounce back and forth between coresite and merit.edu... and now I've got nothing
(ohai lteo)
whee, now traceroute shows me hitting mich.net to reach ARP
dying at a merit hop for me
back again
I'll presume it is a bgp issue
I'm still not able to reach google, I get !N at the arpnetworks router itself
well I guess bGeorge's ARP v6 is working...
worked briefly for me
all my v6 traffic caught up, so I'm guessing my traffic is reaching my ARP VPS, but not getting back out. Suggests it's the ARP v6 router, to me.
oh boy look at it flap
blip!
pardon me, sir, your routes are flapping
:D
v6 routes that is right?
note v6 routing happens on a separate system than v4 routing...
aye
heh
yeh it's openbsd for ipv6
and cisco for ipv4
toddf: brycec:  Thanks for the input. From these comments I gather there isn't a pre-existing documented procedure for doing what I want to do - from the perspective of this IRC channel and it's inhabitants. I've gently perused the support web site. This leads me to opening an actual support ticket. Just doing my homework before bothering someone else.
Right-o mikeputnam, and thank you for being gentle with the documentation.
I've got 20% packetloss from my VPS to the v6 router itself. It's definitely "sick"
ha
It's not dropping packets currently, but it is giving me "no route available" for anything beyond it.
i have 100% loss
mercutio: from ARPVPS-to-ARPv6Router?
from 100% out
from ipv6 out
with mtr
(whee, just dropped a few)
although i redid mtr and then it showed first hop
The good news is I can ping other ARP Networks VPS' from my VPS
...now if only the whole Internet sat behind up_the_irons' /32
heh
maybe bgpd died
mercutio: Right. I'm just addressing the first hop on VPS IPv6 traffic, which is ARP's v6 router... since it appears to be the cause of the problem (and not an upstream riouter)
heh
i imagine the new router will do ipv6
I thought all Cisco stuff for the last decade or so did ipv6. Figured allocations and whatnot were just easier for up_the_irons to do in OpenBSD
not all of it does hardware forwarding of ipv6
but that may be the case too
I couldn't say, never did much with Cisco. Always been out of my price and practicality ranges. Easier to use opensource software and gear
i like openbsd myself
no idea why it's broken though
to be clear - OpenBSD isn't broken, just up_the_irons' box :p
heh
hmm, still broken?
yes
quite possibly the problem is not on their side
rpaulo: if I'm losing packets pinging ARP's router from an ARP VPS, then it's ARP's problem
I can ping the first hop router just fine
2607:f2f8:a528::1
rpaulo: I've been sporadically dropping packets with it. 30% packetloss over the last few hours
Any word from support/arpnetworks?
not yet that I have seen. sent support@ emails and no twitter notice (which usually happens once Garry is aware)
ditto, no response to my email either
I tried to call them and it goes straight to voicemail :-/
I figured someone had done that so I didn't try, I'm sure he'll be aware of what is going on if he isn't already as soon as he's capable of receiving such messages (zopim, phone, support emails, etc)
presumbably you left a VM?
no, I figured they are hammered with support requests
will leave a brief VM, just incase, so others don't think to pile on
left
thanks
rpaulo: toddf brycec : HE peer seems to have taken a dump
ohey there up_the_irons
ohai up_the_irons
up_the_irons: well that's shitty of them
indeed
(FWIW my tunneled v6 connections through their LAX endpoints have been untouched)
their peer isn't even pingable
Throw the SLA at them!
brycec: ^^ That doesn't seem to have worked out very well :P
haha
occaid hmm?
Talk about routes flapping (outside mtr, sitting on HE, directed at an ARP VPS) http://pastie.org/5408844
Who's on occaid?
brycec: new mtr to arp vps showing occiad hops
occaid8
i can't type today :)
orly, interesting. those hops haven't appeared in my mtr's yet
(unless they have and just didn't resolve)
brycec: http://sprunge.us/bAFK
Ah, you're not even starting on HE, so occaid would be NAC's routing
aye
i was seeing merit and HE before
just seeing HE and gblx right now
http://pastebin.com/KPsKh1Zz
That's HE -> GBLX -> Occaid -> fail
brycec: http://sprunge.us/eBhd from my Fremont linode
That shares some of the same hops as mine, including dead-ending at 450:2002
(it so happens my mtr is starting in fmt too)
:)
http://pastebin.com/d9HTSewK is HE -> GBLX -> blackhole, also with 2001:450:2002:de::2 as the last hop
I've seen GBLX, HE and Occaid IPs as the last hop over the past few hours. That makes me slightly suspicious that it's ARP's problem, not the other networks - but that is unfounded speculation.
up_the_irons: sup?
staticsafe: just working with HE on the downed peer
\o/
up_the_irons: Traffic hasn't fallen back to non-HE routes though
plett: yeah, my Trit Networks v6 transit appears to not work any longer :(
TIL ARP needs better failover testing
"We've been seeing some issues with our Any2 connection, so have the port
down while we troubleshoot it."
-- HE
HE peer is back up (although they did not confirm they are done troubleshooting their Any6 connection)
test
i'm still here
I guess the outage is over?
\o/
** RECOVERY Host Alert: ferrovax is UP **
whee
so it seems there are two IPv6 peers: HE and Trix .. HE failed, Trix failed, can't fix what upstreams don't ;-( .. hopefully in the future (my personal desire) there will be more IPv6 peers ;-)
I have not received official word from HE that their Any2 port is out of maintenance, but it does appear working for now
toddf: I've emailed one of our transits, GTT, to give us IPv6 routes; so perhaps in the next week, we'll have those also
awesome
do peers that already peer via v4 do v6 also for free or is that an entirely different peering agreement? (I am barely clueful in what it takes to do a peering agreement)
toddf: yeah, if they do v4, they'll generally always do v6 if they have v6 capability
toddf: we have about 30 or so v6 peers
s3.lax:~> bgpctl show sum | grep -v Active | wc -l
      38
give or take... ;)
hmm, so I guess he.net (being my internet access to arpnetworks mainly) failed to route through your other peers ;-(
thanks for the infoz, bbl...
toddf: no, it's more like, we had no route _back_ to you.  HE will only route through themselves.  Technically, if you were behind a network that peers with us, then that traffic would still have gone through.
and indeed there was a trickle of traffic still working
up_the_irons: do you have transit or peering with he.net?
because a random outage like that seems pretty bizzare for transit, but understandable for peering
mercutio: both
both? :)
over the same circuit?
mercutio: they provide transit over the Any2 peering network b/c they don't have any connectivity outside of One Wilshire (in LA, and we're in CoreSite down the road)
so the cost of transit, with the attitude of peering? :)
it's still considered a peer, just with full routing tables
;)
oh
HE provides free IPv6
oh is that ipv6 only?
ahh yaeh
yeah, like their other v6 products
you don't have ipv4?
through them
so yeah i suppose if one of normal transit providers does ipv6 better off
yeah, so that's why I just asked GTT about it.  Lots of carriers are behind the times (don't offer v6), but i think i read somewhere recently that they do it now, so let's see...
i need food
bbl
oh gtt is nlayer
hmm, ipv6 is still down
to some places
 2001:4428:200:db::2 for exampl
prob that he.net issue
it's on sixxs
woohoo looks like we're back up
up_the_irons: so if I understand what's been said, even if you add GTT, those of us on HE would still have been stuck because HE would never allow our traffic to route away from HE's network, and just keep pointing us at the bum node?
up_the_irons: Also, good work on getting things sorted out. Kinda wish it had been noticed/acknowledged sooner, but... if that's my only gripe with ARP, you're doing pretty good.
brycec: No, if the HE node is down, traffic would get re-route elsewhere by BGP.  Even if you're on HE, they wouldn't see a route to my network (since the peer is down) so they would send it elsewhere (perhaps their default route or another peer / transit that are propagating my routes)
brycec: about timing, I have a monitor on our IPv6 network with uptimerobot (hard to find a service that supports v6 :( )  Unfortunately, they were giving a lot of false positives lately, so I paused the monitor.  ...and of course, *then* IPv6 goes down for real.  Normally, we are much more proactive about these things.  I really need to find a reliable IPv6 monitor.  Perhaps just a VPS
somewhere else that has reliable v6 is the way to go at this point.
up_the_irons: Google's App Engine offering is on reliable ipv6
i'd rather support small business :)
maybe a RootBSD node
in general, my ipv6 has worked flawlessly
here, of course
if it breaks temporarily, file a support request
up_the_irons: This is a local co. in Madison WI that also peers w/HE http://bgp.he.net/AS16842
the network guy is also active in the local hackerspace sector67
nice:)
My Chunkhost+HE has been mostly reliable... Not perfect, but when it's smooth, it's smooth. Today:  http://i.imgur.com/ziab7.png
up_the_irons: thanks for the clarification. Was confused after what was said above and today's failure ("HE will only route through themselves."). HE and Trit were down, but what about the 36 others? ("s3.lax:~> bgpctl show sum | grep -v Active | wc -l")